There is an excellent walking track between the Manly ferry wharf and the Spit Bridge at The Spit. The track takes in sections of the Sydney Harbour National Park and offers great views of Sydney Harbour. There is excellent public transport to/from the start and finish.
The walk is about 9.5km. At a leisurly pace it would take around 4 hours, although I have done it in less than 3 hours.
Although my track file goes from Manly to The Spit, I would recommend doing the walk in the reverse order. That is, start at The Spit and walk to Manly. This is mainly because there are better facilities at Manly, and you may appreciate them after 4 hours of walking.