Potter came through the youth team ranks at Peterborough United, after being released by Wimbledon at the age of 14, then by Millwall aged 17. Before joining Peterborough he spent a few months at the Barking Abbey Centre for Football Excellence, where he made several first-team appearances for Waltham Forest in the Isthmian League. He signed his first professional contract with Peterborough on 1 June 2007, signing a one-year deal.
He joined Kvinesdal of the Norwegian 4th tier on loan in the summer of 2007, but failed to make any appearances. In September 2007 he had a month’s loan spell at Grays Athletic. He made just one appearance, coming on as a substitute against Exeter City.
In November 2007 he made a loan move to Conference South side Havant & Waterlooville, for the rest of the season. He played in Havant’s 5–2 fourth-round FA Cup defeat to Premier League side Liverpool on 26 January 2008. Potter played a big part in Havant’s second goal to put Havant 2–1 ahead when his shot took a deflection off Martin Škrtel. For this feat, Potter was named “Player of the Round”.
In December 2007, he was offered at new two-and-a-half-year deal by Peterborough, and, on 29 January 2008, he was recalled from his loan spell at Havant & Waterlooville.
On 29 February 2008, Potter was loaned to Isthmian League Premier Division side AFC Wimbledon, for a month. He made his first appearances for the Posh in the League upon his return. In July 2008, Potter was close to signing on loan for Conference National newcomers Kettering Town. However, Peterborough initially stalled the deal because of injury worries, but he finally completed his loan move to Kettering on 1 August 2008.
In May 2009, it was revealed that Lincoln City had enquired about Potter and his team-mate Liam Hatch.