Nicolas Anelka struck first with a header and then Shaun Wright-Phillips was fed by Eyal Berkovic's diagonal ball and finished well to make it two-nil.
The game was only 26 minutes old when Anelka scored his second goal.
City made 10 changes at half-time and Bury pulled a goal back through Danny Swailes on 50 minutes, and took the deficit to just a single goal eight minutes later through Lee Connell.
Andy Preece was presented with an open goal to make it 3-3 on 70 minutes.
Then on seventy-nine minutes City youngster Stephen Jordan hit the winner.