They took a 2-1 advantage into the game from the first leg, but Ben Watson's first-half strike levelled the tie on aggregate.
With no further goals - although Watson missed a penalty which would have put the visitors in the driving seat - the game went into extra-time, where it was the home side who found the extra energy.
Lee Trundle scored towards the end of the first period to put them ahead in the tie, and Michael McIndoe smashedin a free-kick to put the result beyond doubt.
City have never been in the top flight, but now stand one win away from rubbing shoulders with the likes of Liverpool and Manchester Untied.
"It is unbelievable, the scenes out there were amazing," the score Trundle told the BBC.
"To get to the play-off final is massive and to score a goal like that at that time was amazing for me."