Iron manager Brian Laws conceded: “We might've lost that game in the end”.
The turning point was referee Trevor Kettle's decision to give a foul against Marcus Williams on Simon Waley in the second half.
Laws added: "Can anyone tell me why that was a penalty?.
"He told one player the penalty was for pulling on a shirt and he told someone else that it was for a push. So you make up your own mind.
"The goal gave Bury a lift and they threw caution to the wind to try and get a second goal.
"But it's all about getting through to the next round and we've got another opportunity to make sure we do."
Bury manager Chris Casper, the youngest manager in the Football League, was delighted by the resilience shown by his side.
He said: "We're not going to the replay to make up the numbers, as I want to build on this performance.
"We've just got to cut out some elementary mistakes, and if the players don't believe they can beat any team in our division or the next league then they are idiots."