Ian Sharps put the Millers ahead with a looped header after 17 minutes, with Jonathan Howson levelling for Leeds on the half-hour mark.
The home side regained the lead just before the half-time interval when Mark Hudson scored, though only after David Lucas had saved his initial penalty kick.
Drewe Broughton and Nick Fenton extended the lead before Enoch Showumni pulled one back for Leeds.
The draw for the third round will take place on Saturday morning, Scunthorpe are in the northern section and could face any of the following teams: Darlington, Leicester City, Tranmere Rovers, Morecambe, Bury, Rochdale or Rotherham United.