Joe Murphy was in goal for the first half, with Joe Wilcox, Izzy Iriekpen, Milne and Marcus Williams in defence.
In midfield, Kevan Hurst played wide right, Matt Sparrow and Andy Wright were partnered in the center with Peter Winn on the left. Ben May partnered Gary Hooper in attack.
United were comfortable in the first half and the central defensive duo of Iriekpen and Milne showed great promise, commanding the back four well throughout.
It was Scunthorpe's new recruit Milne who opened the scoring with a fantastic header, connecting with Hurst's right wing corner.
The hosts looked lively going forward, but shaky at the back. Cummings, Taylor and Jones all went close, the latter with a curling free kick that Murphy just about had covered.
Winn was causing the Rotherham defence havoc down the left, and could have doubled United's lead with a fantastic curling effort. The youth product picked the ball up on the edge of the box and delightfully clipped the ball goalwards, only for it to cannon off the underside of the crossbar.
Josh Lillis came on at the interval replacing Joe Murphy, but United remained outchanged outfield for the second 45.
They could have added a second just after the restart when May found space to deliver a great ball towards Hooper, who's header was tipped against the bar by Cann.
Hayes, Forte and Morris were introduced with May, Hooper and Winn given a rest, and Morris had the best chance of the night to make it 2-0.
He was through on goal one on one with the keeper, but opted for power rather than accuracy and blazed his effort high over the crossbar.
Milne was replaced after troublesome blisters were causing him some pain, and Jake Picton came on to partner Iriekpen.
The hosts equalised through Reid and snatched the winner in the closing minutes when Ryan Taylor fired home past Josh Lillis to seal the win.
It was a disappointing second half from United, who were in control until substitutions were made.