Robert Earnshaw pulled one back for the hosts but Derby's misery was complete as Lewis Nyatanga hauled down Neil Mellor - who then fired home from the spot.
Havant and Waterlooville, part-timers from Hampshire, twice lead Liverpool but their dreams were scuppered as Yossi Benayoun rescued the reds with a hat trick to put them into the hat for the fifth round.
Truant officer Richard Pacquette fired Havant into an unlikely lead. The home side levelled through Lucas, but then fell behind again infront of the Kop.
Benayoun was took the bull by the horns and fired in a hat trick before Peter Crouch netted in the last minute to round off a less than convincing victory for the Reds.
Wolves overcame Championship rivals Watford 4-1 at Vicarage Road with former Iron Andy Keogh grabbing a brace. Coventry are also into the hat after seeing off Millwall at the Ricoh Arena.
West Brom joined them with a 3-0 away win at Peterborough, Southampton eased past 10 men Bury, while Huddersfield recorded a 1-0 win over Oldham.
Chelsea, Arsenal and Portsmouth are also into the hat after triumphs over Wigan, Newcastle and Plymouth respectively.
Middlesborough also went through despite being pushed all the way by League two strugglers Mansfield, who were unlucky to lose 2-0.
Bristol Rovers kept another fantastic cup run going, by seeing off Barnet at Underhill.