The Yorkshire club decided to end Harford's reign after the 2-1 home loss to Yeovil on Saturday, which left them 22nd in League One.
Ironically, it came just 24 hours after he vowed not to walk away from Millmoor.
Knill will take charge of the first-team on a caretaker basis, working alongside fellow coach Mark Robins.
Director David Veal said: "We met with Mick and informed him that we wished to relieve him of his managerial duties.
"This is a sad time for all concerned but we felt we had no other option to take, for the time being the coaching staff will step up and take temporary charge.
"We would like to place on record our appreciation of Mick's efforts to turn around our playing fortunes and wish him well."