And his side hit the headlines last season as they reached the quarter-finals of the Carling cup.
Rovers chairman John Ryan said: "I would like to thank Penney for his wonderful efforts over seven years as a player and manager.
"Despite a poor start to the season I believe we have one of the best squads in League One, together we can turn this around and go for the Championship."
Mickey Walker will act as caretaker, starting with this weekend's game against Port Vale.