Torpey signed for the Imps in the summer after being released by Scunthorpe, but the fans have not taken to a player who spent six years at their local rivals.
He was booed by a small number of fans when he came off the bench during Saturday's 2-0 League Two win over Accrington Stanley.
"I heard some boos as I was about to come on and then when I got my first touch" he told the Lincolnshire Echo.
"After playing just two games for the club, I was astounded to hear it. I think it was a minority, but it's still not nice. No one wants to get booed.
"I'm not going to hide and go into my shell - I'm not that sort of person.
"I've got a year's contract here and I intend to see it through. The manager has brought me here to do a job."