Smith won the award for the fourth time last season, before quitting the club to join Walsall after refusing to take a pay cut.
He returned to Priestfield on loan, but joined fierce rivals Swindon Town in January.
The midfielder has refused to return the trophy as he claims he is still owed unpaid wages by the Kent club.
But a furious Scally told the Gills website: "Every year the player of the year receives trophies which are the club's property, our heirlooms.
"They have their name engraved and then they are returned the following year. Paul Smith is refusing to do so. He says that we owe him a small amount of money but we don't owe him a penny and are happy to challenge that claim.
"This coming from a man who has earned more than £1million from the club and from a man who decided to go to Walsall when told he would have to take a pay cut. To say it's childish would be disrespectful to children."