Ryan Shotton was put clear by Mark Wilson five minutes later, but his low drive from the edge of the box just cleared the outside of a post.
United seemed content to soak up the pressure but came unstuck in the 41st minute. Ed Sonko raced on to a pass from Bas Savage and lobbed Joe Murphy for the opener.
Gary Hooper had seen an early shot blocked and a David Mirfin effort on 26 minutes met a similar fate.
Shotton collected a first-half caution for a foul on Grant McCann and Murphy went in the book for timewasting. Second-half cautions followed for Ian Moore, Savage and United's Sam Togwell.
The visitors were unlucky not to level three minutes after the re-start. Danny Coyne in the home goal somehow deflected over a point-blank range shot by Togwell and then tipped over a Hooper header from the subsequent corner.
There was now much more urgency from both sides, but it was Tranmere who continued to call the tune.
They doubled their advantage on 75 minutes. Shotton saw his first shot blocked by Matt Sparrow, but found the target with an acutely angled effort from the rebound.
United almost pulled a goal back a minute later but a 20-yarder by Hooper rebounded from a post.
Skipper Izzy Iriekpen was always busy for the visitors and Martyn Woolford and Togwell made their presence felt in midfield.
Kay, Shotton and Wilson, in his first home appearance while on loan from Doncaster, were prominent for Rovers.