Whittingham opened the scoring with just nine minutes gone, curling in a superb effort past Mark Schwarzer from the edge of the area.
City doubled their advantage thirteen minutes later when Johnson headed in Whittingham's free-kick, after a foul on Jimmy-Floyd Hasselbaink.
The home side rarely tested Cardiff 'keeper Peter Enckelman in a lack-lustre performance.
The Bluebirds held on for the win which will see them play at Wembley for the first time since 1927.
Elsewhere, Barnsley stunned Liverpool with a 1-0 victory at Oakwell, and Manchester United were dumped out by fellow Premier League side Portsmouth at Old Trafford.