Youga, who is on loan from Charlton Athletic, was sent off for the second time in two games against Blackpool - having previously seen red in the 1-1 draw at Coventry City - and the Iron decided to appeal the latter of the two dismissals.
The appeal committee on Thursday decided that the red card shown by Andy D'Urso was justified.
The 22-year-old will now serve a four-match suspension, starting on Saturday against rock-bottom Queens Park Rangers.
Youga, who has made 18 appearances for The Iron this season, is likely to be replaced by Shaleum Logan at right-back, who is on loan from Manchester City.
The Charlton defender started his career at Lyon before signing for the Addicks in 2005. Youga, who was born in France, has also had loan spells at Bristol City and Bradford City.