Burnley were also reduced to 10 men as Steve Caldwell and Joe Gudjonsson were dismissed - incredibly all four red cards came within an eight minute spell in the second-half.
"The ironic thing is that Mike Riley dropped his red card a couple of minutes into the game and I told him he wouldn't be needing it," said Brown.
"I'll look at it on the video and if Caleb deserves to get the red card I'll deal with it in-house. The same with Jay-Jay, but it disappoints me because it takes the shine off a good performance.
"These kind of dealings need to be kept in-house. Everyone has seen it, but I have the benefit of hindsight with DVDs from all angles and I'll keep it in-house."