Would people who had seen Jesus, those not in on the Brian switch scam, recognise him on the cross as not being JC?
It's likely that Brian was related to JC, a cousin or even a brother. Also his face would have been a bit mangled from his beating by the guards. And he crown of thorns meant that his face would be covered in blood. Making him unrecognisable.