The problems I can see with this are twofold: First, a drow or tiefling, or even goblin, dressed in concealing robes/clothes/armour/leather would be indistinguishable from your average joe human/halfling. You'd have to have some method of determining that, or else Garent has suddenly become the most brilliant detective in the world. Second, when a bad PC gets powerful enough, he could walk in and slaughter a town's defenders easily. That would be hard to deal with in an RP fashion.
Solutions, anyone?