Yeah, that's a good point, Merlin. Common isn't specific to humans as their language but a mixture of various languages from various races and cultures, so "man" and "woman" can be used as the words for any adult male or female, respectively, since English is what we're using for Common and those are the words it has. Racial names could be added for clarity, just like any other adjective (blond, fat, etc) - "Ask the elven woman in the lobby," rather than just "Ask the woman in the lobby."
That doesn't mean we can't invent new words for specific people, just that "man" and "woman" are the general terms for any race's adult males and females.