It means far into the mountain range, which may mean higher altitudes or just far from the foothills, making access somewhat difficult and resulting in greater isolation.
It's a quirk of the English language, and not meant to imply they live under the mountains.