With the 1.69 update, horses scale to the size of the rider, so to answer your specific question, there's no way a half giant can ride a "pony". It would look more like a Clydesdale or a large work horse.
As for the broader question, I'm unsure what will happen when a half-giant tries to ride a horse. It's possible it will work. It's also possible that it will behave badly. My suggestion is to work with a GM, and if it doesn't work, then have the GM refund you the gold.