According to Wikipedia's reference (Gigantism - Wikipedia), there are two versions of. I am assuming that if the Emperor had actual Gigantism disease, that he could not have had the second version (eunuch stage - different symptoms and no sexual glands) but the first one (Acromegalic gigantism) appears to be more commonly found occurring in middle age, not as a child. That is very rare according to this source and also from MEDLINE plus - Gigantism's covering of this topic. Extremely rare, though it does cause extreme height if contracted as a child. I have looked but cannot seem to find any sources stating what the max known height from this anomaly is. Also they did not live very long. At what ripe age is our Emperor expected to live to?

I think in this case, he may have just been very tall genetically, and was purported to be even taller, in myth stature perhaps. Most will not have ever actually seen him standing next to another individual, but either sitting (if seen at all) or on a dais of some kind, so there would be more cause for illusion. (Just my opinion)