If memory serves, it was not a golem lauguage but rather the language of the people who made them. Newer golems, made by other people later on, would have no way of knowing it.

Thus, the golems having a debate in Feet of Clay would be forced to use a common language for their debate - Morporkian.