Mathematics (in a general sense) is closer to English than it is to any a priori of the universe. That is, to answer your question, there are as many mathematicians 'out there' as there are people who have a need or ability to speak this kind of language.
I don't think many of us would dream of an advanced alien species having spoke English in its exact grammatical form at some point in their history (assuming they didn't receive our television signals...). Likewise, we shouldn't expect an advanced alien species to speak our mathematics.
As you said, though, there are many potential mathematical axioms, and hence an 'infinite' number of mathematical languages, however all of these languages are guided by the same requirements that spawned English. Some of these requirements are haphazard, such as coming in contact with certain foreign words and bringing them into the English vocabulary, whereas some of them are 'required' in the sense that if you come in contact with snow, at some point you need to have a word in English which can refer to that word.
Thus, I suppose, it must be with any alien version of mathematics. Some of their mathematical concepts are based on their own particular evolutionary needs, such as a need to see the world in terms of lines and planes, whereas some of it comes from a limitation in the way any conscious mind can function and still work. Hence, I suppose our mathematics might be a little strange at first to any alien intelligence, but the limitations of thought would probably allow them to quickly understand what most of our mathematics is referring to (which happens even when an English speaking person comes in contact over time with a Spanish speaking person, they naturally begin to cross reference the items their words refer to in the world).
The question that still haunts us is why mathematics is so successful in describing the universe and even allowing theoretical prediction to occur, or as Eugene Wigner said, why is mathematics so unreasonably effective? But, a more fundamental question might be, why is language itself effective in describing a world? That is, why is anything intelligible unless there was order to the world that was 'out there' instead of just 'in here'. Afterall, if order is something that really is 'out there', then order of itself exists independently or co-dependently from 'things in themselves' (going back to Kant). Which then raises another question, who ordered for there to be order? The dog continues to chase its tail.