We are getting further from the issue. The issue is that you say that the rules of math exist 'out there'. Okay, let's figure what that means:
1) The formal system that comprises math (axioms, theorems, definitions, etc) isn't just a methodology practiced by mathematicians, but it also describes the workings of reality.
2) This formal system is built upon axioms (using rules of inference, definitions, etc) to construct theorems. Hence, reality itself is 'built' upon the same principle. That is, statements of reality depend on axioms, which is why statements of reality are to be considered true. If the axioms of reality are wrong, then the statements are also wrong.
3) The axioms in a formal system (e.g., math) are *not* proven, they are merely accepted as true by convention. Translating this in terms of reality, there exists a problem as to how deductive statements of reality are true if the axioms are selected by convention. Your view has no means to substantiate the claim of an axiom. That is, it is true because it is obvious to you by viewing the universe, but it is the universe's existence that we are trying to explain here by saying 'math exists'. So, you have a Catch 22. If you say 'math exists', then you are dependent on axioms being true (even though they are not proven in math but are only selected by convention based on the obviousness of how they appear (in the universe)). However, if you accept those axioms as true, then you have no reason to say they are true without injecting the very the thing you are trying to explain - that being the universe. This is a logical fallacy known as 'circulus in demonstrando' (aka circular reasoning).
You have to give account for the meaning of axioms in your model if you want to use a 'math world' as your metaphysical foundation. What my account does (and, Dick, I'll try not to say 'nananan') is translate the axioms and undefined terms used in mathematics (i.e., a formal system) into something meaningful about reality. The undefined aspect of reality that this mathematical world approximates is the 'stuff' that acts as a foundation to mathematical axioms and mathematical undefined terms. This is the only way I see possible in translating the formal system of math into an ontological model of the world (unless, of course, you can eliminate axioms, undefined terms, and terms reliant on the universe existing which I don't see as possible).
Warm regards, Harv