Your error in thought is that you limit this type of thinking. That is, you are willing to accept mathematical beauty as an answer to many of the perplexing problems that science confronts, however you are not willing to extend this mathematical beauty to global systems in general. That is, if I may characterize your thinking process, you are focused solely at the 'particle level'.
For example, if we look at a nucleus you have no problem following the mathematical cause of nucleus to the protons and neutrons that make up that structure. Likewise, you have no problem following the cause of proton and neutrons to quarks. And, if this continues (which it appears that it should at some fundamental level), you would easily accept the consequences of that scientific inductive/deductive discovery process.
If I asked you why is there a nucleus, and why are there protons and neutrons, and why are there quarks, you would look at me, scratch your head, and proceed to explain the math that puts these together. However, if in your explanation I interrupted you, and said, "Alex, what I mean by why is not how it came together mathematically, what I mean is to ask doesn't it strike you kinda funny that all of these rules that humans can barely understand all seem to come out so perfectly that 'stuff can exist at all'?" Your answer would be your failing. Your answer would no doubt be "there is no reason why these things work out the way they do, it just happens. Maybe there are other universes, maybe other physical equations in other universes where such structures don't form and we are just consequentially in the that allows observers (aka, Strong Anthropic Principle). But, you might add, even if there are no other universes to consider, we are still in this universe with these laws purely by chance".
What your (hypothetical?) answer shows is that you are not willing to apply mathematical beauty to a higher and more grand scale. You are only interested in applying such principles inconsistently. That is, if you applied these principles consistently then you would see that mathematical principles work for whole objects (e.g., nuclei, protons, neutrons, quarks, etc) as you treat them, but they also work for any other object. I don't think you can limit the definition of an object in your pure mathematical treatment of them. That is, logical/mathematical relationships compose an object in your view, but such relationships hold for anything and everything - including all spacetime series of events for the whole Universe.
What happens if you treat the whole collection of spacetime events of the Universe as one object? You must describe it using the mathematical beauty that you described protons and neutrons. All of the components work together to produce one unifying Whole.
This concept defeats your mechanistic notion that "that EVERY phenomenon we know, upon close look has its MATHEMATICAL origin from more fundamental phenomenon". Rather, it shows that every mathematical origin is the pieces of the puzzle being snapped into place. The order of the Whole dictates the order of the components. Nothing is left purely to random chance. That entire order to that Whole I'll call the divine plan. The divine plan is mathematically at work, but you are too nearsighted to see it.
Warm regards, Harv