Symmetry is an important situation (some might say tool) to understand the unity and conservation of nature, but we also need asymmetry to explain the differences seen in nature.
For example, if Buriden's Language Removed stood in front of two barn stalls with hay for 2 hours and then as the sun was going down the sunlight hit the hay stalls slightly differently so that right-sided stall with a pile of hay looked more appetizing than the other hay pile (and the donkey made a decision as a result), then we would say that symmetry of the situation had been broken in favor of the right-handed stall. If we were insects that evolved while living on the full donkey (in his right-side stall), we might wonder why we live in a right-side barn stall and not the left-side barn stall. If we flew around the barn we might see that both stalls offered equal amount of hay, etc and for all practical purposes the hay stalls are perfectly symmetrical. We might conclude that the decision by the donkey was completely random. However, we might be surprised to later learn that sunlight hit the hay and that was what made the donkey make an asymmetrical decision. In essence, a whole universe is 'out there' that broke the symmetry of one event.
This is why you cannot say that the universe emerged from a purely mathematical symmetry. It is like saying that Buriden's Language Removed chose one pile of hay based on the symmetry, but this is not true. The symmetry only *underlies* the situation but it is not the 'creator' of the decision of Buriden's Language Removed to eat one particular pile of hay.
Warm regards, Harv