I think Galilei's saying is just mathematics admiration, to show his respect to mathematics. The question which can makes us think for days if not years is that which was first, Mathematics or Physics?

One may say that mathematics is a human made way of modeling or as we said before reasoning for physical phenomena, and by physical we mean all chemical, biological, psychological, social and ... So by this definition it is obvious that this kind of mathematics we talk about had not been existed before physics, if we consider the physics itself was existed before human.

An idea of how physics and mathematics exist |

The topic is much more complex and even weird if we study about some scientific experiments have been done showing that what we human assume as advanced mathematics or physics are natural what other animals know and feel them. For example all of us know that if we throw a ball and ask someone to run and get the ball before it hits the ground, based on the speed, angle of throw, height and perhaps the previous experiences of the one who is going to get it, and ... it is some complex behaviour to get the ball on the air.

But wait a moment, most of the dogs also can do the same, perhaps better than we humans too. Are dogs know physics and mathematics? If you ask me, every plant, animals know physics at its own level of intelligence. For a moment think about the way a cheetah hunting for a gazelle. The cheetah should know many geometry theorems and physics rules to follow a swift and agile animal like a gazelle. Do they now? Yes, they know.

We human invented mathematics just to model the motion, geometric shapes and .... this helps us to have a better understanding of a system in which a cheetah hunts for a gazelle, helps us to create devices to do the same as they do, following things or escaping from a follower.

You see this is a very deep and philosophical argument and you can't bring an exact proof which on was existed before. But it is more likely to be existed some non-human mathematics first and then our physics and then our mathematics.