Complexity vs Simplicity: Thoughts from the Discovery of Heliocentrism

I am recently intrigued by a series of Youtube videos introducing how heliocentrism was discovered. There are lots of general insights about the process of scientific discovery which I found relevant to social science research as well. One of the key tension there is the competition between simplicity and complexity. I'll first briefly introduce the background on Heliocentrism, and afterward, discuss the dual between simplicity and complexity.

Background on Heliocentrism

From the perspective of a modern human being, we are very accustomed to the idea that the earth revolves around the sun. But if you think a bit more about it, it is an extremely counter-intuitive fact. In our view, the sun rises every day from one side of the earth and vanishes from the other side. It is natural to believe that the sun should revolve around the earth. In fact, this is exactly what Aristotle proposed more than two thousand years ago in his theory of the universe. This theory of geocentric explains the day-to-day empirical observations very well.

But one anomaly appeared: the retrograde behavior of planets such as Mars and Saturn. Geocentrism cannot explain why Mars sometimes revolves around Earth in one direction, sometimes in a reverse one. Believers of geocentrism thus proposed a new mechanism to resolve this issue: planets not only revolve around Earth but also revolve around another centroid with a smaller radius. To fully explain all the weird retrograde behavior of planets, the trajectory of planets ended up being extremely complex, as seen from the graph on the bottom.

Alfonso X de Castilla, the king of Spanish at that time who was passionate about astronomy too, famously said:

If I am together with the god when she created the world, I would make some suggestions on improving her design.

Despite this awareness, geocentrism was widely believed and promoted by scientists, churches, emperors and citizens. The stage was set for Copernicus. Copernicus made a simple analogy: if a tourist went to a new city, none of the locals would care about him. He should not assume himself to be the center of the city. Similarly, the universe is so large that it does not make sense to assume Earth to be the center of it. Beyond this analogy, he proposed his theory of the universe: planets revolve around the Sun, while also revolving around themselves. This theory of Heliocentrism is much simpler than the complex trajectory of the planets required by geocentrism, and it fits the empirical observations as perfectly. The acceptance of Heliocentrism, of course, was an extremely tough process that went beyond science. But from a purely scientific view, people argue that the contribution of Heliocentrism, besides the theory itself, was one of the first times humans could get rid of the subjective human-centric view of the world and embrace rationality. This is the philosophical foundation of science.

Takeaways

My first takeaway from this story is that a simple theory is usually closer to reality than a complex one, which is also the idea of the famous "Occam's razor". This is often the case in natural sciences. It seems that economists hold a similar belief when formulating theories about human behaviors. Theories like Adam Smith's invisible hand seem to have achieved this objective of having enough prediction power on reality while keeping simple. But we have to admit that the prediction produced by economic theories are by no means as accurate as those produced by theories in physics. Human behaviors are complex.

This focus on simplicity can be even more puzzling if we consider the recent rise of machine learning (ML). ML is built upon the rationale of making things as complex as possible to fit reality, as opposed to focusing on producing a "simple" model. This approach has proven to be useful. It is even imaginable that we can fit a complex ML model based on geocentrism which may predict the location of the planets as accurately as our simple model using heliocentrism. In other words, we don't need to know the "truth". A complex model with no assumptions is enough for predicting the outcomes. Isn't the prediction power what we want ultimately?

Simplicity and complexity are both viable approaches to acquiring the prediction power we want. When simplicity is hard to achieve, we pursue the alternative route of complexity. This is often the case in social sciences. (the use of probability theory in social sciences is an example). Complexity, on the contrary, is no longer hard to achieve given the modern computers and advancement in ML and statistics. Does it mean that we should give up looking for simple theories, and instead just use large and complex models to do everything? There are several limitations to this idea:

1. Complex models need lots of data for fitting. Many important real-world problems do not have much data (e.g. macro-economic problems).
2. Accurate measurement may not be possible. Constructs such as culture are difficult to be measured accurately using numbers.
3. It cannot (yet) analyze brand-new environments which do not appear in history at all.

Even though these issues may all be resolved one day with the invention of super AI, I think it is still valuable for us to pursue simple theories. After all, we still want the "truth", aren't we?