So in the previous episode of our Progress series we have seen that standards are good for you. In fact, without standards, you would not be able to read this article at all. You would probably not even own anything that is capable of displaying this article either: the only reason the chips in your electronic device could be build, and can talk to each other, are standards.
This is also how science works: using common standards (of measurement), one theory can be based on another. This way, researchers don’t have to go and re-invent all the foundations and theories before they finally can start producing original work¹.
So old is good, right? We can build on a foundation happily ever after?
We can. Except, of course, when the foundation is flawed. Because the same principles that finally allow us to build high, also allow us to build on errors of others until our house falls down.
Take the science of Astronomy for example. For thousands of years it was nothing more than astrology — because it could not provide us with any insights more useful then that. Because, in turn, it was based on the flawed premise: the Ptolemaic Universe. It was only after Copernicus and Galilei that we started to be able to look into the actual workings of the universe, because they found the fundamental flaw in the theory of the world to that point: the Earth was not the center of the universe.
If not for the Copernicus, would Galileo have been able to to observe the laws of motion? And what about Newton — without Galileo’s insights, would he have been able to formulate the theory of gravitation at all?
On the other hand… if not for Aristotle and Ptolemy, would somebody have discovered the laws two thousand years earlier?
Like with most historical what-ifs, we’ll never know for sure. But what we do know, is, that about 2250 years ago, a man called Aristachus of Samos had already developed a very modern view of the solar system. But the geocentric model of Aristotle was viewed as a much simpler explanation for why there is no parallax — why stars don’t visibly “wobble” when the Earth moves around the Sun: in Aristotle’s model the Earth simply doesn’t move. The calculations for the positions of the planets produced results accurate enough to look right, and so the “simpler” explanation won out.
As you see, with too little information, Occam’s Razor can work both ways.
This piece of history also shows that based on the same data, you can develop different models, all of which appear perfectly functional, match the data perfectly, and all of them can be wrong to some degree. Ptolemy’s model is a perfect example of what is known today as “over-fitting”: it matches all the available data (the observable track of planets known at the time), but can not make useful predictions outside this data set. It also does not explain any of the principles involved. (Doesn’t this remind you of some modern unification theories?). The Copernican model, on the other hand, left some questions unexplained at the time (nobody had thought the stars could possibly be that far away) — but it allowed, in the end, to deduce the principles of gravitation. Which led to Einstein’s General Theory of Relativity, the moon landing, and GPS. Extended use, so to say.²
However, often enough, you don’t see which model is “more right” until later — very much later. And therefore, competing theories, competing libraries, competing standards, are good… if there are good reasons to believe that the current system they are competing with is really fundamentally inadequate and hampers development.
But even if software people finally agree on standards, even if people actually cooperate instead of developing against each other, even if we all build on the same good foundation… we are still shooting at a moving target, and it is hampering our progress.
How so? Well… see you in the next episode of the progress wars!
¹:This is why, when somebody says, “I am just as entitled to an opinion about meteorology as scientist X” he is talking BS — unless, of course, he actually spent as much time studying the theories behind meteorology as scientist X, and has no bias.
A bias might be his conviction, or it might be his boss telling him that whatever the logical answer would be, the answer that will keep his job safe, is the one that his boss wants to hear. Unfortunately, the laws of physics are not political, only the people studying them are. But this is a different topic.
²: An analogy would be an open-source software library: there, I can learn from something other people already invented and thought through before me. If I see enough of that, I can also pick the components that I think are right, and discard the components that I think are wrong. In other words, I can learn.
With closet, err, closed-source code, this is impossible. You can only build on top of it, and hope that the developer really knew what he was doing and made no mistakes that will affect what you are doing. It is as if Galileo and Newton had no insights into the equations, but just saw the numbers coming out of the theory. And remember — the numbers coming out of Ptolemy’s model were correct — for the known objects.
It is also like building on ground where you can’t check whether it actually is a swamp. In fact, it is illegal for you to check whether it is a swamp, and you have to build on that ground because “everybody else does it”. And where the holy creator of the ground thought that investing 20% of the necessary work is good enough.