I’ve been writing a lot about intentionality in data science, about how having a sense of history (present and future), can be incredibly powerful for any enterprise.
Think about how archaeologists use data to seek the truth, as compared with how astronomers do it.
Archaeology starts with digging. It’s all about studying the data that’s buried in the system (i.e. the fossil record), which means studying things that probably weren’t put there intentionally (depending on your belief system). Without a time machine, it’s impossible to change the structure of the record, to apply intention to the signal, so we do the best we can with what we’ve got: we mine through the accidental signals, discarding the (literal) mountains of noise, in an effort to find the truth about history. Perhaps as should be expected, this effort is expensive and leads to mixed results.
On the other hand, Astronomy is a very different field. Astronomy starts way earlier than digging — it starts with planting. At instrumentation-time, astronomers can point the telescopes where they want, measuring and recording the signals they want. Unlike archaeologists, astronomers have the ability to design the record and its structure, to choose the signals with intention. Doing this intentionally sets up the data record to yield the discoveries they already know they want to make, but also to be somewhat future-proof (which means it can yield unpredicted, emergent discoveries, to be harvested down the road — often by different people than the planters).
Now let’s compare the results of these two fields.
Astronomy (with its cousin Astrophysics) has taught us amazing lessons, things about the motion of the galaxy, the origin of the universe, and the underlying physical principles of multi-dimensional reality.
Turning our gaze to the stars, we learn about the earth. That’s pretty impressive.
Meanwhile, as of last year, Archaeology is still struggling to figure out how many years ago Homo sapiens emerged. And they can’t seem to agree on it, even though all the data is right under our noses. This isn’t because they’re incompetent (some of the best pattern-seeking humans in all of science work in archaeology), but rather because the data sucks.
Clearly, one of these truth-seeking disciplines is a lot more powerful than the other, and at Keen IO, we contend this is because they can control the data model. Data modeling is powerful indeed.
Inspection of any kind — be it human introspection or scientific inquiry — is more powerful when you can apply a variety of observational frameworks, choosing the best of them.
