A few weeks ago a mentor of mine sent me a gift: a book by Bruce Temper, titled Staying Alive in Avalanche Terrain. It documents the science and technique behind avalanche forecasting, which is a life or death science for those guys. Nobody can afford to get caught up in an avalanche.
It’s one of those books that’s about one specific subject but really applies to a bunch of other things in life. As the book reads on it gets more philosophical in nature.
Not knowing is true knowledge. Presuming to know is a disease.
Temper references this Lao Tzu quote and boils it down to the idea that survival favors those who make evidence-based decisions and disfavors those who cling to beliefs.
He then writes that belief inertia is the time delay between when facts are acquired and when they sink in (when we “get it”). A shorter time delay means you allow your beliefs to change more easily based on evidence. In the avalanche forecasting world, this kind of thinking saves lives.
When I think about where this evidence comes from, I see it all around us. On the mountain it is the temperature, soil samples, rock formations, snow depth. In our businesses it lies in the data we collect. Making informed decisions based on data can separate a successful business from the rest.
We increasingly use the buzzword Big Data, but to me, Big Data is a bunch of data laying around, data that’s easy to presume is right. We may think we are making an informed decision, but when I look at the world around me, when I look at the Internet and what is has become, I will be the first one to tell you - I don’t know anything. I mean I literally have no clue what is going on half the time. Yeah, I said it.
When you get to that point where you can admit to yourself (and hopefully to others) that you don’t know anything, all of a sudden you start to open your mind to how you acquire knowledge and why you believe things. There is a process that takes place where you are exposed to ideas and you ask questions and acquire evidence and that evidence updates your beliefs. You repeat that cycle hundreds if not thousands of times until you eventually funnel to the correct belief about something.
Small data is a thing. It is the little bits of evidence that we consume over and over again to funnel our beliefs and let us make informed decisions. This definition places a few requirements on small data: that it be distilled into bite sized consumable bits and that it be rapidly accessible to power our belief inertia. The next decade will belong to small data.