The most powerful technology ever only has one job. Some thoughts about the scientific method how it works and what its one fundamental function actually is. This is a (rough) transcript of
for those who prefer reading.
I love science. Well, what I mean is I love understanding how things work.
But have you spent much time thinking about how we accumulate knowledge and specifically, the scientific method: like what it is at it’s core, what it’s really really for?
Let me share some of my thoughts on a topic I think is pretty interesting and maybe you can give me a few insights of your own.
Now, it was only sometime after my physics studies that I finally got this. For reasons of pure survival, I think we’re all–like 100% of us–we’re all born scientists. Sure, it gets beaten out of many (mainly by the school system), but we start out with all the basics nicely built-in.
Curiosity is the main driver, and that juice is hard-wired to a bunch of systems.
Loewenstein’s “information gap theory” is interesting –maybe look into that if you’re curious about curiosity–but the short version being that the reason kids will ask and infinite series of “why?”, and people will actually pay to hear answers they’re curious about, is that noticing a gap in knowledge causes a mental itch and filling that gap gives you a nice dopamine hit, getting you hooked on finding answers.
I, for one, am definitely hooked.
Now, if the gap is too small, you don’t give a crap, and if it’s an insurmountable chasm, it might all seem scary or like gibberish so there’s a sweet spot–or a sweet range–that is just enough to get you moving. There’s some mapping, or at least some similarity, between this and the sweet spot for flow-state–another thing I’m addicted to–where the difficulty of a problem or performance that is just at the limit of your range will let you get in the zone, where you’re locked-in, the flow of time is suspended and you’re fully absorbed.
So curiosity light’s the spark and draws you in to start filling a gap and flow sustains you in the grind while you’re practically working something out.
We all start out with the basics of a scientific intuition: we definitely have curiosity, we’re also pattern recognizer, so you can spot repeated things, as well as breaks in a pattern, we work by trial and error–which is just experimentation, where you act, observe, tweak and try again.
And with all that, “intuitive physics” studies, on babies and other animals, using the proxy of how long these little speechless creatures will stare at unexpected things, like objects disappearing or defying gravity or whatever. These studies we have some built-in expectations of consistency.
So, that’s our starting point.
And I can tell you, for reasons I’m still working out, all of this was powerfully turned on for me as a child–and I got lucky because it didn’t turn into defensive avoidance just ever greater hunger for knowledge.
This one time in high school, we were supposed to do a 15 minute presentation on a topic of our choice and I elected to summarize Stephen Hawkins’ “A brief history of time”. I came in with graphs, and cookware with plastic food wrap and balls–to demonstrate gravity, you know–and basically took up a whole hour, rather than my 15 minutes, to talk about it, because I loved it and thought other’s would to.
And so, at the end of the class, I had a few people come up to me, and thank me… but it was all because they’d also been scheduled to present that day, and got to skip it thanks to my blathering on and on.
So, ok, maybe it’s not turned on to the same extent, or at least on absolutely every subject, for everyone. Still: we have this built-in drive and mechanism, but… well, it’s not perfect. Now, though it may have killed the cat, I simply have nothing bad to say about curiosity, I think it’s all win.
Pattern recognizing, however: that’s not a double-edged sword, it’s a double-edged sword with a handle made of … more sword. It’s almost all blade.
It gets a whole lot of work done, but you have to handle that thing with respect and a hint of dread.
The pattern recognizer–at least in mine but I suspect in all brains–is an always-on over-fitting machine that never stops whispering in your ear. That thing is tricky and I–someone with a scientific background, interested in how knowledge is acquired, somewhat obsessed about biases and false beliefs and memories–have felt the pull of superstition and magical thinking that are the hallmarks of misapplying the pattern recognizer. This is especially strong in times of crisis but starts getting louder whenever something is particularly important to you.
We did pretty good for a long while relying on just this so-so built-in system and, at some point, got the opportunity to just defer to Aristotle whenever the nature of the universe came up.
You know, Aristotle: the guy claiming that women have fewer teeth than men. How long would it have taken to get off your ass and turn to your lady with, “hey, could you open your mouth for a sec, there’s something I want to check”?
For a long time, mainly thanks to college, I thought philosophy was the set of those useless questions that by definition, we can never resolve. Now, I don’t think the question of the number of angels dancing on the head of a pin was ever actually discussed, and frankly if you’re going to have angels in your model of the universe then “angelology”–questions about angels’ nature, location, and capabilities–actually are worth posing.
Well, they are so long as there’s some way of getting an answer.
Anyway, at this point, I have a more fine-grained opinion of philosophy–navel gazing BS can be a fun exercise but is mostly a garbage waste of time, but questions of ethics, knowledge and its limits, theories of mind, politics and what is actually real and what that means, definitely have their place.
So, to me, there 3 basic categories of question:
Factual questions: there is a definite answer, out there, independent of whether or not you can’t get to it.
Normative questions: which cover things that should be, and the answers are all based in values. I also lump in conceptual questions like definitions of meaning, and categories because these are basically agreements we make more than anything objective. And finally,
there’s good old bullshit questions: things for which the only valid answer is “mu!”,
These are questions that collapse either because they are paradoxical (you know “what was it like before time began?”), which is what Wittgenstein calls “language on holiday” or just plain gibberish.
So that last one goes straight to the bin after you’ve identified it.
The normative questions, like which economic system should we adopt–and that’s normative without a doubt: you can use facts to corroborate and prop up your opinion, but its 100% about what you value most… individual freedom, progress however that’s defined, fairness however that’s defined, etc.
Questions like that are important and setup how we’re going to work together, so those are good ones to spend time on.
However, only the factual questions can really benefit from the scientific method and all those factual questions, yes, including the number of teeth in a woman’s head, have to be put through the wringer of this method if you want to build an edifice of useful knowledge that won’t crash down at some point, with potential catastrophic consequences.
So, I’m sure lots of folks came to the same conclusions, but back in the 1600s things really start rolling when Francis Bacon starts emphasizing observation to build up general principles and then testing those in the real world.
This is a long term collaborative effort, we have Galileo actually dropping stuff and looking at what happens, Descartes pushing skepticism and logic and on and on.
So here’s how what they built up goes for most interesting questions where this is applicable:
you see something weird, or wonder how it works, and then
play with it in your mind for a little bit and come up with what you think is a probable explanation, mechanism or answer.
Now you can stop there, like Aristotle did with the teeth, but you’re mileage may vary. The hit rate isn’t likely to be great. To be scientific about it, you have to actually look at the world but it’s not even enough to compile a bunch of evidence from out there.
It takes all the way until the 1930s to really get that it has to be possible to know if your proposed answer is wrong.
So in comes Popper with the notion of falsifiability, saying that if a theory can’t be falsified, then it can be twisted to explain anything and it just isn’t science, what it is is storytelling. And this is the crux of it: you figure out how the world is different in the case that your idea is true, as compared to the case where it’s false, right?
If there is such a difference, and it can actually be seen one way or another, then you arrange to pose that question to the universe–that’s what an experiment is, just a repeatable question with a clear result–and mark the answer you get down.
Boom! Science.
Now there’s lots of stuff we add on to the basic protocol: publishing your methods, peer review, etcetc, but that’s just building in resilience and double-checking.
So that’s the basic method and it points to what it’s actually for.
You see, the thing it doesn’t do, something it doesn’t cover at all, is where the hypothesis comes from.
Science isn’t what generates hypotheses.
The hypothesis, your theory about how things actually work, is an idea and it comes from intuition, maybe informed by past experience, but the spark of it is from somewhere deep inside you–or from, well everywhere, if you’re a panpsychist–but the point is humans are idea generating machines. We just can’t stop and it seems the more you do it, the faster it keeps happening.
Now ideas can be good ideas or can be total shite. The thing is, the further we get along, the higher the proportion of garbage.
This is like mutation: a random mutation might be advantageous. Yeah, it might. But the more complex, fine tuned, intricate the machine is, the more it becomes likely that a random change is going to break things rather than improve them.
If you’re standing at the edge of tens of thousands of years of building up society, technology whatever, it gets pretty likely that your brilliant new idea is going to be total crap. Either it’s been tried and discarded a million times–which is why there’s some push for publishing negative findings, though the incentive structures just aren’t setup to actually encourage that–or it really is new and probably just junk.
Now the process isn’t completely random, like mutation, but it’s just that there a so few good ideas and true things in the infinite sea of infinite of bad ideas and false things, that it really doesn’t make a difference. So, that, my friends, is the purpose of the scientific method: it is a sieve, a colander, a filter to keep the garbage out of our knowledge soup.
And that’s it, that’s the whole thing.
You get to be human, and play with ideas, and generate an unending stream of possibility and, when you care to try and only want to include stuff that is valid and useful, you pull out our old pal the scientific method, and you carefully and honestly distill delicious veracity from the mountains of refuse, or at least inch your way just a tiny bit closer to truth.
Just make careful note of the difference between science and The Science.
Note the capital letters there, in “The Science”
The. Science. is something handed down by authority, blasted at you continuously. It’s enforced with peer pressure, half-understood cargo-cult incantations cloaked in the right language and backed by a tsunami of nothingburger. Its propelled by feelings of righteousness and the dynamics of tribalism and group belonging.
A good test is, if you ask too many or the wrong questions, you are [insert expletive of your choice here], dangerous and must be silenced. That’s just how propaganda works.
Science, on the other hand, isn’t a thing or even a set of beliefs but just process of seeing which ideas get the right to stick around a bit longer. Now no group of people is uniform: if you saw that thing with the flat-earthers not long ago, where some pastor or something brought a bunch of them to the antarctic to actually do some observation, you could tell that some of them were actually interested in knowing, and those people publicly adjusted their views based on new evidence. Good job.
Others… well not so much, and they just shifted around to find a way stubbornly hold onto their prior position.
The flat earth notion is really quite stupid and so easily disproven–you don’t actually have to go to the poles to do that–but, once unleashed, paranoia is tough beast to tame and some people have been so mistreated, are so frightened, and they’ve acted and said things that would be painful to backtrack on and eventually their identity gets so intertwined that the certainty of hanging on to a wrong answer is better than anything else.
So, all this in a nutshell:
let curiosity and ideas run rampant,
figure out if your current question is one that is of a factual nature,
decide if you care about getting closer to truth more than you care about getting some particular answer
and, if so, apply the most powerful technology we’ve ever devised to the question.
The Scientific Method
The most powerful technology ever only has one job. Some thoughts about the scientific method how it works and what its one fundamental function actually is.
This is a (rough) transcript of
for those who prefer reading.
I love science. Well, what I mean is I love understanding how things work.
But have you spent much time thinking about how we accumulate knowledge and specifically, the scientific method: like what it is at it’s core, what it’s really really for?
Let me share some of my thoughts on a topic I think is pretty interesting and maybe you can give me a few insights of your own.
Now, it was only sometime after my physics studies that I finally got this. For reasons of pure survival, I think we’re all–like 100% of us–we’re all born scientists. Sure, it gets beaten out of many (mainly by the school system), but we start out with all the basics nicely built-in.
Curiosity is the main driver, and that juice is hard-wired to a bunch of systems.
Loewenstein’s “information gap theory” is interesting –maybe look into that if you’re curious about curiosity–but the short version being that the reason kids will ask and infinite series of “why?”, and people will actually pay to hear answers they’re curious about, is that noticing a gap in knowledge causes a mental itch and filling that gap gives you a nice dopamine hit, getting you hooked on finding answers.
I, for one, am definitely hooked.
Now, if the gap is too small, you don’t give a crap, and if it’s an insurmountable chasm, it might all seem scary or like gibberish so there’s a sweet spot–or a sweet range–that is just enough to get you moving. There’s some mapping, or at least some similarity, between this and the sweet spot for flow-state–another thing I’m addicted to–where the difficulty of a problem or performance that is just at the limit of your range will let you get in the zone, where you’re locked-in, the flow of time is suspended and you’re fully absorbed.
So curiosity light’s the spark and draws you in to start filling a gap and flow sustains you in the grind while you’re practically working something out.
We all start out with the basics of a scientific intuition: we definitely have curiosity, we’re also pattern recognizer, so you can spot repeated things, as well as breaks in a pattern, we work by trial and error–which is just experimentation, where you act, observe, tweak and try again.
And with all that, “intuitive physics” studies, on babies and other animals, using the proxy of how long these little speechless creatures will stare at unexpected things, like objects disappearing or defying gravity or whatever. These studies we have some built-in expectations of consistency.
So, that’s our starting point.
And I can tell you, for reasons I’m still working out, all of this was powerfully turned on for me as a child–and I got lucky because it didn’t turn into defensive avoidance just ever greater hunger for knowledge.
This one time in high school, we were supposed to do a 15 minute presentation on a topic of our choice and I elected to summarize Stephen Hawkins’ “A brief history of time”. I came in with graphs, and cookware with plastic food wrap and balls–to demonstrate gravity, you know–and basically took up a whole hour, rather than my 15 minutes, to talk about it, because I loved it and thought other’s would to.
And so, at the end of the class, I had a few people come up to me, and thank me… but it was all because they’d also been scheduled to present that day, and got to skip it thanks to my blathering on and on.
So, ok, maybe it’s not turned on to the same extent, or at least on absolutely every subject, for everyone. Still: we have this built-in drive and mechanism, but… well, it’s not perfect. Now, though it may have killed the cat, I simply have nothing bad to say about curiosity, I think it’s all win.
Pattern recognizing, however: that’s not a double-edged sword, it’s a double-edged sword with a handle made of … more sword. It’s almost all blade.
It gets a whole lot of work done, but you have to handle that thing with respect and a hint of dread.
The pattern recognizer–at least in mine but I suspect in all brains–is an always-on over-fitting machine that never stops whispering in your ear. That thing is tricky and I–someone with a scientific background, interested in how knowledge is acquired, somewhat obsessed about biases and false beliefs and memories–have felt the pull of superstition and magical thinking that are the hallmarks of misapplying the pattern recognizer. This is especially strong in times of crisis but starts getting louder whenever something is particularly important to you.
We did pretty good for a long while relying on just this so-so built-in system and, at some point, got the opportunity to just defer to Aristotle whenever the nature of the universe came up.
You know, Aristotle: the guy claiming that women have fewer teeth than men. How long would it have taken to get off your ass and turn to your lady with, “hey, could you open your mouth for a sec, there’s something I want to check”?
For a long time, mainly thanks to college, I thought philosophy was the set of those useless questions that by definition, we can never resolve. Now, I don’t think the question of the number of angels dancing on the head of a pin was ever actually discussed, and frankly if you’re going to have angels in your model of the universe then “angelology”–questions about angels’ nature, location, and capabilities–actually are worth posing.
Well, they are so long as there’s some way of getting an answer.
Anyway, at this point, I have a more fine-grained opinion of philosophy–navel gazing BS can be a fun exercise but is mostly a garbage waste of time, but questions of ethics, knowledge and its limits, theories of mind, politics and what is actually real and what that means, definitely have their place.
So, to me, there 3 basic categories of question:
These are questions that collapse either because they are paradoxical (you know “what was it like before time began?”), which is what Wittgenstein calls “language on holiday” or just plain gibberish.
So that last one goes straight to the bin after you’ve identified it.
The normative questions, like which economic system should we adopt–and that’s normative without a doubt: you can use facts to corroborate and prop up your opinion, but its 100% about what you value most… individual freedom, progress however that’s defined, fairness however that’s defined, etc.
Questions like that are important and setup how we’re going to work together, so those are good ones to spend time on.
However, only the factual questions can really benefit from the scientific method and all those factual questions, yes, including the number of teeth in a woman’s head, have to be put through the wringer of this method if you want to build an edifice of useful knowledge that won’t crash down at some point, with potential catastrophic consequences.
So, I’m sure lots of folks came to the same conclusions, but back in the 1600s things really start rolling when Francis Bacon starts emphasizing observation to build up general principles and then testing those in the real world.
This is a long term collaborative effort, we have Galileo actually dropping stuff and looking at what happens, Descartes pushing skepticism and logic and on and on.
So here’s how what they built up goes for most interesting questions where this is applicable:
and then
Now you can stop there, like Aristotle did with the teeth, but you’re mileage may vary. The hit rate isn’t likely to be great. To be scientific about it, you have to actually look at the world but it’s not even enough to compile a bunch of evidence from out there.
It takes all the way until the 1930s to really get that it has to be possible to know if your proposed answer is wrong.
So in comes Popper with the notion of falsifiability, saying that if a theory can’t be falsified, then it can be twisted to explain anything and it just isn’t science, what it is is storytelling. And this is the crux of it: you figure out how the world is different in the case that your idea is true, as compared to the case where it’s false, right?
If there is such a difference, and it can actually be seen one way or another, then you arrange to pose that question to the universe–that’s what an experiment is, just a repeatable question with a clear result–and mark the answer you get down.
Boom! Science.
Now there’s lots of stuff we add on to the basic protocol: publishing your methods, peer review, etcetc, but that’s just building in resilience and double-checking.
So that’s the basic method and it points to what it’s actually for.
You see, the thing it doesn’t do, something it doesn’t cover at all, is where the hypothesis comes from.
Science isn’t what generates hypotheses.
The hypothesis, your theory about how things actually work, is an idea and it comes from intuition, maybe informed by past experience, but the spark of it is from somewhere deep inside you–or from, well everywhere, if you’re a panpsychist–but the point is humans are idea generating machines. We just can’t stop and it seems the more you do it, the faster it keeps happening.
Now ideas can be good ideas or can be total shite. The thing is, the further we get along, the higher the proportion of garbage.
This is like mutation: a random mutation might be advantageous. Yeah, it might. But the more complex, fine tuned, intricate the machine is, the more it becomes likely that a random change is going to break things rather than improve them.
If you’re standing at the edge of tens of thousands of years of building up society, technology whatever, it gets pretty likely that your brilliant new idea is going to be total crap. Either it’s been tried and discarded a million times–which is why there’s some push for publishing negative findings, though the incentive structures just aren’t setup to actually encourage that–or it really is new and probably just junk.
Now the process isn’t completely random, like mutation, but it’s just that there a so few good ideas and true things in the infinite sea of infinite of bad ideas and false things, that it really doesn’t make a difference. So, that, my friends, is the purpose of the scientific method: it is a sieve, a colander, a filter to keep the garbage out of our knowledge soup.
And that’s it, that’s the whole thing.
You get to be human, and play with ideas, and generate an unending stream of possibility and, when you care to try and only want to include stuff that is valid and useful, you pull out our old pal the scientific method, and you carefully and honestly distill delicious veracity from the mountains of refuse, or at least inch your way just a tiny bit closer to truth.
Just make careful note of the difference between science and The Science.
Note the capital letters there, in “The Science”
The. Science. is something handed down by authority, blasted at you continuously. It’s enforced with peer pressure, half-understood cargo-cult incantations cloaked in the right language and backed by a tsunami of nothingburger. Its propelled by feelings of righteousness and the dynamics of tribalism and group belonging.
A good test is, if you ask too many or the wrong questions, you are [insert expletive of your choice here], dangerous and must be silenced. That’s just how propaganda works.
Science, on the other hand, isn’t a thing or even a set of beliefs but just process of seeing which ideas get the right to stick around a bit longer. Now no group of people is uniform: if you saw that thing with the flat-earthers not long ago, where some pastor or something brought a bunch of them to the antarctic to actually do some observation, you could tell that some of them were actually interested in knowing, and those people publicly adjusted their views based on new evidence. Good job.
Others… well not so much, and they just shifted around to find a way stubbornly hold onto their prior position.
The flat earth notion is really quite stupid and so easily disproven–you don’t actually have to go to the poles to do that–but, once unleashed, paranoia is tough beast to tame and some people have been so mistreated, are so frightened, and they’ve acted and said things that would be painful to backtrack on and eventually their identity gets so intertwined that the certainty of hanging on to a wrong answer is better than anything else.
So, all this in a nutshell:
Hypothesize hard. Test harder.