https://old-wiki.lesswrong.com/wiki/Rationality#Instrumental_rationality
influenced by cognitive bias
“rationality is not about recognizing facts, but about which facts are relevant”
https://www.lesswrong.com/posts/CqyJzDZWvGhhFJ7dY/belief-in-belief (It’s easier to believe you ought to believe X than to actually believe X.)
https://www.lesswrong.com/w/arguments-as-soldiers
https://www.lesswrong.com/posts/fQ5bKXZ69qQ5kZhpd/dealing-with-curiosity-stoppers
When you find yourself in philosophical difficulties, the first line of defense is not to define your problematic terms, but to see whether you can think without using those terms at all. Or any of their short synonyms. And be careful not to let yourself invent a new word to use instead. Describe outward observables and interior mechanisms; don’t use a single handle, whatever that handle may be.
Saying the right words (“light is waves,” “heat conduction”) isn’t the same as understanding.
School trains you to guess which verbal password the teacher wants, not to form mental models that actually predict what you’ll observe.
Understanding = your model controls your anticipations.
Generate predictions from your model and check them against reality.
I suspect the most dangerous habit of thought taught in schools is that even if you don’t really understand something, you should parrot it back anyway. One of the most fundamental life skills is realizing when you are confused, and school actively destroys this ability - teaches students that they “understand” when they can successfully answer questions on an exam, which is very very very far from absorbing the knowledge and making it a part of you.
Much of this problem may come from needing to take three 4-credit courses per quarter, with a textbook chapter plus homework to be done every week - the courses are timed for frantic memorization, it’s not possible to deeply chew over and leisurely digest knowledge in the same period. College students aren’t allowed to be confused; if they started saying, “Wait, do I really understand this?
Maybe I’d better spend a few days looking up related papers, or consult another textbook,” they’d fail all the courses they took that quarter. A month later they would understand the material far better and remember it much longer - but one month after finals is too late; it counts for nothing in the lunatic university utility function.
It may be dangerous to present people with a giant mass of authoritative knowledge, especially if it is actually true. It may damage their skepticism.
So what could you do? Teach students the history of physics, how each idea was replaced in turn by a new correct one? “Here’s the old idea, here’s the new idea, here’s the experiment - the new idea wins!” Repeat this lesson ten times and what is the habit of thought learned? “New ideas always win; every new idea in physics turns out to be correct.” You still haven’t taught any critical thinking, because you only showed them history as seen with perfect hindsight. You’ve taught them the habit that distinguishing true ideas from false ones is perfectly clear-cut and straightforward, so if a shiny new idea has anything to recommend it, it’s probably true.
Maybe it would be possible to teach the history of physics from a historically realistic point of view, without benefit of hindsight: show students the different alternatives that were considered historically plausible, re-enact the historical disagreements and debates.
Maybe you could avoid handing students knowledge on a silver platter: show students different versions of physics equations that looked plausible, and ask them to figure out which was the correct one, or invent experiments that would distinguish between alternatives. This wouldn’t be as challenging as needing to notice anomalies without hints and invent alternatives from scratch, but it would be a vast improvement over memorizing a received authority.