Pretend for a moment you are a 40 year old women who just had a routine mammogram. Your doctor tells you that only 1% of women your age have breast cancer, that 80% of women with breast cancer will get positive mammographies, and that 9.6% of women without breast cancer will also get positive mammographies. Suppose your mammogram was positive. What is the probability that you actually have breast cancer?
Think about it for a moment, and take a guess. Or try to work it out.
Now, consider that when doctors are given this problem, usually about 15% of them get it correct. Feel free to revise your estimate. I'll give the answer at the end of the post.
The US government is currently "fighting" a "war on drugs". In spite of this, drug usage is up (http://www.msnbc.msn.com/id/39200536/ns/health-addictions/t/illegal-drug-use-higher-nearly-decade-report-finds/). Everyone who examines the issue that I'm aware of has suggested that legalize-and-tax is a better strategy both to minimize overall harm and for the goal of reducing drug usage. For example, it's been very effective against smoking (http://www.washingtonpost.com/wp-dyn/content/article/2009/11/16/AR2009111603336.html). Yet politicians are extremely reluctant to change tactics, despite the massive collateral damage the war on drugs causes (for example, many gangs exists solely to distribute drugs; also, drugs are produced in uncontrolled environments and hence cause more damage to their users than necessary).
Both the British and French governments realized that the Concord would lose money before they finished it. Yet they continued to fund its development because they'd invested so much in it already.
In a study, a group of people was asked how much they would pay to save 2000 migratory birds from drowning in oily ponds. A second group was asked the same question of 20,000 birds, and a third was asked about saving 200,000 birds. The three groups answered $80, $78, and $88, respectively. People base their charitable giving on how good it makes them feel, not on how much good it actually does in the world.
Rationality is the art and science of making your map actually reflect the territory (epistemic rationality). Further, it is the art and science of effectively modifying the territory to match your preferences, or values (instrumental rationality). (Read my past couple posts for clarification on what I mean by map and territory, if needed!)
Note that even though the words are similar, it's not the same thing as "rationalization" in any way. Rationalization is the art of believing what you want to believe, evidence be damned.
The consequences of irrationality are all around us; I listed a few above. We've gotten used to them. In some cases we don't even realize that we can do better.
Rationality caries a price, too. When you find out your map is wrong, you have to change it. Changing your mind and admitting that you've been wrong can be extremely painful. Is it worth the cost? You'll have to decide that for yourself: would you like to continue believing everything you currently believe even if it wasn't true?
As for the problem: most doctors say 70-80%, but this ignores the prior probability of cancer in the general population and is wildly incorrect. For the correct answer and why it is correct, read: http://yudkowsky.net/rational/bayes
For more about the birds and scope insensitivity in general, see: http://lesswrong.com/lw/hw/scope_insensitivity/
For a more detailed introduction to rationality, see: http://lesswrong.com/lw/7e5/the_cognitive_science_of_rationality/