Why Understanding the Data-Generation Process Is More Important Than the Data Itself | by Zijing Zhu, PhD | Dec, 2023


Photo by Ryoji Iwata on Unsplash

“The Book of Why” Chapters 5&6, a Read with Me series

Zijing Zhu, PhD

During the early stages of infancy, our brains already learn to associate correlation with causation and try to find an explanation for everything happening around us. If a car behind us takes the same turns we do for a long time, we assume it’s following us, which is a causal assumption. However, when we snap out of the movie mood, we then think we are properly just heading to the same destination — — a confounder. A common cause introduces a correlation between the two cars’ movements. This vivid and relatable example that Pearl gives proves how the human brain works.

What about the correlations that we couldn’t fathom a reasonable explanation? Such as two diseases that are uncorrelated among the whole population but correlated among the hospitalized population. If you recall my last article that discussed different causal structures, it points out that conditioning the colliders (hospitalized) generates an explain-away effect that makes two uncorrelated variables spuriously correlated. In other words, the hospitalized population is not an accurate representation of the general population, and any observations made from this sample cannot be generalized.

Collider Bias, image by author based on “The Book of Why” Chapter 6

Collider-induced correlations are not intuitive to human brains, thus generating these so-called paradoxes. In this article, I will explore more interesting paradoxes that create optical illusions in our brains like magic tricks but can be explained with causal diagrams. Understanding what’s causing these paradoxes is meaningful and educational. It is the 4th article for the “Read with Me” series, and it’s based on Chapters 5 and 6 from “The Book of Why” by Judea Pearl. This should be a fun read based on all the examples Pearl gives in these two chapters! You can find the previous article here:



Source link

This post originally appeared on TechToday.

Leave a Reply

Your email address will not be published. Required fields are marked *