Research and a three-year-old

The incident that triggered the thought process behind this post happened when my son was about 3 years old. This was the time when he was trying to figure out 'cause and effect' relationships. So he used to say things like "If I shout in the class, my teacher will scold me", "I ran very fast in the park. That is why I fell down" etc. Those days we used to have an evening ritual. I would put my son on my shoulders and go for a walk. This 'sitting on the shoulders' arrangement made conversations easy even when there was a lot of noise all around. So this led to a lot of interesting discussions. There is nothing quite like a conversation with a curious, confident and talkative three-year-old to force one to be aware of and to question one's assumptions!

These walks would take us near a manned railway crossing/gate. Since he likes to see trains, we would stand there for a long time. After a few days he told me about a 'discovery' he has made "The gate has closed. That is why the train is coming"! Now, we all know that the 'causation' (if any) is the other way around. But purely based on his observations this was not so. He sees one thing happening (gate closes). After that something else always happens (train comes). Based on his 'life experience so far' (or his understanding of the 'system'/'universe') it was reasonable for him to think that if something happens and something else always happens after that the first thing might be causing the second thing (this principle had worked for him in the two examples mentioned above - running in the park and shouting in the class).

So, how would I convince him that his conclusion was wrong? The only way that I could think of was to tell him about the larger system (the railway system in this case - that makes the trains run and the gates close). This solution 'worked' only because there was someone around who knew about the larger system. He could not have come to the 'correct conclusion' purely based on his observations and his life experience thus far (i.e. based on his understanding of the 'system' at that point) .

Now, if we look at the research in behavioral science (or may be research in general), often we don't have the luxury of fully knowing the larger system in which the phenomena that we are observing are happening. Also there might not be anyone who has an adequate understanding of the system to 'enlighten' us. Actually, such understanding might not even exist! (as all the 'possible' events/system behaviors might not have been observed or even taken place so far - e.g. unusual/rare events/system behaviors like those that could result from malfunctioning of railway signals, human error, train breakdowns, accidents etc. or events like 'two trains passing through the railway gate at the same time on parallel tracks' that could arise from from a peculiar/uncommon combination of factors - if we stick with our original example). Often, there is no way we can study the 'entire system' (actually it would be very difficult even to determine the exact boundaries of the relevant 'system' in a particular study). We might not be in a position to look at all the data. So have to decide what data we would study and what data we would leave out. This could bring in biases (e.g. selection bias, survivorship bias etc.) and limitations. Thus, there is a significant risk that we might make the wrong inference (since we are limited by our observations and our current level of understanding of the system).

In addition to this, there are the standard problems with spurious correlations, mistaking correlation for causation, determining the direction of causation ('A causes B' or 'B causes A' or 'C causes both A and B' etc.) and assumptions regarding the homogeneity/uniformity of the system (assuming that findings that are valid in one part of the system are equally valid in other parts of the system). Of course, there are ways of expanding both our 'current level of understanding' and our data set/observations (e.g. study of the existing 'research' in the domain- if relevant and available). But, if we examine most of the 'research' that happens within organizations (for diagnosis and decision making - to solve the immediate problems in particular organization contexts), the pressures of time and resources might dilute the efforts to expand the 'understanding and data set'. Again, it is possible that the 'system' might have changed (in subtle but significant ways - without us noticing it) from what it was at the time we studied it/derived inferences on system behavior. Considering the nature and pace of change in many of the human systems that we are taking about, this could pose a big challenge for making available 'valid actionable inferences' to guide our decision making. Keeping all this in mind, can we expect to do always better than what my three-year-old had managed to do?

Note: I am not saying that useful behavioral research can't be conducted in organizations. My point is just that it requires a convergence of 'realistic expectations', 'will' and 'resources' - which, unfortunately, is not very common in most 'real world' organization contexts. If the 'research problem' can be defined narrowly, I would not even rule out the possibility of 'experiments' (though 'experiments' might not be a 'politically correct' term in organization contexts ; 'pilot studies' might be more appropriate). If such experiments can be conducted in the filed of medicine (where - literally - 'life and death' issues are involved), why can't we try them in business organizations (with proper precautions)? Of course, the problems like the ones that I have mentioned above (e.g. too many variables, difficulty in conducting 'controlled experiments', insufficient understanding of the system, biases in selection of data, assumptions about homogeneity and stability of the population/system etc.) still apply. But we might still get some useful information and/or insights.

Any comments/thoughts/ideas?

See somewhat related posts here, here and here.