IJE Advance Access originally published online on January 28, 2009
International Journal of Epidemiology 2009 38(2):368-370; doi:10.1093/ije/dyn355
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Commentary: ‘Is the Social World Flat? W.S. Robinson and the Ecologic Fallacy’
Department of Sociology, Pennsylvania State University, USA.
E-mail: firebaug@pop.psu.edu
Accepted 9 September 2008
If the social world were ‘flat’ in the sense that it did not matter where you lived or with whom you associated—no place effects, no context effects, no contagion effects—then single-level analysis would do. The message of Subramanian, Jones, Kaddour and Krieger1 is that the social world usually is not flat, so multilevel analysis usually is called for. The idea that single-level analysis is problematic when there are multilevel effects is quite consistent with Robinson's classic warning about the ecologic fallacy.2 In fact, I suspect that Robinson himself would have embraced multilevel analysis had it existed in his day.
Because Robinson has had a multiplicity of interpreters, it is sometimes difficult to separate what Robinson actually said from a caricature of what he said (his article is only seven pages long, and I recommend that readers examine it for themselves). In some instances interpreters have ‘out-Robinsoned’ Robinson. To cite one example: after defining terms, Robinson lists, in his third paragraph, more than a dozen pre-1950 articles that used ecological correlations to estimate individual-level correlations. In the next paragraph, referring to ‘these studies’ (sic), Robinson writes that ‘Ecological correlations are used simply because the properties of individuals are not available’ (p. 352). Although that statement refers specifically to the aims of the studies Robinson has just listed, it is sometimes quoted out of context as axiomatic for all social research.
To be true to Robinson, let us begin where he did, with the issue of whether individual-level relationships can be reliably inferred from aggregate-level correlations. Consider this simple example from US history: the vote for George Wallace in the 1968 Presidential election. Suppose we want to know how much less likely blacks were (than others) to vote for Wallace, a four-term governor of Alabama and well-known segregationist who ran as a third-party candidate. Because we do not have individual-level election data on who voted for whom, we might try to infer the relationship between race and Wallace vote by calculating the correlation of percent black and percent vote for Wallace across Congressional districts. The correlation, it turns out, is strongly positive (r = 0.55 for districts in the South3).
Robinson's contribution was to show the danger of inferring individual-level relationships from aggregate-level associations, a common practice at the time he was writing. In our example, although Wallace received a higher share of votes in regions with higher percentages of blacks, it does not necessarily follow that blacks were more likely to vote for him. In fact, blacks were much less likely to vote for Wallace4 (p. 230): in one post-election survey, of the 87 black respondents who had voted, not a single one had voted for Wallace; in another survey with 237 black voters, only two had voted for Wallace (among non-black voters, in contrast, one in eight had voted for Wallace). Hence if we inferred individual-level relationships from the district-level association, we would conclude, quite erroneously, that blacks were disproportionately inclined to vote for Wallace.
Although Robinson was not the first to recognize the danger of inferring individual-level relationships from aggregate-level associations, his article brought the issue to the fore by showing that the discrepancy between aggregate-level rs and their individual-level counterparts can be huge—indeed, the signs can even be reversed, as in the Wallace example. ‘The purpose of this article’, Robinson2 (p. 352) wrote, ‘is to clarify the ecological correlation problem by stating, mathematically, the exact relation between ecological and individual correlations’. Robinson then devotes much of the remainder of his article to mathematical expressions of the relationship between aggregate- and individual-level correlation coefficients. On the basis of those expressions, he concluded that ‘the only reasonable assumption is that an ecological correlation is almost certainly not equal to its corresponding individual correlation’ (p. 357). (Although Robinson focused on correlation coefficients, regression coefficients are vexed by the same problem, albeit discrepancies may be less pronounced in the case of regression coefficients.5,6)
Subramanian and colleagues1 make the same point as Robinson, but in substantive terms. Simply put, because single-level analyses by definition measure effects at a single level of aggregation (individual or precinct or county, etc.), single-level analyses are subject to severe omitted-variables bias in the presence of multilevel effects. The implication could hardly be plainer: for causal analysis, you need multilevel models when there are multilevel effects. And, as Subramanian and colleagues1 also observe, the ecologic fallacy described by Robinson is but one specific instance (albeit the most prominent one) of the more general set of problems involved in failing to heed the admonition to use multilevel analysis in the face of multilevel effects.
Consider again the example of vote for Wallace in the 1968 Presidential election, and let us ask why it is that Wallace tended to receive more support in areas with higher percentages of blacks. Because the black vote for Wallace was negligible, we know that the answer must lie with others’ vote: it must have been the case that non-blacks were more likely to vote for Wallace when they lived in black districts. In other words, racial context mattered. And, since race also mattered at the individual level—blacks were less likely than others to vote for Wallace—we know that we need data at both the individual and regional levels to give a full account of how race affected the Wallace vote.
Alternatively, we may simply want to describe how much less likely blacks were than others to vote for Wallace. If that is what is meant by ‘individual-level relationship’ or (as in Robinson's title) ‘the behavior of individuals’, individual-level data alone will do. But if we want to know how race affected the Wallace vote, we need multilevel analysis, since the effect of race operated at more than one level. As social research moves increasingly from description to cause, then, multilevel analysis becomes more important, particularly in instances where there are large context effects.
In short, there are substantive explanations for Robinson's observation that aggregate- and individual-level coefficients will almost certainly differ, and those explanations involve the presence of effects at multiple levels. Subramanian and colleagues1 contribute by making this point more explicitly. They also contribute by reminding us that the cross-level fallacy issue cuts both ways: when aggregate- and individual-level associations differ, then neither is a reliable estimator of the other. (But to be fair to Robinson, at the time he was writing the misinterpretation of aggregate-level associations as individual-level associations was by far the more common error.)
Let me close by noting that there is nothing particularly mysterious about the ecologic fallacy and its cousin, the individualist fallacy, when we realize that these fallacies are possible because of omitted-variables bias. Hence Subramanian and colleagues1 note that Robinson's model neglects the important role played by Jim Crow education laws that varied across states, and they obtain very different results when they add those state-level measures to Robinson's analysis of race/nativity and illiteracy. The point is that in enhancing Robinson's model, Subramanian and colleagues1 are not doing anything alien—they are trying to reduce omitted variables bias by adding important variables, as we always try to do in causal modelling in observational research. The only twist is that, with multilevel models, those unmeasured confounders might exist at multiple levels.
 |
References |
|---|
 Top References |
|---|
1 Subramanian SV, Jones K, Kaddour A, Krieger N. Revisiting Robinson: the perils of individualistic and ecologic fallacy. Int J Epidemiol (2009) 38:342–60.
2
Robinson WS. Ecological correlations and the behavior of individuals.
Am Sociol Rev (1950) 15:351–57. (reprinted in Int J Epidemiol
2009;38:337–41).[CrossRef][Web of Science]
3 Schoenberger RA, Segal DR. The ecology of dissent: the southern Wallace vote in 1968. Midwest J Polit Sci (1971) 15:583–86.[CrossRef]
4 Firebaugh G. Seven Rules for Social Research. (2008) Princeton, NJ: Princeton University Press.
5 Goodman L. Some alternatives to ecological correlation. Am J Sociol (1959) 64:610–25.[CrossRef]
6 King G. A Solution to the Ecological Inference Problem. (1997) Princeton, NJ: Princeton University Press.
CiteULike 
Connotea 
Del.icio.us What's this?
Related articles in Int. J. Epidemiol.:
- Response: The value of a historically informed multilevel analysis of Robinson's data
- SV Subramanian, Kelvyn Jones, Afamia Kaddour, and Nancy Krieger
Int. J. Epidemiol. 2009 38: 370-373.[Extract] [Full Text] - Commentary: Individual, ecological and multilevel fallacies
- J Michael Oakes
Int. J. Epidemiol. 2009 38: 361-368.[Extract] [Full Text] - Ecological Correlations and the Behavior of Individuals
- WS Robinson
Int. J. Epidemiol. 2009 38: 337-341.[Extract] [Full Text] - Revisiting Robinson: The perils of individualistic and ecologic fallacy
- S V Subramanian, Kelvyn Jones, Afamia Kaddour, and Nancy Krieger
Int. J. Epidemiol. 2009 38: 342-360.[Abstract] [Full Text]
This article has been cited by other articles:
|
|
 |
|
  S. Subramanian, K. Jones, A. Kaddour, and N. Krieger Erratum Int. J. Epidemiol., June 1, 2009; 38(3): 891 - 894. [Full Text] [PDF]
|
|
|
|
|
|
S. Subramanian, K. Jones, A. Kaddour, and N. Krieger Response: The value of a historically informed multilevel analysis of Robinson's data Int. J. Epidemiol., April 1, 2009; 38(2): 370 - 373. [Full Text] [PDF]
|
|
|
|
|
|
J. Wakefield Multi-level modelling, the ecologic fallacy, and hybrid study designs Int. J. Epidemiol., April 1, 2009; 38(2): 330 - 336. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
