American Sociological Association

Comment: Some Challenges When Estimating the Impact of Model Uncertainty on Coefficient Instability

I once had a colleague who knew that inequality was related to an important dependent variable. This colleague knew many other things, but I focus on inequality as an example. It was difficult for my colleague to know just how to operationalize inequality. Should it be the percentage of income held by the top 10 percent, top 5 percent, or top 1 percent of the population? Should it be based on the ratio of median black income to median white income, or should it be the log of that ratio? Should it be based on the Gini index, or perhaps the Theil index would be better? How would my colleague know which measure of inequality was best? Well, it should certainly have the correct sign based on the theory under discussion. To differentiate between indices with the correct sign, we could rely on the magnitude of the relationship controlled by the other independent variables in the model and what using that measure did to other independent variables of interest. The number of models to be examined in the quest to find the right combination to fit what is known about the correct model is for this researcher immense. With all of these attempts to find the correct model, how much confidence can we have about the magnitude and statistical significance reported in the two to four model results submitted by my colleague for publication? Muñoz and Young this volume, pp. 1–33 henceforth MY) address this sort of question.1 My commentary is meant to be constructive in the context of MY’s thought-provoking paper; it is in the form of questions and observations about their paper and related topics.

Authors

Robert M. O’Brien

Volume

48

Issue

1

Starting Page

34

Ending Page

39