Join Andrew Gelman and his team of experts as they delve into the fascinating world of statistical modeling and its applications, with a particular focus on the social sciences. From Bayesian statistics to multilevel modeling, causal inference, and political science, this weblog covers it all. With contributions from Gelman himself, his students, and esteemed colleagues, get ready for insightful and thought-provoking discussions. Engage in the conversation by sharing your comments, suggestions, and solutions to the open problems they present. Don't miss out on this valuable resource for researchers and enthusiasts alike.

In this weblog, we will report on recent research and ongoing half-baked ideas, including topics from our research groups on Bayesian statistics, multilevel modeling, causal inference, and political science. See Andrew Gelman’s webpage for more background.
We will also post on anything that interests us that relates to statistical ideas and their applications.
Contributing to the blog will be Samantha Cook, Andrew Gelman, and various of Andrew’s students and colleagues who would like to add things.
Comments will be helpful to suggest open problems, to point out mistakes in the work that we post, and to suggest solutions to some of the open problems we raise.

A weblog for research in statistical modeling and applications, especially in social sciences

Discover the surprising truth about the Electoral College and its impact on voters in small states in this eye-opening article by Andrew. Unveiling the flawed notion that larger states have more voting power, Andrew dives into the probabilities and statistics behind decisive voting and electoral influence. Uncover why swing states and small swing states play a crucial role in shaping election outcomes, and how your own voting power may be more substantial than you think. Prepare to question preconceived notions as this article challenges conventional wisdom with insightful analysis and thought-provoking findings.

It is well known that the Electoral College favors small states: every state, no matter how small, gets at least 3 electoral votes, and so small states have more electoral votes per voter. This “well known fact” is, in fact, true.
To state this slightly more formally: if you are a voter in state X, then the probability that your vote is decisive in the Presidential election is equal to the probability that your vote is decisive within your state (that is, the probability that your state would be exactly tied without your vote), multiplied by the probability that your state’s electoral votes are decisive in the Electoral College (so that, if your state flips, it will change the electoral vote winner), if your state were tied.
If your state has N voters and E electoral votes, the probability that your state is tied is approximately proportional to 1/N, and the probability that your state’s electoral votes are necessary is approximately proportional to E. So the probability that your vote is decisive–your “voting power”–is roughly proportional to E/N, that is, the number of electoral votes per voter in your state.
A counterintuitive but wrong idea
The point has sometimes been obscured, unfortunately, by “voting power” calculations that purportedly show that, counterintuitively, voters in large states have more voting power (“One man, 3.312 votes,” in the oft-cited paper of Banzhaf, 1968). This claim of Banzhaf and others is counterintuitive and, in fact, false.
Why is the Banzhaf claim false? The claim is based on the same idea as we noted above: voting power equals the probability that your state is tied, times the probability that your state’s electoral votes are necessary for a national coalition. The hitch is that Banzhaf (and others) computed the probability of your state being tied as being proportional to 1/sqrt(N), where N is the number of voters in the state. This calculation is based (explicitly or implicitly) on a binomial distribution model, and it implies that elections in large states will be much closer (in proportion of the vote) than elections in small states.

Above is the result of the oversimplified model. In fact, elections in large states are only very slightly closer than elections in small states. As a result, the probability that your state’s election is tied is pretty much proportional to 1/N, not proportional to 1/sqrt(N). And as a result of that, your voting power is generally more in small states than in large states.
Realistically . . .
Realistically, voting power depends on a lot more than state size. The most important factor is the closeness of the state. Votes in so-called “swing states” (Florida, New Mexico, etc.) are more likely to make a difference than in not-so-close states such as New York.

Above is a plot of “voting power” (the probability that your vote is decisive) as a function of state size, based on the 2000 election. These probabilities are based on simulations, taking the 2000 election and adding random state, regional, and national variation to simulate the uncertainty in state-by-state outcomes.

And above is a plot showing voting power vs. state size for a bunch of previous elections. These probabilities are based on a state-by-state forecasting model applied retroactively (that is, for each year, the estimated probability of tie votes, given information available before the election itself).
The punch line: you have more voting power if you live in a swing state, and even more voting power if you live in a small swing state. And, if you’re lucky, your voting power is about 10^(-7), that is, a 1 in 10-million chance of casting a decisive vote.
Here’s the full paper to appear in the British Journal of Political Science (joint work with Jonathan Katz and Joe Bafumi)
Go here for more details on the statistical models (joint work with Jonathan Katz and Francis Tuerlinckx)
Actually, though, it’s still rational for you to vote, at least in many of the states.

The Electoral College favors voters in small states

Exploring the rationality behind voting, Andrew presents an intriguing cost-benefit analysis that challenges conventional explanations. While the probability of one vote changing the election outcome is minuscule, Andrew argues that the impact of a preferred candidate winning can have far-reaching benefits for the country as a whole. Drawing on empirical evidence and social motivations, the article invites readers to reevaluate their perception of voting and consider the implications for collective well-being. Discover why voting may be more rational than previously believed in this thought-provoking essay.

The chance that your vote will be decisive in the Presidential election is, at best, about 1 in 10 million. So why vote? 
Schematic cost-benefit analysis
To express formally the decision of whether to vote:
U = p*B – C, where
U = the relative utility of going and casting a vote
p = probability that, by voting, you will change the election outcome
B = the benefit you would feel from your candidate winning (compared to the other candidate winning)
C = the net cost of voting
The trouble is, if p is 1 in 10 million, then for any reasonable value of B, the product p*B is essentially zero (for example, even if B is as high as $10000, p*B is 1/10 of one cent), and this gives no reason to vote.
The usual explanation
Actually, though, about half the people vote. The simplest utility-theory explanation is that the net cost C is negative for these people–that is, the joy of voting (or the satisfying feeling of performing a civic duty) outweighs the cost in time of going out of your way to cast a vote.
The “civic duty” rationale for voting fails to explain why voter turnout is higher in close elections and in important elections, and it fails to explain why citizens give small-dollar campaign contributions to national candidates. If you give Bush or Kerry $25, it’s not because you’re expecting a favor in return, it’s because you want to increase your guy’s chance of winning the election. Similarly, the argument of “it’s important to vote, because your vote might make a difference” ultimately comes down to that number p, the probability that your vote will, in fact, be decisive.
Our preferred explanation
We understand voting as a rational act, given that a voter is voting to benefit not just himself or herself, but also the country (or the world) at large. (This “social” motivation is in fact consistent with opinion polls, which find, for example, that voting decisions are better predicted by views on the economy as a whole than by personal financial situations.)
In the equation above, B represents my gain in utility by having my preferred candidate win. If I think that Bush (or Kerry) will benefit the country as a whole, then my view of the total benefit from that candidate winning is some huge number, proportional to the population of the U.S. To put it (crudely) in monetary terms, if my candidate’s winning is equivalent to an average $100 for each person (not so unreasonable given the stakes in the election), then B is about $30 billion. Even if I discount that by a factor of 100 (on the theory that I care less about others than myself), we’re still talking $300 million, which when multiplied by p=1/(10 million) is a reasonable $30.
Some empirical evidence
As noted above, voter turnout is higher in close elections and important elections. These findings are consistent with the idea that it makes more sense to vote when your vote is more likely to make a difference, and when the outcome is more important.
As we go from local, to state, to national elections, the size of the electorate increases, and thus the probability decreases of your vote being decisive, but voter turnout does not decrease. This makes sense in our explanation because national elections affect more people, thus the potential benefit B is multiplied by a larger number, canceling out the corresponding decrease in the probability p.
People often vote strategically when they can (in multicandidate races, not wanting to “waste” their votes on candidates who don’t seem to have a chance of winning). Not everyone votes strategically, but the fact that many people do is evidence that they are voting to make a difference, not just to scratch an itch or satisfy a civic duty.
As noted above, people actually say they are voting for social reasons. For example, in the 2001 British Election Study, only 25% of respondents thought of political activity as a good way to get “benefits for me and my family” whereas 66% thought it a good way to obtain “benefits for groups that people care about like pensioners and the disabled.”
Implications for voting
First, it can be rational to vote with the goal of making a difference in the election outcome (not simply because you enjoy the act of voting or would feel bad if you didn’t vote). If you choose not to vote, you are giving up this small but nonzero chance to make a huge difference.
Second, if you do vote, it is rational to prefer the candidate who will help the country as a whole. Rationality, in this case, is distinct from selfishness.
See here for the full paper (joint work with Aaron Edlin and Noah Kaplan)

Why it’s rational to vote

Andrew delves into the philosophical underpinnings of statistical inference in this thought-provoking essay. He explores the connection between the deductive reasoning of classical statistics and the inductive reasoning of Bayesian statistics. By challenging the assumptions and principles of both approaches, Andrew encourages us to critically evaluate our models and strive for deeper understanding. Whether you're a statistician or simply curious about the philosophy of science, this article will challenge your preconceptions and broaden your perspective on the art of inference.

Is statisticsl inference inductive or deductive reasoning? What is the connection between statistics and the philosophy of science? Why do we care?
The usual story
Schools of statistical inference are sometimes linked to philosophical approaches. “Classical” statistics–as exemplified by Fisher’s p-values and Neyman’s hypothesis tests–is associated with a deductive, or Popperian view of science: a hypothesis is made and then it is tested. It can never be accepted, but it can be rejected (that is, falsified).
Bayesian statistics–starting with a prior distribution, getting data, and moving to the posterior distribution–is associated with an inductive approach of gradually moving forward, generalizing from data to learn about general laws as expressed in the probability that one model or another is correct.
Our story
Our progress in applied modeling has fit the Popperian pattern pretty well: we build a model out of available parts and drive it as far as it can take us, and then a little farther. When the model breaks down, we take it apart, figure out what went wrong, and tinker with it, or else try a radically new design. In either case, we are using deductive reasoning as a tool to get the most out of a model, and we test the model–it is falsifiable, and when it is falsified, we alter or abandon it. To give this story a little Kuhnian flavor, we are doing “normal science” when we apply the deductive reasoning and learn from a model, or when we tinker with it to get it to fit the data, and occasionally enough problems build up that a “new paradigm” is helpful.
OK, all fine. But the twist is that we are using Bayesian methods, not classical hypothesis testing. We do not think of a Bayesian prior distribution as a personal beliefs; rather, it is part of a hypothesized model, which we posit as potentially useful and abandon to the extent that it is falsified.
Subjective Bayesian theory has no place for falsification of prior distributions–how do you falsify a belief? But if we think of the model just as a set of assumptions, they can be falsified if their predictions–our deductive inferences–do not fit the data.
Why does this matter?
A philosophy of deductive reasoning, accompanied by falsifiability, gives us a lot of flexibility in modeling. We do not have to worry about making our prior distributions match our subjective knowledge, or about our model containing all possible truths. Instead we make some assumptions, state them clearly, and see what they imply. Then we try to falsify the model–that is, we perform posterior predictive checks, creating simulations of the data and comparing them to the actual data. The comparison can often be done visually; see chapter 6 of Bayesian Data Analysis for lots of examples.
I associate this “objective Bayes” approach–making strong assumptions and then testing model fit–to the work of E. T. Jaynes. As he has illustrated, the biggest learning experience can occur when we find that our model does not fit the data–that is, when it is falsified–because then we have found a problem with our underlying assumptions.
Conversely, a problem with the inductive philosopy of Bayesian statistics–in which science “learns” by updating the probabilities that various competing models are true–is that it assumes that the true model is one of the possibilities being considered. This can does not fit our own experiences of learning by finding that a model doesn’t fit and needing to expand beyond the existing class of models to fix the problem.
I fear that a philosophy of Bayesian statistics as subjective, inductive inference can encourage a complacency about picking or averaging over existing models rather than trying to falsify and go further. Likelihood and Bayesian inference are powerful, and with great power comes great responsibility. Complex models can and should be checked and falsfied.
Postscript
These ideas are also connected to exploratory data analysis
(see this paper in the International Statistical Review).
Also, Sander Greenland has written on the philosophy of Bayesian inference from a similar perspective–he has less sympathy with Popperian terminology but similarly sees probabilistic inference within a model as deductive (“Induction versus Popper: substance versus semantics” in International Journal of Epidemiology 27, 543-548 (1998).)

Bayes and Popper

Andrew delves into the overrepresentation of small states/provinces in the political systems of various countries, with a particular focus on the United States. He highlights the disparities in electoral votes, Senate seats, and federal spending per capita between large and small states. Andrew investigates the reasons behind this imbalance and explores possible explanations such as the bargaining power of small states during constitution-writing processes and the influence of rural demographics. He also sheds light on the "USA Today effect" and how states are often presented as equals, despite significant population differences.

It is well known that states are overrepresented in the U.S. political system. For example, Wyoming has 0.2% of the U.S. population but has 0.6% of the Electoral College votes for President, and 2% of the U.S. senators; while California has 12% of the population, 10% of the electoral votes, and still only 2% of the senators. To put it another way: Wyoming has 6 electoral votes and 2 senators per million voters, while California has 1.5 electoral votes and 0.06 senators per million voters. There is also a disparity in federal funding; for example, Wyoming received $7200 and California only $5600 in direct federal spending per capita in 2001.
On the whole, the disparity in the Electoral College is pretty minor, but the U.S. Senate disparity is huge: the 21 smallest states have the population of California but 42 Senators compared to California’s two.
Other countries too
We have looked at other countries (Mexico, Canada, Japan, Argentina, Thailand, . . .) and found similar patterns: smaller states (provinces) have more legislative representatives per capita and more spending per capita.
For any country, we can summarize the relationship of representation and population by making a graph, with one dot per state (province), plotting the logarithm of #representatives per capita on the y-axis, and the logarithm of population on the x-axis. The graph typically has a negative slope: larger states within a country tend to have fewer representatives per person.
We then fit a regression line to the data and summarize it by the slope. A slope of 0 is complete fairness with respect to population size: neither small nor large states get more per-capita representation. A slope of -1 is like the U.S. Senate, with a huge imbalance in representation per person. A positive slope would imply that voters in large states are actually overrepresented.
What do the data look like? Slopes are typically negative, ranging from 0 down to -1. Small states consistently get more representatives per person.
Similarly, we make graphs of log(federal spending per capita) vs. log(population) for the states in each country, and we find negative slopes throughout. Typical slopes are between -0.2 and -0.5, so that small states get more funding per person–no surprise, given their overrepresentation.
Why do we care?
Well, it seems unfair, violating the “one-person, one-vote” norm. The spending inequalities represent real money. Also, it’s interesting to see such a strong pattern. If it only occurred in U.S., we’d wonder what was special about our system. Since the small-state bias seems to occur all over, we wonder what general features of political systems make it possible.
How did it happen and how does it persist?
There are lots of possible stories. In countries such as the United States that formed from federations, the small states had some bargaining power in the constitution-writing process. (It has been pointed out, however, that the ratio between the largest and smallest state populations has increased greatly over the lifetime of the U.S.) Similarly, small states have some veto power in the European Union, which may have help explain how they have achieved their overrepresentation in the Council of Ministers.
Small states tend to be more rural and have larger land areas per person, both of which may be sources for federal spending (although this wouldn’t explain the overrepresentation).
The USA Today effect
Another set of reasons for inequality to persist is if it’s not recognized as such. Suppose we were to think of states as persons. Then it makes sense to give each state an equal number of Senators–that’s equal representation, right? From an individualist standpoint, this sounds silly, but in many contexts, the states are presented as equals.
For example, USA Today has a feature with news from each of the 50 states. There are a lot more people in Los Angeles or Chicago than in several of the smallest states, but they manage to find an equal-length snippet from each. It’s not always easy! For example, here’s Idaho’s entry for today, October 15:
Post Falls – Families are protesting the periodic removal of mementos they place on graves at the Evergreen Cemetery. The items include ceramic angels, American flags and baby bracelets. Cemetery sexton Bob Harvego says the city must keep the graves tidy and that the mementos are stored.
A cognitive illusion
To think of it another way, California has 12% of the U.S. population. Suppose it were overrepresented and get 40% of the Senators. It would bother people that almost every other Senator is from California–it would just seem weird, and it would be clear that the state is overrepresented. But, here in the real world, the 21 smallest states–whose total population is less than California’s–get 42 senators, people don’t seem to mind that! These small states have different names, so their overrepresentation is not so obvious.
[no link to a paper here; megumi and the rest of us are still working on the paper summarizing the empirical results and possible explanations]
P.S.  See here and here for a partisan take on this issue.

Overrepresentation of small states/provinces, and the USA Today effect

Delve into the intricate world of criminal justice economics with Andrew as he examines Joanna Shepherd's latest research on the deterrence effect of the death penalty. Discover how Andrew plans to test the sensitivity of Shepherd's findings by exploring changes in model specifications and analyzing county-by-year data. Explore the nuances and complexities of this thought-provoking topic as Andrew strategizes and collaborates with expert colleagues. Get ready for an eye-opening journey into the realm of academic research and data analysis in the realm of criminal justice economics.

Just a quick note on what we’re doing with Shepherd’s paper, and why…
There’s a long standing economics literature (beginning with Ehrlich in 1975) on the question of whether the death penalty deters murders. Since the death penalty is used in some states and not others, and has been used to differing extents over the years, it tends to be treated as a natural experiment. 
However, capital punishment is not implemented at random — be it the political climate, current crime rates, or inherent differences across states, there’s something that drives some states to legalize the death penalty, and others not to. Furthermore, the “deterrent effect” of the death penalty varies by state and time, making it difficult to make a single causal claim of deterrence or the lack thereof. 
Since there are such differences across states and time, the question of deterrence may lend itself to a multilevel model. Most modern papers look at data by state and year, and a multilevel model allows some flexibility in the appropriate level of aggregation.
Joanna Shepherd’s recent paper (available on the Wiki) uses a series of equations to predict murder rates by county and year, as a function of deterrent (and other) measures, and predict the probabilities of arrest, death sentences, and executions, based on murder rates and a number of other factors.
She finds that in some states, executions “deter” crime, in others there is no effect, and in still others, executions “cause” crime. We plan to test the sensitivity of these findings to changes in model specifications.
We don’t have her data yet; at the moment I’m using Stata to play with some state-level data to try and get a handle on her model. The county-by-year data might add more to a multilevel model than state-level data, since the death penalty is legalized by state, but deterrent effects may be felt more locally.
We’re also planning to test some of the other model specifications — the reliance on publicity, differences between high-execution and low-execution states, and so forth. 
I’m meeting with Jeff on Friday, who has some ideas of what we should be testing, and will relay that conversation…

Sensitivity Analysis of Joanna Shepherd’s DP paper

David Samuels, coauthor of a groundbreaking paper on unequal representation, shares insightful comments on the overrepresentation of small states/provinces. He recommends exploring Thies' research on over-representation in Japan and highlights Edward Gibson's work on the consequences of apportionment distortions in Latin America. Samuels also sheds light on the unique approach taken by the United States in implementing a 1-person 1-vote solution. His thought-provoking insights challenge democratic ideals and shed light on the political strategies that shape representation in different countries.

David Samuels of the University of Minnesota, who coauthored a paper with Richard Snyder in 2001 on unequal representation for the British Journal of Political Science (“The Value of a Vote: Malapportionment in Comparative Perspective”), had some helpful comments on our post on overrepresentation of small states/provinces:
Check out Thies’ work on the consequences of over-representation in Japan, if you haven’t seen it already. A paper that appears in the last year in LSQ by two Japanese political scientists (and that won an award from the Legis. Studies section of APSA) was on the same subject and country.
Edward Gibson’s work (and with former grad students – articles in World Politics, etc.) has looked at the consequences of apportionment distortions in Latin America for fiscal policy, e.g. My book, published last year, looks at the relationship between federalism and executive-legislative relations in Brazil, with particular attention to malapportionment of under-populated states. Federalism and Territorial Politics in Latin America.
I’d also again say that the piece Rich and I have in Gibson’s edited volume (Federalism and Territorial Politics in Latin America) might be good food for thought for you as for causal stories. For example, in Latin America the key political dynamic that *generated* malapportionment was not federalism per se (the case of Chile bears this out quite clearly), but that both authoritarian and democratic leaders have sought to manipulate the “rules of representation” to favor their allies. This includes forgoing a decennial census, which has the effect of under-representing any city experiencing population growth over time (Chile), creating 2 new states out of 1 old state (Argentina, Brazil), etc. etc. The US seems unique, or close to it, in that the court system has imposed a 1-person 1-vote solution in the lower chamber of the legislature. In some countries, e.g. the UK, political negotiations lead to a similar outcome (e.g. the last Reform Act), but in many countries, it seems that political-electoral concerns favoring under-represented districts overwhelm lofty democratic notions of the equality of the weight of the vote for all citizens – even in single-chamber legislatures in non-federal systems.
Good luck,
David

Unequal representation: comments from David Samuels

Andrew delves into the challenges of using heterogeneous choice models in statistical inference and their impact on estimating individual choices. Drawing on examples from political science, he highlights how variation in choices can be more intriguing than the determinants themselves. However, he discovers that current estimation techniques are flawed, leading to biased estimates and incorrect standard errors. Even under perfect specifications, bias and measurement errors persist. To overcome these limitations, Andrew suggests exploring alternative methods, specifically Bayesian estimation techniques, to offer a more effective approach to modeling heterogeneous choices. Discover the complexity and potential solutions in this eye-opening essay.

Introduction
Heterogeneous choice model allows researchers to model the variance of individual level choices. These models have their roots in heteroskedasticity (unequal variance) and the problems it creates for statistical inference. In the context of linear regression, unequal variance does not bias the estimates, rather it inflates or underestimates the true standard errors in the model. Unequal variance is more problematic in discrete choice models, such as logit or probit (and their ordered or multinomial variants). If we have unequal variances in the error term of a discrete choice model, not only are the standard errors incorrect, but the parameters are also biased.
Example
The classic political science example involves ambivalence and value conflict. Alvarez and Brehm (1995) modeled the variation in responses to survey questions on abortion to demonstrate that this variation results not from respondents offering ill-informed opinions, but instead is a product of the ambivalence that results from wrestling with a difficut and important choice. This is a case where the variability in the choice is actually more interesting than what determines the choice.
Estimation
Estimation of heterogeneous choice models is fairly straightforward. A researcher that suspects heterogeneous choices can select a set of covariates and model the heterogeneity. However, the properties of these models are not well understood. There is little analytical or empirical evidence how well these models perform.
Monte Carlo Experiments
Using monte carlo experiments, we examine how well these models estimate the parameters used to make inferences about heterogeneous choices. We find that these models are deeply flawed. Not only are these model an ineffective “cure” for heteroskedasticity, the estimates of these models are biased and provide incorrect estimates of the standard errors.
The estimated sampling variability and coverage rates were less than ideal even under a perfect specification. Measurement error in the variance model induced significant amounts of bias, and almost any specification error causes the estimates of both the choice and variance model to be completely unreliable. Even in models where teh variable used to estimate the choice was correlated with the true variable at 0.90, all the parameters were estimated very poorly, being biased by over 60%. And, it is entirely possible to misspecify both the choice and variance model, which should only make the estimates worse.
Conclusion
Of course it is easier to tear down than to build up. To that end, we intend to devise an alternative way to model heterogeneous choices. Bayesian estimation techniques offers such a possibility, with its focus on the variance of all parameters. Such an approach should give us better leverage over the heterogeneity of individual choices

Problems with Heterogeneous Choice Models

Join Morris Fiorina as he debunks the myth of a polarized America in his upcoming discussion on C-SPAN. In an era plagued by political division, Fiorina challenges popular belief and reveals that most Americans actually hold moderate views on major issues. As a senior fellow at the Hoover Institution and professor of political science at Stanford University, Fiorina brings insightful analysis to the table. Don't miss out on this eye-opening talk that challenges our assumptions about the culture war. Tune in this Sunday at 5:15pm and get ready to question everything you thought you knew.

I read that Morris Fiorina will discuss the culture war in America this Sun at 5:15pm on C-SPAN. He may talk about Red/Blue states.
Culture War?: The Myth of a Polarized America by Morris Fiorina
Description: Morris Fiorina challenges the idea that there is a culture war taking place in America. Mr. Fiorina argues that while both political parties and pundits the country is constantly present the country as being divided, the reality is that most people hold middle-of-the-road views on almost all of the major issues. The talk was hosted by Stanford University’s Hoover Institution.
Author Bio: Morris Fiorina is a senior fellow at the Hoover Institution and a professor of political science at Stanford University. His books include “The New American Democracy” and “The Personal Vote: Constituency Service and Electoral Independence.”
Publisher: Pearson Longman 1185 Avenue of the Americas New York, NY 10036

Morris Fiorina on C-SPAN

In this intriguing article, Andrew presents a unique and engaging demonstration for statistics classes that explores the challenges of random sampling. By using a bag of candies and a digital scale, students are tasked with estimating the weight of 100 candies. The results, which are surprisingly lower than expected, lead to a thought-provoking discussion on sampling methods and their implications. Andrew's demonstration serves as a fun and interactive way to introduce important statistical concepts, leaving students with a deeper understanding of the complexities involved in obtaining accurate data.

Here’s a fun demonstration for intro statistics courses, to get the students thinking about random sampling and its difficulties.
As the students enter the classroom, we quietly pass a sealed envelope to one of the students and ask him or her to save it for later. Then, when class has begun, we pull out a digital kitchen scale and a plastic bag full of a variety of candies of different shapes and sizes, and we announce:
This bag has 100 candies, and it is your job to estimate the total weight of the candies in the bag. Divide into pairs and introduce yourself to your neighbor. [At this point we pause and walk through the room to make sure that all the students pair up.] We’re going to pass this bag and scale around the room. For each pair, estimate the weight of the 100 candies in the bag, as follows: Pull out a sample of 5 candies, weigh them on the scale, write down the weight, put them candies back in the bag and mix them (no, you can’t eat any of them yet!), and pass the bag and scale to the next pair of students. Once you’ve done that, multiply the weight of your 5 candies by 20 to create an estimate for the weight of all 100 candies. Write that estimate down (silently, so as not to influence the next pair of students who are taking their sample). [As we speak, we write these instructions as bullet points on the blackboard: “Draw a sample of 5,” “Weigh them,” etc.]
Your goal is to estimate the weight of the entire bag of 100 candies. Whichever pair comes closest gets to keep the bag. So choose your sample with this in mind.
We then give the bag and scale to a pair of students in the back of the room, and the demonstration continues while the class goes on. Depending on the size of the class, it can take 20 to 40 minutes. We just cover the usual class material during this time, keeping an eye out occasionally to make sure the candies and the scale continue to move around the room.
When the candy weighing is done (or if only 15 minutes remain in the class), we continue the demonstration by asking each pair to give their estimate of the total weight, which we write, along with their first names, on the blackboard. We also draw a histogram of the guesses, and we ask whether they think their estimates are probably too high, too low, or about right.
We then pass the candies and scale to a pair of students in front of the class and ask them to weigh the entire bag and report the result. Every time we have done this demonstration, whether with graduate students or undergraduates, the true weight is much lower than most or all of the estimates—so much lower that the students gasp or laugh in surprise. We extend the histogram on the blackboard to include the true weight, and then ask the student to open the sealed envelope, and read to the class the note we had placed inside, which says, “Your estimates are too high!”
We conclude the demonstration by leading a discussion of why their estimates were too high, how they could have done their sampling to produce more accurate estimation, and what analogies can they draw between this example and surveys of human populations. When students suggest doing a “random sample,” we ask how they would actually do it, which leads to the idea of a sampling frame or list of all the items in the population. Suggestions of more efficient sampling ideas (for example, picking one large candy bar and four smaller candies at random) lead to ideas such as stratified sampling.
—
When doing this demonstration, it’s important to have the students work in pairs so that they think seriously about the task.
It’s also most effective when the candies vary greatly in size: for example, take about 20 full-sized candy bars, 30 smaller candy bars, and 50 very small items such as individually-wrapped caramels and Life Savers.

A fun demo for statistics class

Stat Modeling

Atul Gawande wrote an interesting article in the New Yorker a couple months ago on the varying effectiveness of medical centers around the U.S. in treating cystic fibrosis (CF), a genetic disease that reduces lung functioning in children.  Apparently, the basic treatment is the same everywhere–keep the kid’s lungs as clear as possible, from an early age–but some hospitals are much better at it than others:  “In 2003, life expectancy with CF had risen to thirty-three years nationally, but at the best center it was more than forty-seven.”
I’ll discuss the article and give a long quote from it, then give my thoughts.
Agressive doctors
Gawande goes to an average-performing center (in Cincinnati) and the nuber-one center (in Minneapolis) and inteviews and observes people at both places.  The difference, at least in how he reports it, is that in the top center the doctors are super-agressive and really involved with each patient, getting to know them individually and figuring out what it takes to get them on the treatment:


The director of Fairview-University Children’s Hospital’s cystic-fibrosis center for almost forty years has been none other than Warren Warwick, the pediatrician who had conducted the study of LeRoy Matthews’s suspiciously high success rate. Ever since then, Warwick has made a study of what it takes to do better than everyone else. The secret, he insists, is simple, and he learned it from Matthews: you do whatever you can to keep your patients’ lungs as open as possible. Patients with CF at Fairview got the same things that patients everywhere did–some nebulized treatments to loosen secretions and unclog passageways (a kind of mist tent in a mouth pipe), antibiotics, and a good thumping on their chests every day. Yet, somehow, everything he did was different.
In the clinic one afternoon, I joined him as he saw a seventeen-year-old high-school senior named Janelle, who had been diagnosed with CF at the age of six and had been under his care ever since. She had come for her routine three-month checkup. She wore dyed-black hair to her shoulder blades, black Avril Lavigne eyeliner, four earrings in each ear, two more in an eyebrow, and a stud in her tongue. Warwick is seventy-six years old, tall, stooped, and frumpy-looking, with a well-worn tweed jacket, liver spots dotting his skin, wispy gray hair–by all appearances, a doddering, mid-century academic. He stood in front of Janelle for a moment, hands on his hips, looking her over, and then he said, “So, Janelle, what have you been doing to make us the best CF program in the country?”
“It’s not easy, you know,” she said.
They bantered. She was doing fine. School was going well. Warwick pulled out her latest lung-function measurements. There’d been a slight dip, as there was with Alyssa. Three months earlier, Janelle had been at a hundred and nine per cent (she was actually doing better than normal); now she was at around ninety per cent. Ninety per cent was still pretty good, and some ups and downs in the numbers are to be expected. But this was not the way Warwick saw the results.
He knitted his eyebrows. “Why did they go down?” he asked.
Janelle shrugged.
Any cough lately? No. Colds? No. Fevers? No. Was she sure she’d been taking her treatments regularly? Yes, of course. Every day? Yes. Did she ever miss treatments? Sure. Everyone does once in a while. How often is once in a while?
Then, slowly, Warwick got a different story out of her: in the past few months, it turned out, she’d barely been taking her treatments at all.
He pressed on. “Why aren’t you taking your treatments?” He appeared neither surprised nor angry. He seemed genuinely curious, as if he’d never run across this interesting situation before.
“I don’t know.”
He kept pushing. “What keeps you from doing your treatments?”
“I don’t know.”
“Up here”–he pointed at his own head–“what’s going on?”
“I don’t know,” she said.
He paused for a moment. And then he began speaking to me, taking a new tack. “The thing about patients with CF is that they’re good scientists,” he said. “They always experiment. We have to help them interpret what they experience as they experiment. So they stop doing their treatments. And what happens? They don’t get sick. Therefore, they conclude, Dr. Warwick is nuts.”
“Let’s look at the numbers,” he said to me, ignoring Janelle. He went to a little blackboard he had on the wall. It appeared to be well used. “A person’s daily risk of getting a bad lung illness with CF is 0.5 per cent.” He wrote the number down. Janelle rolled her eyes. She began tapping her foot. “The daily risk of getting a bad lung illness with CF plus treatment is 0.05 per cent,” he went on, and he wrote that number down. “So when you experiment you’re looking at the difference between a 99.95-per-cent chance of staying well and a 99.5-per-cent chance of staying well. Seems hardly any difference, right? On any given day, you have basically a one-hundred-per-cent chance of being well. But”–he paused and took a step toward me–“it is a big difference.” He chalked out the calculations. “Sum it up over a year, and it is the difference between an eighty-three-per-cent chance of making it through 2004 without getting sick and only a sixteen-per-cent chance.”
He turned to Janelle. “How do you stay well all your life? How do you become a geriatric patient?” he asked her. Her foot finally stopped tapping. “I can’t promise you anything. I can only tell you the odds.”
In this short speech was the core of Warwick’s world view. He believed that excellence came from seeing, on a daily basis, the difference between being 99.5-per-cent successful and being 99.95-per-cent successful. Many activities are like that, of course: catching fly balls, manufacturing microchips, delivering overnight packages. Medicine’s only distinction is that lives are lost in those slim margins.
And so he went to work on finding that margin for Janelle. Eventually, he figured out that she had a new boyfriend. She had a new job, too, and was working nights. The boyfriend had his own apartment, and she was either there or at a friend’s house most of the time, so she rarely made it home to take her treatments. At school, new rules required her to go to the school nurse for each dose of medicine during the day. So she skipped going. “It’s such a pain,” she said. He learned that there were some medicines she took and some she didn’t. One she took because it was the only thing that she felt actually made a difference. She took her vitamins, too. (“Why your vitamins?” “Because they’re cool.”) The rest she ignored.
Warwick proposed a deal. Janelle would go home for a breathing treatment every day after school, and get her best friend to hold her to it. She’d also keep key medications in her bag or her pocket at school and take them on her own. (“The nurse won’t let me.” “Don’t tell her,” he said, and deftly turned taking care of herself into an act of rebellion.) So far, Janelle was O.K. with this. But there was one other thing, he said: she’d have to come to the hospital for a few days of therapy to recover the lost ground. She stared at him.
“Today?”
“Yes, today.”
“How about tomorrow?”
“We’ve failed, Janelle,” he said. “It’s important to acknowledge when we’ve failed.”
With that, she began to cry.
Warwick’s combination of focus, aggressiveness, and inventiveness is what makes him extraordinary. He thinks hard about his patients, he pushes them, and he does not hesitate to improvise. Twenty years ago, while he was listening to a church choir and mulling over how he might examine his patients better, he came up with a new stethoscope–a stereo-stethoscope, he calls it. It has two bells dangling from it, and, because of a built-in sound delay, transmits lung sounds in stereo. He had an engineer make it for him. Listening to Janelle with the instrument, he put one bell on the right side of her chest and the other on her left side, and insisted that he could systematically localize how individual lobes of her lungs sounded.
He invented a new cough. It wasn’t enough that his patients actively cough up their sputum. He wanted a deeper, better cough, and later, in his office, Warwick made another patient practice his cough. The patient stretched his arms upward, yawned, pinched his nose, bent down as far as he could, let the pressure build up, and then, straightening, blasted everything out. (“Again!” Warwick encouraged him. “Harder!”)
He produced his most far-reaching invention almost two decades ago–a mechanized, chest-thumping vest for patients to wear. The chief difficulty for people with CF is sticking with the laborious daily regimen of care, particularly the manual chest therapy. It requires another person’s help. It requires conscientiousness, making sure to bang on each of the fourteen locations on a patient’s chest. And it requires consistency, doing this twice a day, every day, year after year. Warwick had become fascinated by studies showing that inflating and deflating a blood-pressure cuff around a dog’s chest could mobilize its lung secretions, and in the mid-nineteen-eighties he created what is now known as the Vest. It looks like a black flak jacket with two vacuum hoses coming out of the sides. These are hooked up to a compressor that shoots quick blasts of air in and out of the vest at high frequencies. (I talked to a patient while he had one of these on. He vibrated like a car on a back road.) Studies eventually showed that Warwick’s device was at least as effective as manual chest therapy, and was used far more consistently. Today, forty-five thousand patients with CF and other lung diseases use the technology.
Like most medical clinics, the Minnesota Cystic Fibrosis Center has several physicians and many more staff members. Warwick established a weekly meeting to review everyone’s care for their patients, and he insists on a degree of uniformity that clinicians usually find intolerable. Some chafe. He can have, as one of the doctors put it, “somewhat of an absence of, um, collegial respect for different care plans.” And although he stepped down as director of the center in 1999, to let a protege, Carlos Milla, take over, he remains its guiding spirit. He and his colleagues aren’t content if their patients’ lung function is eighty per cent of normal, or even ninety per cent. They aim for a hundred per cent–or better. Almost ten per cent of the children at his center get supplemental feedings through a latex tube surgically inserted into their stomachs, simply because, by Warwick’s standards, they were not gaining enough weight. There’s no published research showing that you need to do this. But not a single child or teen-ager at the center has died in years. Its oldest patient is now sixty-four.

My thoughts as a statistician
As a statistician, I wonder:  could we measure this agressive approach to treatment?  That is, not just measure the overall effectiveness (higher life expectancy, better lung functioning, and so forth), but also have a measure of “agressiveness” and see if it correlates with the objective outcome measures.  Would this approach work in other hospitals and for other diseases, and if so, how could it be encouraged?
My thoughts as a teacher
As a teacher, I wonder if this agressive strategy could be used for teaching.  The number-one challenge for me is getting students to work on their homeworks.  Would a serious, all-out attack, pressing students every week on what they know and don’t know, do the job?  Would it be an effective tool for someone like me, who knows the material but can’t always motivate students to take that extra step to learn it themselves?  Right now, we’re just satistfied with mediocrity in our teaching, and that doesn’t make me happy.
Maybe such teaching practices are out there already and I’m not aware of them.  In writing my book about statistics teaching, I read about a lot of studies demonstrating the effectiveness of student-focused teaching.  My guess is that, to start with, one would have to have a clearly-defined syllabus, with similar exams and expectations every year.  Right now, it can be hard for us to motivate students partly because we don’t know ourselves exactly what will be on the final exam.
Anyway, I’d be interested in doing this, at least with one of my classes, but I think I’d need some help getting it to work.  Probably the same is true of the doctors and the medical therapies.

Agressive treatment, agressive teaching

Carrie McLaren of Stay Free magazine had a self-described “rant” about Blink, the new book by science writer Malcolm Gladwell.  I’ll give Carrie’s comments below, but my interest here isn’t so much in Gladwell’s book (which seems really cool) or Carrie’s specific comments (which are very thought-provoking, and she also points to this clarifying discussion by Gladwell and James Surowecki in Slate magazine).
Political ideology and attitudes toward technology
Right now, though, I’m more interested in what these exchanges reveal about the intersections of political ideology and attitudes toward technology.  Historically, I think of technology as being on the side of liberals or leftists (as compared with conservatives who would want to stick with the old ways).  Technology = “the Enlightenment” = leftism, desire for change, etc.  Even into the 20th century, I’d see this connection, with big Soviet steel factories and New Deal dams.  But then, in the 1960s and 1970s?, it seems to me there was a flip, in which technology is associated with atomic bombs, nuclear power, and other things that are more popular on the right than on the left.  The environmentalist left has been more skepical about technological solutions.  In another area of scientific debate, right-leaning scientists have embraced sociobiology and related ideas of bringing genetics into social policy.
But…perhaps recently things have switched back?  In battles over the teaching of evolution, it is the liberals who are defending the scientific method and conservatives who are holding back, wanting to respect local culture rather than scientific universals.  Similarly with carbon dioxide and climate change.
But, again, I’m not trying here to argue the merits of any of these issues but rather to ask whether it is almost a visceral thing, at any point in time, with one’s political allegiances being associated with a view of science.
Is Gladwell’s argument inherently anti-rational?  Is anti-rationality conservative?
This is what I saw in Carrie’s posting on Gladwell.  She was irritated by his use of scientific studies to support a sort of irrationalism–a favoring of quick judgments instead of more reasoned analyses.  From this perspective, Gladwell’s apparent advocacy of unconscious decisions is a form of conservatism.  (His position seems more nuanced to me, at least as evidenced in the Slate interview–where he suggests sending police out individually instead of in pairs so they won’t be emboldened to overreact–but perhaps Carrie’s take on it is correct in the sense that she is addressing the larger message of the book as it is perceived by the general public, rather than any specific attitudes of Gladwell.)
Rationality and ideology
As a larger issue, in the social sciences of recent decades, I think of belief in rationality and “rational choice modeling” as conservative, both in the sense that many of the researchers in this area are politically conservative and in the sense that rationality is somehow associated with “cold-hearted” or conservative attitudes on cost-benefit analyses.  But at the same time, quantitative empirical work has been associated with left-leaning views–think of Brown v. Board of Education, or studies of income and health disparities.  There’s a tension here, because in the social sciences, the people who can understand the technical details of empirical statistical work are the ones who can understand rational choice modeling (and vice versa).  So I see all this stuff and keep getting bounced back and forth.
(I’m sure lots has been written about this–these ideas are related to a lot of stuff that Albert Hirschman has written on–and I’d appreciate relevant references, of course.  Especially to empirical studies on the topic.)

Carrie’s words
OK, now, as promised, here are Carrie’s comments:

I hope you’ll pardon me while I rant for a minute. [Just to be clear:  that’s Carrie ranting, not me!] New Yorker writer Malcolm Gladwell (The Tipping Point) has a new book out, BLINK, that has become the talk of the town, even prompting one reviewer — Farhad Manjoo of Salon — to state that, “You won’t find a reader who doesn’t at least like Gladwell” and “There’s just no arguing with Gladwell.”
I’d like to know what planet Mr. Manjoo is living on; Gladwell’s work ALWAYS makes people want to argue. As I’ve written here before, his writing follows a simple formula: put forth a counterintuitive argument, then cleverly select points that advance this claim while ignoring and obscuring those that don’t.
I haven’t read the book in question, so you can take this all with a grain of salt, but the premise alone is preposterous: Gladwell claims that “rapid cognition”–“the kind of thinking that happens in the blink of an eye”–is underappreciated. As Gladwell writes, “I think the Rapid Cognition Model needs to be taken far more seriously–that it’s smarter and more sophisticated and certainly more influential than we generally give it credit for.”
Oh, really? What about the advertising industry, which does nothing if not appreciate humanity’s ability to make unconscious, split-second decisions (and profit from them). Every year, marketers pour billions of dollars into researching and exploiting “blink.”
What about the recent election of a president who acted on “gut instinct” over a man noted for careful deliberation? What about the widespread assumption that it’s important to make a good first impression… or, for that matter, the belief in love at first sight?
Gladwell devotes a chunk of his book to the work of the John Gottman, who videotapes couples and says that within 15 minutes he can tell with 90 percent accuracy whether the couple will be married 15 years later. According to Gladwell, Gottman’s abilities illustrate the power of blink. But Gottman’s work could just as well illustrate the power of careful, deliberative analysis. I first heard about the Gottman Institute on NPR’s This American Life; in that story, Gottman discussed how he acquired his ability to read couples through extensive trial and error. It took him over a decade of watching and analyzing to get to a point where he figured things out quickly. It seems to me that this gets to the heart of the problem with touting blink: at least a solid part of its strength is dependent on the kind of analysis that Gladwell suggests is overrated.
The very reason that Gottman’s work interests us in the first place is because it’s so unusual, the exception to the rule. The truth is that most of us aren’t very good at knowing whether our own relationships will last, let alone those of our peers. Yet Gladwell maintains that the power of blink is democratic, as useful for lay persons as experts. If that’s the case, why is the divorce rate for people who fall in love at first sight no better than those who trod a slow-moving path?
It’s also really hard to swallow Gladwell’s love of blink in light of its role in the social stereotypes that play against the female, dark-skinned, disabled, or physically unattractive among us. Gladwell and his New Yorker colleague James Surowiecki debate this point in an enlightening Slate article. )
To make his case, Gladwell discusses the hiring practices of top orchestras. For years, such orchestras, which conducted open auditions, overwhelmingly selected male performers. But in the 1980s, as Gladwell writes, orchestras “started putting up screens in audition rooms, so that the committee could no longer see the person auditioning. And immediately — immediately! — orchestras started hiring women.”
Might this indicate that relying on quick impressions isn’t such a good thing? After all, it suggests that committee members who had relied on first impressions were likely to assume a female player wasn’t very good. Gladwell’s retort: people rely on their biases regardless of how quickly they make a decision. The problem, he suggests, is bias, not the style (or speed) of decision-making. To bolster his point, he sites the overwhelming presence of tall men heading up corporations. Even very deliberate decisions, he points out, reflect bias.
But this reasoning is ridiculous. The fact that reasoned decisions often reflect bias doesn’t mean that reasoning can’t help minimize it. When you eliminate reasoning and deliberation, you eliminate even the chance of countering biased first impressions.
Gladwell’s solution is no solution at all: “We can put up the equivalent of screens. We can find ways of editing out nonessential information.” When you consider that we form prejudices based on a person’s name, skin color, voice, height, gender, medical history, and appearance, the equivalent of screens would be a soundproof, windowless blackbox.
I’m not saying Gladwell is a bad writer, or that none of his points have merit. I think his skills lie precisely where Farhad Manjoo denies them: in getting readers to argue and discuss. He’s also good at weaving engaging narratives. But, for me, his penchant for overselling arguments–and for concealing significant counterpoints–overshadows his obvious talents.
Gladwell’s thesis would be more accurate in stating that split-second decision-making isn’t worthless — that it can at times be channeled effectively, and that knowing when to do so is key. But that argument sounds a lot less sexy. At any rate, it wouldn’t make for a Malcolm Gladwell book. –CM

P.S.  Since I know Carrie McLaren and I don’t know Malcolm Gladwell, I’m referring to them as “Carrie” and “Gladwell”, since that seems right.  Just in case you were wondering why I was inconsistent with the first and last names.

Rationality and ideology:   Carrie McLaren’s reactions to Malcolm Gladwell

Eric Johnson (a psychologist at the Columbia Business School) spoke today at the Decision Sciences seminar.
A fascinating talk
His topic was “decisions as memory” (maybe i’m getting the exact words wrong here), and the key idea was that, in the process of making a decision, a person queries his or her memory, thinking of good and bad aspects of different decision options.  There’s lots of research on memory that covers all sorts of artifacts (for example, when you remember one item, you will be led to similar items).  The idea of this research program is that, if memory is a key part of judgment and decision making, then many of the weird (or at least, non-normative) aspects of decision making–which have been studied by Kahneman, Tversky, and others over the years–can maybe be explained at a more cognitively basic level as quirks of how we remember things and how we access these memories.
My comments/questions
I had a bunch of comments on the talk (which will probably be incomprehensible unless you were there, but it’s helpful for me to put them down):

– Many of his examples were consumer purchases (buying a car, paying for a mug, deciding whether to eat dessert at a restaurant).  He also said that the sort of tradeoffs people have are “too many things they want to do.”  Later on he discussed the choice of which candidate to vote for in an election.  I suspect there are big differences between self-focused consumer decisions and other-focused decisions about political preferences.
– Simmilarly, he referred to the “cost to buy that product, make that commute, prefer that policy.”  Once again, I see a difference between buying a product (which involves personal costs and benefits) and choosing among policies (no personal costs, and with larger societal effects).
From a psychological perspective, I don’t have a problem with what Johnson was saying.  All these decisions might involve accessing memories, so there might be important similarlities in how we think about these questions.  But at some level it seems inappropriate to use the language of personal cost and choice to describe policy decisions.
– In discussing people’s willingness to donate organs upon their deaths, he referred to some suggestions that have been made to set up a market for human organs.  He said that “a market implies that people have implicit preferences.”  I don’t see why he said that.  A market is a forum for buying and selling.  Why are implicit preferences necessary?  Or is he saying that a market would collapse if preferences are not implicit?  I don’t understand.
– Also in discussing organ donation, he said that the reason people check off the “I’ll be a donor” box is in anticipation of their future happiness.  He also said that “the benefits are the good feeling about being a donor.”  When I checked off that box (or however I consented), I thought the benefits are that someone’s live might be saved after I die.  Or maybe the benefit is just the aesthetic satisfaction of recycling.  I don’t know exactly, but I wouldn’t really say it gave me a good feeling, or that I did it because I thought it would make me happy.
Once again, I’m fascinated by the idea of memory-querying being a key part of decision making.  But I’m skeptical of the hedonistic framework being applied so indiscriminately.
– At some point during the talk I was thinking about Dave Krantz’s statement that hard decisions involve tradeoffs–tradeoffs between decision options, and tradeoffs between attributes of any given decision.  Johnson’s theory didn’t seem to capture that struggle-in-my-mind that I feel when making a decision.  (But, again, the theory has a really cool, unifying explanation of cognitive biases such as “anchoring.”)
– Johnson refes to decisions as “predicitons of future utility.”  I don’t understand why he uses the term “utility.”  In the decision-analytic literature, I always thought that “utility” is defined as that thing which, when averaged over, describes a set of consistent decisions (as in Neumann’s utility theory, where the very existence of utilities is derived from a set of axioms of consistent preferences).  Since Johnson is working outside the “expected utility” framework, I don’t really know what he means when he talks about “utility” (or, for that matter, anticipated future utility). 
– In describing a buy-the-mug experiment, he characterized the “endownment effect” by saying: “that’s a disturbing effect for economists.”  Perhaps true–I expect Johnson knows more about economists’ views than I do–but if true, I’m disturbed that it’s a disturbing effect.  I would think that economists’ main goal would be to describe and understand our commercial behavior, buying and selling, working, poverty and wealth, etc.  Learning about the endowment effect should allow them to understand these things better–why would it be disturbing?
– He said something about the virtue of “choice” in retirement plans, which I didn’t quite follow because earlier he had said that people tend to choose the default retirement plan–which would seem to imply that most people don’t want choice in that aspect of their lives (or, at least, that they don’t want choice so badly that they would actually exercise it). In my experience, there has never been more freedom of choice when it comes to retirement planning. There are many people who choose to live at home during retirement, wanting to stay in their own homes and preferring to be in their own space. They are simply happy with downsizing to a smaller property if they need to do so. However in recent years due to the lack of retirement homes or affordable properties, many people choose to either remain in the homes they have been in for years or go on to assisted living or nursing homes. People can go for great services like assisted living lincoln ne facilities or a wide range of other options available. For example, whereas, of course, there are always going to be people who do choose the default option, I know of quite a few people who have gone on to open individual retirement accounts. You can learn more about IRAs by checking out this useful IRA calculator on the SoFi website. Ultimately, if you are reaching retirement age or simply planning for the future, it is fundamental that you consider all of your options. Furthermore, while it is true that Employee Benefits help people decide on a retirement plan that works for them, there are other factors that should be taken into account such as personal circumstances and overall health.
– Johnson is a professor of marketing.  Some of the studies he described have the flavor of “subliminal advertising.”  For example, changing the background of a web site affected what products people wanted to buy.  Now, fundamentally, I don’t see anything wrong with this sort of manipulation–after all, the website has to have some background, so why not set it to get what you want.  But at the same time there’s something vaguely creepy about it, no?
– From a political scientist’s perspective, this work reminded me of the studies of issue framing.  The research goal would be to connect “framing” with “memory,” I guess.
Johnson et al.’s papers
The papers (by Eric Johnson, Elke Weber, and Daniel Goldstein) underlying the talk are here and here.
P.S.  Links don’t work anymore!

Thoughts on Eric Johnson’s talk

I had a little dialogue with Meredith Rolfe after reading her papers on political participation and social networks:

AG:
You write, “why would a rational individual participate in an election in which the chance of her vote influencing the outcome of the election is essentially zero?”
Actually, there is a good reason.  Basically, the evidence all points to the idea that people vote for what they perceive is beneficial to the country–and the stakes can be high enough that it is rational to make that vote.  The ideas of “social preference” in our paper seem related to your ideas of social networks (although you are working at a more sophisticated level than we are).
Just thought you might be interested in these connections–also, I didn’t want you to dismiss the rational explanation so quickly.  As we discuss in our paper, “rationality” is often (and unnecessarily) conflated with “selfishness”.
MR:
I do start (at least in my own mind!) with the assumption that voter turnout is largely a product of the perception that turnout is good for the all of the citizens in a population (like you, I try to avoid the functionalist fallacy by stressing the perceived benefits of voting, instead of the actual benefits.)
In my work, though, I focus not on the social preferences themselves, but on conditional compliance with this widespread belief that voting is good for the country. I assume that about 10-15% of the population is willing to contribute unconditionally, or vote regardless of what the people around them are doing (these are, in essence, the people who may simply weigh up social benefits and personal costs and cooperate). The vast majority of citizens, though, are simply responding to the decisions of those around them, with the importance of the election being a primary determinant of how many people they will discuss politics with. In other words, their decisions are conditional (either linear/ “fair”, or non-linear/ “conformist”.)
One question I had was how a social preferences model would deal with the fact that people can acknowledge that something is good for the group, and still not do it (most non-voters claim that voting is always the right thing to do, and most players in social dilemmas claim that it is best to contribute at least half of a personal endowment to the group).
Also, in an experimental situations, it is possible to get a person to change the amount they are willing to contribute to the group without changing the benefit to the group (N or the $ amount/importance of election), but by changing their perceptions of who will find out about their decision, or the amount that they expect/observe to be given by Others?
Do you think it is possible to generate conditional decision-making from a dual utility model? I don’t believe that conditional cooperators are irrational, either in the sense of ordered preferences, or, I suspect, material self-interest as well. However, it is less clear to me in the short term what the logical or analytical path would be between social benefits and conditional responses in local (versus global) social networks. That is why I chose to dock the model to empirical evidence instead of a model of rational preferences.
In the next few years, I have in the back of my mind playing around with either an axelrod-esque competition where groups of strategies have to compete with each other (funny that the latest round of his competition was actually won by a group of strategies!) or allowing single players to play n-player games within social networks or maybe even trying to see how far I could push the notion of correlated equilbria. I’m hoping that these efforts would produce an adaptively rational mooring for the conditional cooperation model, but who knows…
On to the topic of network models:
MR:
Are you familiar with some of the p*/exponential random graph models? These models focus on the ways that triads (not dyads) deviate from random expectations, but still with some eye to looking at the role of homophily/individual characteristics in friendship formation. I’d be interested in your observations of how it relates to the approach you propose. Tom Snijders is the guy to check out if you aren’t familiar with this relatively new body of work.
AG:
I wasn’t familiar with the random graph models you mention.  I think that for problems such as social/political polarization, the issue is not so much the structure of the graph but rather how certain “labels” (e.g., michaels, or american indians, or liberals or conservatives) are distributed in that network.
I took a look at your paper on conditional decision making.  It’s an intersting topic and I like the idea of it being considered from a more political-sciencey perspective.  My big idea here is that the news media should be considered as part of the network.  It’s not all about people talking with their friends.
MR:
You’ll see what you think about the ERG models in terms of the underlying statistical assumptions. Technically, you can use them to estimate the effects of categorical variables on the way friendships are distributed.  What I like about your approach is that it is more suitable to use with survey data (or the Killworth/Bernard/McCarthy populations type data) instead of hard to collect data on complete populations. I think that is really important for political scientists who rely heavily on survey data.
I’m pretty torn on the issue of the media. It is obviously an issue.   I saw a recent presentation by Jamie Druckman that was pretty strong in the claims for influence of media on candidate choices and descriptions that really raised some questions for me. Plus, my husband is also a media guy, he looks at how media imagery affects racial attitudes. But, what do you do with the fact that media coverage is designed to attract viewers? That politicians take stands to be popular? But you know, you may be very right, by putting media into the “network” it could both be influential and be influenced by the people it was connected to, hmmm… That is actually a very, very cool idea. I’ll think about it more.

More on social networks and voting

Aleks points to this blog entry from “HedgeFundGuy” on bias in decision making.  HedgeFundGuy passes on a report that finds that people’s opinions are strongly biased by their political leanings, then he gives his take on the findings–he thinks that this so-called bias isn’t really a problem, it’s just evidence of reasonable Bayesian thinking.
I’ll first copy out what HedgeFundGuy had to say (including his own copy of the report of the study), then give my take, which is slightly different than his.

HedgeFundGuy writes:

A recent Miami Herarld article on some academic research on bias is quite illuminating. Because it’s a registration required link (ugh) I’ll snip the best parts.

Drew Westen is a professor of psychology at Emory University, and author of a new and still-unpublished study testing whether people make decisions based on bias or fact. Bias won hands down.
In a key scenario, respondents were lead to believe a soldier was accused of torturing people at Abu Ghraib prison in Iraq. The fictional soldier claimed to have been following orders from superiors who told him the Geneva Convention had been suspended. He supposedly wanted to subpoena President Bush and Defense Secretary Donald Rumsfeld to prove his case. Respondents were asked if he should have that right.
Some were presented with strong ”evidence” corroborating the soldier’s story. Others had only his word to go on.
But the strength or weakness of the evidence turned out to be immaterial. Researchers were able to predict people’s opinion over 80 percent of the time based simply on their opinions of the Bush administration, the GOP, the military and human rights groups. Those who had less affection for the president sided with the soldier even when the evidence was weak. And fans of the president tended to side with him even when the evidence was overwhelming.
We believe what we want, facts be damned.
”The scary thing,” says Westen, “is the extent to which you can imagine this influencing jury decisions, boardroom decisions, political decisions . . .”

It sounds like solid research and I believe it. But I’m not so pessimistic on the conclusion. Instead of bias, I would just say we are all Bayesians. We see things through a filter, but that allows us to process information faster and more efficiently. Sure, sometimes our preconceptions are mistaken and unhelpful, but generally we apply preconceptions every day to social and logistical problems big and small.
If you told me that someone I generally find unreliable or mistaken in his worldview (for me, Michael Moore or Ralph Nader), who believed X, you would have to add a lot of data clearly pointing to X in order for me to also believe X. In contrast, If you told me that Milton Friedman or Richard Posner believed Y, I could probably withstand seeing some data suggesting not-Y and still believe Y, based on my faith in Friedman or Posner. In a more pedestrian fashion, when my wife says my shirt doesn’t match, I believe her without checking myself, but if she told me my spark plugs needed changing, I would pretty much ignore her. People and groups have credibility on different issues, and their alignment with certain positions causes me to have greater or lesser belief in those positions irrespective of the data. Those starting points then require more or less corroborating data depending on my initial skepticism.
I remember Fama and French’s influential Journal of Finance 1992 article showing little evidence for the CAPM. It was so persuasive because the authors were and are efficient markets advocates, and CAPM is aligned with the efficient markets camp. Their conclusion against the CAPM suggested the data must have been very weak indeed. If that article was written by an unknown, or some ‘animal spirits’ advocate at Harvard it would not have been nearly as persuasive. That’s bias, but that’s also rational.

My take on it:
I sympathize with HedgeFundGuy’s desire to debunk, or demystify [actually, I first typed this as “demistify” but that makes sense too!], the study.  (As Aleks points out, this is the direction of Gigerenzer’s work on interpreting common mental “mistakes” as cognitively efficient behavior.)
However, I’m a little skeptical of HedgeFundGuy’s skepticism.  First off, I disagree with HedgeFundGuy’s claim that “we are all Bayesians.”  As far as I am aware of the research on judgment and decision making by Kahneman, Tversky, Krantz, . . ., we are not Bayesians–at least, our preferences and decisions do not follow Bayesian rules.  (Technically, in many settings people do not use base rate information, and in just about all settings, people fail to account for sample size in a way consistent with likelihood/Bayesian inference.)
Now, maybe that’s OK that we’re not Bayesian–I suspect Gigerenzer isn’t bothered by it–but, given that so many experiments show that people don’t make use of base rate information when they definitely should, I’m wary of suddenly turning around and applauding an experiment that shows that, in some settings, people use base-rate information too much.
There’s base-rate information, and there’s data information (well, that division is artificial, since as HedgeFundGuy points out, we interpret the data information (the “likelihood”) in light of our beliefs about its source), and people weight them different ways in different circumstances.  There’s clearly something “psychological” going on, and some of it can perhaps be interpreted as efficient use of scarce mental resources.  But interpreting this as Bayesian inference–no way.
To the extent that we all have political opinions, and we would like those who disagree with us to learn a bit from unpleasant facts, I think findings such as reported in this paper are indeed disturbing.

Using base rate information?

Gonzalo pointed me to a paper by Michael Hout, Laura Mangels, Jennifer Carlson, and Rachel Best at Berkeley that points out some systematic differences between election outcomes in e-voting and non-e-voting counties in Florida. To jump to the punch line: they have found an interesting pattern, which closer study suggests arises from just two of the e-voting counties: Broward and Palm Beach, which unexpectedly swung about 3% toward Bush in 2004. They also make some pretty strong causal claims which I would think should be studied further, but with some skepticism.
Pretty pictures
Before getting to a discussion of this paper, let me show you a few pictures (adapted from an analysis performed Bruce Shaw here at Columbia).
First, a scatterplot of the counties in Florida, displaying the change in the Republican vote percentage from 2000 to 2004, plotting vs. the Republican vote percentage in 2000 (in both cases, using the Republican % of the two-party vote). Red circles indicate the counties that used e-voting in 2000, and black circles used optical scans. The radius of each circle is roughly proportional to the log of the #votes in the county.

There are three obvious patterns in the figure:
1. The e-voting counties, especially the largest of them, were more Democrat-leaning.
2. For the optical scan counties, there was a consistent trend that the counties that favored Bush more in 2000, tended to move even more toward Bush in 2000.
3. For the e-voting counties, no such trend is apparent. In particular, the two large red circles on the left of the plot (Broward and Palm Beach) moved strongly toward the Republicans.
Next: looking at previous years, and commenting on the Hout et al. paper.
Looking at previous years
A natural question when looking at the above picture is: is this something new (possibly related to e-voting), or is it just a persistent pattern within these particular counties?
Here is the same picture, but plotting the change from 1996 to 2000 vs. the vote in 1996. (In this graph, we’re using the same color scheme as in our 2004 plots. That is, red indicates counties that used e-voting in 2004.)

And now 1992 to 1996:

And now 1988 to 1992:

Interesting…the pattern we saw in 2004 did not seem to happen in previous elections. However, other years have their own idiosycratic features (see the 1996 plot).
Cheating?
Something unusual seems to have happened in Broward and Palm Beach counties in 2004. One possibility, as suggested by Hout et al., is cheating, possibly set up ahead of time (e.g., by loading extra votes into the machines before the election or by setting it up to switch or not count some votes). This explanation makes a certain amount of sense, in that, if someone wanted to cheat ahead of time, it would make sense to do it in Florida, and it would make sense to do it in the large-population counties where a 5%-or-so swing in votes could make a difference in the statewide total.
A glance at the first graph above makes it clear that the swings-toward-Bush-that-don’t-fit-the-general-pattern-in-the-state are a feature just of Broward and Palm Beach counties–not of all the e-voting counties. If you remove the two big red circles from the left of the plot, there doesn’t seem to be much going on.
Just plain voting?
Two counties explain all the difference. I don’t know what was going on in these counties, what else was on the ballot, etc., but an obvious alternative explanation is that, for various reasons, 3% more people in those counties preferred Bush in 2004, compared to 2000. As can be seen in the graphs above for 2000, 1996, and 1992, such a swing would be unusual (at least compared to recent history), but that doesn’t mean it couldn’t happen!
Comment on the Hout et al. paper
The a paper by Hout et al. raises some interesting questions about differences between e-voting and optical scan counties in Florida. They are mostly careful to distinguish between correlation and causation; for example, in their a bstract, they refer to “the association between electronic voting and increased support for President Bush” [emphasis added]. However, in one place in their article, they are a little sloppy and imply causation: “Electronic voting raised President Bush’s advantage from the tiny edge he held in 2000 to a clearer margin of victory in 2004.”
More relevantly, especially for the prospect of studying this further, the only anomalies appear in Broward and Palm Beach counties (the most Democrat-leaning of the e-voting counties), at least as far as we can see in our first scatterplot above. Thus, although Hout et aal. perform regressions using “e-voting” as their predictor, it would make sense to focus study on these two counties. (They write that the “impact [of e-voting] was proportional to the Democratic support in the county,” but we only see discrepancies in those two counties, not in the others. Conversely, they find a “slightly
negative effect in the counties where Mr. Bush did the best in 2000”, suggesting that e-voting actually helped Kerry in some counties, but that claim seems clearly to be an artifact of their regression model.) Of course, we only look at vote in 2000, and their regression analysis controls for other things, but I doubt our conclusions would change much by adding in those other predictors.
Conclusion (so far)
It would make sense to look further at Broward and Palm Beach counties, where swings happened which look unexpected compared to the other counties and compared to 2000, 1996, and 1992. But lots of unexpected things happen in elections, so we shouldn’t jump to the conclusion that e-voting is related to these particular surprises.

Vote swings in Florida in counties with and without e-voting

Discover why pre-election polls are so unpredictable and often misleading in this thought-provoking op-ed by Philip Meyer. Exploring the work of a team led by a Columbia University professor, Meyer uncovers the challenges faced by pollsters in capturing both voter choice and the composition of the actual electorate accurately. With the likelihood of voters shifting in and out of the "likely-voter" group depending on external factors, Meyer argues that instead of focusing solely on election predictions, polls should be used to understand coalition formations and the issues resonating with different voter groups. Dive into this insightful analysis of the polls to gain a fresh perspective on election dynamics.

Op/Ed from USA Today (10/28/04)
Are the unsteady poll numbers making you queasy? Me, too. But now, a team led by a Columbia University professor has figured out why the Bush-Kerry pre-election polls have jumped around so much.
The polls are trying to capture two moving targets at the same time, and that multiplies the motion. One target is presidential choice. The other, more difficult one, is the composition of the actual electorate — the people who will exercise their right to vote.
While it is a pretty good bet that more than half of the voting-age population will turn out on Nov. 2 (or before, in states that allow early voting), nobody knows for sure who will be in that active half. A study led by Robert Erikson of Columbia’s department of political science analyzed data from Gallup’s daily tracking polls in the 2000 election and found that the tools for predicting voter participation are even more uncertain than the tools for identifying voter choice.
With every fresh poll, people move in and out of the likely-voter group, depending on who is excited by the news of the day. Only now, this close to the election, can we expect the likely-voter group to stop gyrating and settle down.
This is why predicting elections is the least useful application of early pre-election polls. They ought to be used to help us see what coalitions are forming in the electorate, help us understand why the politicians are emphasizing some issues and ducking others. None of that can be done if the measures are unstable.
Years ago, when George Gallup and Louis Harris dominated the national polling scene, pre-election polls focused on the voting-age population and zeroed in on the likely voters only at the last minute.
This year, the likely-voter model kicked in for the USA TODAY/CNN/Gallup Poll with the first reading in January. News media reported both numbers — the likely-voter and the registered-voter choices. But the more volatile likely-voter readings got the most play.
The volatility may be newsworthy, but it’s artificial. Erikson and his colleagues point out that most of the change in support for President Bush “is not change due to voter conversion from one side to the other but rather, simply, changes in group composition.”
Who are the likely voters?
The decision on who belongs in the likely-voter group is made differently by different pollsters. Gallup, which has done this the longest, uses eight questions, including, “How much thought have you given to the upcoming elections, quite a lot or only a little?”
Responses to these questions tend to “reflect transient political interest on the day of the poll,” said Erikson and his co-authors in an article scheduled for a forthcoming issue of Public Opinion Quarterly. That transient interest bobs up and down with the news and creates short-lived effects on the candidate choices. “Observed differences in the preferences of likely and unlikely voters do not even last for three days,” the scholars reported. “They can hardly be expected to carry over to Election Day.”
So what are the polls for?
A likely-voter poll is the right thing to do if all you want is to predict the outcome of the election — but that’s a nonsensical goal weeks before the event. Campaigns change things. That’s why we have them.
It would be far more useful to democracy if polls were used to see how the candidates’ messages were playing to a constant group, such as registered voters or the voting-age population. Whoever is elected will, after all, represent all of us.
Behind all of the campaign hoopla is a determined effort by each party to organize a majority coalition. Polls, properly used and interpreted, could illuminate that process.
For example, a key question in this campaign is the stability of the low-to-middle income groups that have been voting Republican. They are subject to switching because they are tugged by GOP social issues, such as abortion and gay marriage, on one side and by their economic interests, such as minimum wage, health care and Social Security, on the other.
The polls produce that kind of information, but we are blinded to it because the editors and headline writers like to fix on the illusory zigzags in the horse race.
Scoreboard shouldn’t block field of play
Some critics argue that media should stop reporting the horse-race standings. That solution is too extreme. A game wouldn’t be nearly as interesting without a scoreboard. But a game where we can’t see what the players are doing because the scoreboard blocks our view would not be much fun, either.
We are now close enough to the election for the likely-voter models to settle down and start making more sense. Expect the polls to start converging in the next few days, and they will probably be about as accurate as they usually are.
In the 17 presidential elections since 1936, the Gallup poll predicted the winner’s share of the popular vote within two percentage points only six times. But it was within four points in 13 of the 17. It ought to do at least that well this time.
Philip Meyer is a Knight Professor of Journalism at the University of North Carolina at Chapel Hill. He also is a member of USA TODAY’s board of contributors. His next book, The Vanishing Newspaper: Saving Journalism in the Information Age, will be published in November.

Why poll numbers keep hopping around by Philip Meyer

Get ready for a deep dive into the world of causal inference! In this article by Daniel Ho, he engages in a thought-provoking conversation with Kosuke Imai and Don Rubin about the advantages of matching methods over regression in social science research. They explore the robustness and flexibility of matching, discussing how it offers more reliable inferences and avoids assumptions tied to regression models. If you're a social scientist curious about the strengths of matching and eager to explore alternative approaches to statistical analysis, this article is a must-read. Don't miss out on this stimulating discussion in the pursuit of better causal inference methods.

Daniel Ho, Kosuke Imai, Gary King, and Liz Stuart recently wrote a paper on matching, followed by regression, as a tool for causal inference. They apply the methods developed by Don Rubin in 1970 and 1973 to some political science data, and make a strong argument, both theoretical and practical, for why this approach should be used more often in social science research.
I read the paper, and liked it a lot, but I had a few questions about how it “sold” the concept of matching. Kosuke, Don, and I had an email dialogue including the following exchange.
[The abstract of the paper claims that matching methods “offer the promise of causal inference with fewer assumptions” and give “considerably less model-dependent causal inferences”]
AG: Referring to matching as “nonparametric and non-model-based” might be misleading. It depends on how you define “model”, I guess, but from a practical standpoint, information has to be used in the matching, and I’m not sure there’s such a clear distinction between using a “model” as compared to an “algorithm” to do the matching.
DR: I think much of this stuff about “models” and non-models is unfortunate. Whatever procedure you use, you are ignoring certain aspects of the data (and so regarding them as irrelevant), and emphasizing aspects as important. For a trivial example, when you do something “robust” to estimate the “center” of a distn, you are typically making assumptions about the definition of “center” and the irrelevance of extreme observations to the estimation of the it. Etc.
KI: I want to take this opportunity and ask you one quick question. When I talk about matching methods to political scientists who are so used to running a bunch of regressions, they often ask why matching is better than regression or why they should bother to do matching in combination with regressions. What would be the best way to answer this question? I usually tell them about the benefits of the potential outcome framework, available diagnostics, and flexibility (e.g., as opposed to linear regression) etc. But, I’m wondering what you would say to social scientists!
AG: Matching restricts the range of comparison you’re doing. It allows you to make more robust inferences, but with a narrower range of applicability. See Figure 7.2 on page 227 of our book for a simple picture of what’s going on, in an extreme case. Matching is just a particular tool that can be used to study a subset of the decision space. The phrase I would use for social scientists is, “knowing a lot about a little”. The papers by Dehejia and Wahba discuss these issues in an applied context: http://www.columbia.edu/%7Erd247/papers/w6586.pdf and http://www.columbia.edu/%7Erd247/papers/matching.pdf 
DR: Also look at the simple tables in Cochran and Rubin (1973) or Rubin (1973) or Rubin (1979) etc. They all show that regression by itself is terribly unreliable with minor nonlinearity that is difficult to detect, even with careful diagnostics. This message is over three decades old!
KI: I agree that matching gives more robust inferences. That’s the main message of the paper that I presented and also of my JASA paper with David. The question my fellow social scientists ask is why matching is more robust than regressions (and hence why they should be doing matching rather than running regressions). One answer is that matching removes some observations and hence avoids extrapolation. But how about other kinds of matching that use all the observations (e.g., subclassification and full matching)? Are they more robust than regressions? What I usually tell them is that regressions often make stronger functional form assumptions than matching. With stratification, for example, you can fit separate regressions within each strata and then aggregate the results (therefore it does not assume that the same model fits all the data). I realize that there is no simple answer to this kind of a vague, and perhaps illposed, question. But, these are the kind of questions that you get when you tell soc. sci. about matching methods!
AG: I think the key idea is avoiding extrapolation, as you say above. But I don’t buy the claim that regression makes stronger functional form assumptions than matching. Regressions can (and should) include interactions. Regression-with-interaction, followed by poststratification, is a basic idea. It can be done with or without matching.
The social scientists whom I respect are more interested in the models than in the estimation procedures. For these people, I would focus on your models-with-interactions. If matching makes it easier to fit such models, that’s a big selling point of matching to me.
KI: One can think of it as a way to fit models with interactions in the multivariate setting by helping you create matches and subclasses. One thing I wanted to emphasize in my talk is that matching is not necessary a substitute for regressions, and that one can use matching methods to make regressions perform more robust.

Matching, regression, interactions, and robustness

Here’s another journalistic account of the Red/Blue divide. It’s from today’s (11/03/04) NY Times by Nicholas D. Kristof. He asserts that the poor (from America’s heartland) vote Republican and the wealthy (from suburban America) vote Democratic.
In the aftermath of this civil war that our nation has just fought, one result is clear: the Democratic Party’s first priority should be to reconnect with the American heartland.
I’m writing this on tenterhooks on Tuesday, without knowing the election results. But whether John Kerry’s supporters are now celebrating or seeking asylum abroad, they should be feeling wretched about the millions of farmers, factory workers and waitresses who ended up voting – utterly against their own interests – for Republican candidates.
One of the Republican Party’s major successes over the last few decades has been to persuade many of the working poor to vote for tax breaks for billionaires. Democrats are still effective on bread-and-butter issues like health care, but they come across in much of America as arrogant and out of touch the moment the discussion shifts to values.
“On values, they are really noncompetitive in the heartland,” noted Mike Johanns, a Republican who is governor of Nebraska. “This kind of elitist, Eastern approach to the party is just devastating in the Midwest and Western states. It’s very difficult for senatorial, Congressional and even local candidates to survive.”
In the summer, I was home – too briefly – in Yamhill, Ore., a rural, working-class area where most people would benefit from Democratic policies on taxes and health care. But many of those people disdain Democrats as elitists who empathize with spotted owls rather than loggers.
One problem is the yuppification of the Democratic Party. Thomas Frank, author of the best political book of the year, “What’s the Matter With Kansas: How Conservatives Won the Heart of America,” says that Democratic leaders have been so eager to win over suburban professionals that they have lost touch with blue-collar America.
“There is a very upper-middle-class flavor to liberalism, and that’s just bound to rub average people the wrong way,” Mr. Frank said. He notes that Republicans have used “culturally powerful but content-free issues” to connect to ordinary voters.
To put it another way, Democrats peddle issues, and Republicans sell values. Consider the four G’s: God, guns, gays and grizzlies.
One-third of Americans are evangelical Christians, and many of them perceive Democrats as often contemptuous of their faith. And, frankly, they’re often right. Some evangelicals take revenge by smiting Democratic candidates.
Then we have guns, which are such an emotive issue that Idaho’s Democratic candidate for the Senate two years ago, Alan Blinken, felt obliged to declare that he owned 24 guns “and I use them all.” Of course, this doesn’t mean that he’s reckless or poses a threat to people just because he has this many guns. This may not be the case at all. He could go on a lot of hunting trips, or he may like to go down to the shooting range as often as he can, and so will need to read something like this Review of Boberg XR9s if this is the best gun for his needs. The reality is is that we just don’t know, and if he wants to own this many guns, no one can stop him. People may not like it, but it’s his choice. But after this revelation of owning 24 guns and using them all, he still lost.
As for gays, that’s a rare wedge issue that Democrats have managed to neutralize in part, along with abortion. Most Americans disapprove of gay marriage but do support some kind of civil unions (just as they oppose “partial birth” abortions but don’t want teenage girls to die from coat-hanger abortions).
Finally, grizzlies – a metaphor for the way environmentalism is often perceived in the West as high-handed. When I visited Idaho, people were still enraged over a Clinton proposal to introduce 25 grizzly bears into the wild. It wasn’t worth antagonizing most of Idaho over 25 bears.
“The Republicans are smarter,” mused Oregon’s governor, Ted Kulongoski, a Democrat. “They’ve created … these social issues to get the public to stop looking at what’s happening to them economically.”
“What we once thought – that people would vote in their economic self-interest – is not true, and we Democrats haven’t figured out how to deal with that.”
Bill Clinton intuitively understood the challenge, and John Edwards seems to as well, perhaps because of their own working-class origins. But the party as a whole is mostly in denial.
To appeal to middle America, Democratic leaders don’t need to carry guns to church services and shoot grizzlies on the way. But a starting point would be to shed their inhibitions about talking about faith, and to work more with religious groups.
Otherwise, the Democratic Party’s efforts to improve the lives of working-class Americans in the long run will be blocked by the very people the Democrats aim to help.

Read Stat Modeling

Choose Your Reading Path

Full Backlog

Best Hits

You'll start with:

A weblog for research in statistical modeling and applications, especially in social sciences

The Electoral College favors voters in small states

Why it’s rational to vote

About Stat Modeling