Are evangelicals really politically different from other Americans? Joshua Wu wrote an intriguing piece “Myth of the Evangelical Voter” for for the Emerging Scholars blog. He used some advanced statistical analysis called “propensity score matching” and found that evangelicals are not different from other Americans. I loved his approach, but there were other ways to look at the same data. I asked Ryan Burge to respond. Ryan has published research using propensity score matching to study religion and politics. Ryan’s response to Wu explains how differences in our definition of “evangelical” can change whether we conclude the existence of the evangelical voter to be myth or reality.

Propensity score matching pairs people who are very similar to each other. Photo by Toni Blay via Flickr creative commons https://www.flickr.com/photos/toniblay/47802555/

Guest post by Ryan Burge

Religion and politics is notoriously difficult to study. A person’s religion can take many forms. It’s what someone does; it’s also what a person believes; and it’s also the religious community the person belongs to. Sociologists often call these the three B’s (belief, behavior, and belonging). Each of the “b’s” overlap. To make matters more complicated, these aspects of belief are intertwined with demographics like race, ethnicity, geography, gender, and education.

How can we sort out what part of religion shapes politics? How do we compare people while taking into account other factors?

One of the most effective ways to untangle these overlapping explanations is the use of propensity score matching. Think of matching like creating miniature experiments inside survey data that has already been collected. We can’t put people in a lab and randomly give some religion and others not.  What we can do is match people who are very, very similar to see if their differences are truly due to religion or not. Propensity score matching allows a researcher to find and compare those subjects in a pre-collected dataset like the General Social Survey.

The most important component of matching is how to define what individuals will be matched on. Joshua Wu compared people who self-describe as “born again” and those who do not. Each “born again person” is matched with people who are very similar to them except that they are “not born again.” For example, a born again person is compared with someone who is not born again but who shares the same race, income, education, partisanship, religious commitment, and religious beliefs. The results show whether being “born again” truly distinguishes someone or if it’s just an artifact of having other things in common.

Wu found that there is little political difference between evangelicals and everyone else. There is nothing wrong with Wu’s statistics. Because Wu was so careful in his analysis (including a detailed appendix) I was able to replicate his results. I found the same results as Wu: Taking everything else into account, those who say they are born again are not really different from others in the population. (note: Wu and I use born again, along with some measures for church attendance and basic beliefs)

The real question is whether or not the best way to identify evangelicals is to ask people if they are born again. Do the results hold up if we use other ways of identifying evangelicals?

There are some reasons why researchers often don’t use the born again question. One is that when we speak of “evangelicals” we often mean those in churches that are historically white. These churches often have members who are black or an ethnic minority, but they have a different history than historically black churches. The born again question also includes those in churches that are not Protestant. For example in the 2014 GSS nearly 22% of Catholics identify as born again. Using the born again question, one ends up with a larger, more diverse group of Christians than we typically think of as being evangelical.

Think of Wu’s analysis as being a study of the biggest tent. Anyone who is a Christian church of any kind who says they’re born again are in the evangelical tent. What happens if we use a definition of evangelical that is more narrow and includes only those who are in a Protestant church that is historically part of the evangelical tradition. What happens if the tent is smaller and includes only those who are part of an evangelical church? It’s a method in which no one who is Catholic, no one who attends a mainline Protestant church, and no one from an historically black denomination are included.

Instead of using born again, I ran a matching model using the same data (2014 GSS) and same covariates as Wu, but instead used a measure of evangelical that is derived from what church they attend. Sociologists use the shorthand “reltrad” (for religious tradition) to sort individuals into categories based on what type of church that the individual affiliates. For example, the Southern Baptist Convention is considered evangelical under this scheme. Using that definition of evangelical some stark differences emerge from Wu’s findings.

Note:  Results in red denote differences between evangelicals and non-evangelicals that are statistically significant (at p<.05 level)

Just like in Wu’s analysis, matching shrinks the difference between evangelicals and others. This makes sense: being an evangelical may be important, but there are always other factors at work that inflate the differences.

However, I find something different from Wu: evangelicals are different politically from others who are otherwise identical to them. When using the reltrad definition of evangelical, there are still statistically significant differences between the groups even after matching. These results indicate that evangelicals are, in fact, more conservative that the population writ large. Even after matching evangelicals with others with the same partisanship, race, religious beliefs, religious practices, and other important factors, I find that evangelicals are

• More likely to identify as Republican
• More likely to think that it’s important that Americans are Christian
• More likely to oppose illegal immigration
• Less likely support same-sex marriage
• Equally likely to vote

Bottomline: If we use a big-tent measure of evangelicals, we don’t find evangelicals to be different from other Americans. But if we use a measure based on what church people attend, then we find that there is an evangelical voter who is more likely to be politically conservative.

It’s great to see more work using this type of analysis to untangle religion and politics. My doctoral dissertation expands upon the use of matching to untangle the three B’s and finds that religious behavior (weekly church attendance) generates a number of positive outcomes for democratic society including higher levels of political tolerance and greater likelihood of turning out to vote for Republican presidential candidates. As more survey data is collected and statistical methods become more sophisticated researchers will be better equipped to undercover the mechanism(s) that distinguish evangelicals from the rest of the population.

Ryan Burge @ryanburge researches religion and politics. He is currently studying the politics of the emerging church movement.

Don’t miss any more posts from the Corner of Church & State. Click the red subscribe button in the right hand column. Follow @TobinGrant on Twitter and on the Corner of Church & State Facebook page.