# Research

Working Paper
Knox, Dean, Will Lowe, and Jonathan Mummolo. “The Bias Is Built In: How Administrative Records Mask Racially Biased Policing”. (Working Paper). Web. Publisher's VersionAbstract
Researchers often lack the necessary data to credibly estimate racial bias in policing. In particular, police administrative records lack information on civilians that police observe but do not investigate. In this paper, we show that if police racially discriminate when choosing whom to investigate, using administrative records to estimate racial bias in police behavior amounts to post-treatment conditioning, and renders many quantities of interest unidentified---even among investigated individuals---absent strong and untestable assumptions. In most cases, no set of controls can eliminate this statistical bias, the exact form of which we derive through principal stratification in a causal mediation framework. We develop a bias-correction procedure and nonparametric sharp bounds for race effects, replicate published findings, and show traditional estimation techniques can severely underestimate levels of racially biased policing or even mask discrimination entirely. We conclude by outlining a general and feasible design for future studies that is robust to this inferential snare.
Esberg, Jane, and Jonathan Mummolo. “Explaining Misperceptions of Crime”. (Working Paper). Print.Abstract
Promoting public safety is a central mandate of government. But despite decades of dramatic improvements, most Americans believe crime is rising—a mysterious pattern that may pervert the criminal justice policymaking process. What explains this disconnect? We test five plausible explanations: survey mismeasurement, extrapolation from local crime conditions, lack of exposure to facts, partisan cues and the racialization of crime. Cross-referencing over a decade of crime records with geolocated polling data and original survey experiments, we show individuals readily update beliefs when presented with accurate crime statistics, but this effect is attenuated when statistics are embedded in a typical crime news article, and confidence in perceptions is diminished when a copartisan elite undermines official statistics. We conclude Americans misperceive crime because of the frequency and manner of encounters with relevant statistics. Our results suggest widespread misperceptions are likely to persist barring foundational changes in Americans’ information consumption habits, or elite assistance.
Submitted
Mummolo, Jonathan, Erik Peterson, and Sean Westwood. “Conditional Party Loyalty”. (Submitted). Print.Abstract
Scholars have long debated the strength of voters' partisan attachments, asking whether party identification plays a near-deterministic role in political decisions. A large literature argues that partisan cues dominate political choice, but other studies show that competing information can rival, and even sometimes outweigh, the effects of partisanship. We reconcile this conflicting evidence by theorizing that variation in the dominance of partisanship is due to the salience of the other considerations pitted against it. Using survey experiments that characterize the role of partisanship relative to both high and low salience issues, we demonstrate a pattern of Conditional Party Loyalty.'' Partisan dominance occurs when political choice is centered around issues with low public salience. However, partisans defect from co-partisan candidates if they disagree on enough high-salience issues. These findings illuminate when and why partisanship fails to dominate political choice. They also suggest that, on many issues, public opinion minimally constrains politicians.

2018
Mummolo, Jonathan. “Militarization Fails to Enhance Police Safety or Reduce Crime but May Harm Police Reputation”. Proceedings of the National Academy of Sciences of the United States of America 15.37 (2018): , 15, 37, 9181-9186. Print.Abstract
The increasingly visible presence of heavily armed police units in American communities has stoked widespread concern over the militarization of local law enforcement. Advocates claim milita-rized policing protects ofﬁcers and deters violent crime, while critics allege these tactics are targeted at racial minorities and erode trust in law enforcement. Using a rare geocoded census of SWAT team deployments from Maryland, I show that milita-rized police units are more often deployed in communities with large shares of African American residents, even after control-ling for local crime rates. Further, using nationwide panel data on local police militarization, I demonstrate that militarized policing fails to enhance ofﬁcer safety or reduce local crime. Finally, using survey experiments—one of which includes a large oversample of African American respondents—I show that seeing militarized police in news reports may diminish police reputation in the mass public. In the case of militarized policing, the results suggest that the often-cited trade-off between public safety and civil liberties is a false choice.
Mummolo, Jonathan, and Erik Peterson. “Demand Effects in Survey Experiments: An Empirical Assessment”. American Political Science Review (2018). Web. Publisher's VersionAbstract
Survey experiments are ubiquitous in social science. A frequent critique is that positive results in these studies stem from Experimenter Demand Effects (EDEs)---bias that occurs when participants infer the purpose of an experiment and respond so as to help confirm a researcher's hypothesis. We argue that survey experiments have several features that make them robust to EDEs, and test for their presence in studies that involve over 12,000 participants and replicate five experimental designs touching on all empirical political science subfields. We randomly assign participants information about experimenter intent and show that providing this information does not alter the treatment effects in these experiments. Even financial incentives to respond in line with researcher expectations fail to consistently induce demand effects. Research participants exhibit a limited ability to adjust their behavior to align with researcher expectations, a finding with important implications for the design and interpretation of survey experiments.
Hainmueller, Jens, Jonathan Mummolo, and Yiqing Xu. “How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice”. Political Analysis (2018). Web. Publisher's VersionAbstract

Multiplicative interaction models are widely used in social science to examine whether the relationship between an outcome and an independent variable changes with a moderating variable. Current empirical practice tends to over- look two important problems. First, these models assume a linear interaction effect that changes at a constant rate with the moderator. Second, estimates of the conditional effects of the independent variable can be misleading if there is a lack of common support of the moderator. Replicating 46 interaction effects from 22 recent publications in five top political science journals, we find that these core assumptions often fail in practice, suggesting that a large portion of findings across all political science subfields based on interaction models are modeling artifacts or are at best highly model dependent. We propose a checklist of simple diagnostics to assess the validity of these assumptions and offer flexible estimation strategies that allow for nonlinear interaction effects and safeguard against excessive extrapolation. These statistical routines are available in both R and STATA.

Mummolo, Jonathan, and Erik Peterson. “Improving the Interpretation of Fixed Effects Regression Results”. Political Science Research and Methods 64 (2018): , 6, 4, 829-835. Web. Publisher's VersionAbstract

Fixed effects estimators are frequently used to combat selection bias by eliminating large swaths of variation in observational data. For example, it is well-known that unit fixed effects in panel data discard all between-unit variation, resulting in estimates of an independent variable's effect as it changes within units over time. In this article, we replicate several recent studies which used fixed effects estimators to show how descriptions of the substantive significance of results can be improved by precisely characterizing the variation being studied and presenting plausible counterfactuals---i.e., shifts in the independent variable likely to occur within the variation being used for estimation---when assessing the substantive impact of the treatment. We provide a checklist for the interpretation of fixed effects regression results to help avoid these interpretative pitfalls.

Mummolo, Jonathan. “Modern Police Tactics, Police-Citizen Interactions and the Prospects for Reform”. The Journal of Politics 80.1 (2018): , 80, 1, 1-15. Print.Abstract

High-profile incidents of police misconduct have led to widespread calls for law enforcement reform. But prior studies cast doubt on whether police commanders can control officers, and offer few policy remedies because of their focus on potentially immutable officer traits like personality. I advance an alternative, institutional perspective, and demonstrate that police officers—sometimes characterized as autonomous—are highly responsive to managerial directives. Using millions of records of police-citizen interactions alongside officer interviews, I evaluate the impact of a change to the protocol for stopping criminal suspects on police performance. An interrupted time series analysis shows the directive produced an immediate increase in the rate of stops producing evidence of the suspected crime. Interviewed officers said the order signaled increased managerial scrutiny, leading them to adopt more conservative tactics. Procedural changes can quickly and dramatically alter officer behavior, suggesting a reform strategy sometimes forestalled by psychological and personality-driven accounts of police reform.

Grimmer, Justin, et al.Obstacles to Estimating Voter ID Laws’ Effect on Turnout”. The Journal of Politics 80.3 (2018): , 80, 3, 1045-1051. Print.Abstract

Widespread concern that voter identification laws suppress turnout among racial and ethnic minorities has made empirical evaluations of these laws crucial.  But problems with administrative records and survey data impede such evaluations.  We replicate and extend Hajnal, Lajevardi and Nielson (2017), which reports that voter ID laws decrease turnout among minorities, using validated turnout data from five national surveys conducted between 2006 and 2014.  We show that the results of the paper are a product of data inaccuracies; the presented evidence does not support the stated conclusion; and alternative model specifications produce highly variable results.  When errors are corrected, one can recover positive, negative, or null estimates of the effect of voter ID laws on turnout, precluding firm conclusions.  We highlight more general problems with available data for research on election administration and we identify more appropriate data sources for research on state voting laws' effects.

2017
Hopkins, Daniel J., and Jonathan Mummolo. “Assessing the Breadth of Framing Effects”. Quarterly Journal of Political Science 12.1 (2017): , 12, 1, 37-57. Print.Abstract

Issue frames are a central concept in studying public opinion, and are thought to operate by foregrounding related considerations in citizens’ minds. But scholarship has yet to consider the breadth of framing effects by testing whether frames influence attitudes beyond the specific issue they highlight. For example, does a discussion of terrorism affect opinions on proximate issues like crime or even more remote issues like poverty? By measuring the breadth of framing effects, we can assess the extent to which citizens’ political considerations are cognitively organized by issues. We undertake a population-based survey experiment with roughly 3,300 respondents which includes frames related to terrorism, crime, health care, and government spending. The results demonstrate that framing effects are narrow, with limited but discernible spillover on proximate, structurally similar issues. Discrete issues not only organize elite politics but also exist in voters’ minds, a finding with implications for studying ideology as well as framing.

2016
Mummolo, Jonathan, and Erik Peterson. “How Content Preferences Limit the Reach of Voting Aids”. American Politics Research 45.2 (2016): , 45, 2, 159-185. Print.Abstract

Voters are often uninformed about the political candidates they choose between. Governments, media outlets, and civic organizations devote substantial resources to correcting these knowledge deficits by creating tools to provide candidate information to voters. Despite the widespread production of these aids, it remains unclear who they reach. We collect validated measures of online voter guide use for more than 40,000 newspaper readers during a state primary election. We show this newspaper-produced voter guide was primarily used by individuals with high levels of political interest and knowledge, a finding in contrast to earlier hypotheses that providing guides directly to voters online would reduce disparities in use based on political interest. A field experiment promoting the voter guide failed to diminish these consumption gaps. These results show that the same content preferences that contribute to an unequal distribution of political knowledge also impede the effectiveness of subsequent efforts to close knowledge gaps.

Mummolo, Jonathan. “News from the Other Side: How Topic Relevance Limits the Prevalence of Partisan Selective Exposure”. The Journal of Politics 78.3 (2016): , 78, 3, 763-773. Web. Publisher's VersionAbstract

Prior research has demonstrated a preference among partisans for like-minded news outlets, a key mechanism through which the media may be polarizing Americans. But in order for source reputations to cause widespread selective exposure, individuals must prioritize them above other competing attributes of news content. Evaluating the relative influence of various contributors to media choice is therefore critical. This study pits two such factors, source reputation and topic relevance, against one another in conjoint survey experiments offering randomly paired news items to partisans. Making a news source’s reputation politically unfriendly lowers the probability that an individual chooses an item, but this negative effect is often eclipsed by the positive effect of making a news topic relevant to the individual. In many popular modern news consumption environments, where consumers encounter a diverse mixture of sources and topics, the ability of source reputations to contribute to polarization via partisan selective exposure is limited.

Hopkins, Daniel J., et al.Out of Context: The Unexpected Absence of Spatial Variation in U.S. Immigrants’ Perceptions of Discrimination”. Politics, Groups and Identities 43 (2016): , 4, 3, 363-392. Print.Abstract

Immigrants’ perceptions of discrimination (PD) correlate strongly with various political outcomes, including group consciousness and partisan identity. Here, we examine the hypothesis that immigrants’ PD vary across US localities, as threatened responses by native-born residents may increase perceived discrimination among neighboring immigrants. We also consider the alternative hypothesis that barriers to the expression and detection of discrimination decouple native-born attitudes from immigrants’ perceptions about their treatment. We test these claims by analyzing three national surveys of almost 11,000 first-generation Latino, Asian, and Muslim immigrants. The results indicate that immigrants’ PD hardly vary across localities. While anti-immigrant attitudes are known to be geographically clustered, immigrants’ PD prove not to be. This mismatch helps us narrow the potential causes of perceived discrimination, and it suggests the value of further research into perceived discrimination’s consequences for immigrants’ social and political incorporation.

Mummolo, Jonathan, and Clayton Nall. “Why Partisans Do Not Sort: The Constraints on Political Segregation”. The Journal of Politics 79.1 (2016): , 79, 1, 45-59. Web. Publisher's VersionAbstract

Social divisions between American partisans are growing, with Republicans and Democrats exhibiting homophily in a range of seemingly nonpolitical domains. It has been widely claimed that this partisan social divide extends to Americans’ decisions about where to live. In two original survey experiments, we confirm that Democrats are, in fact, more likely than Republicans to prefer living in more Democratic, dense, and racially diverse places. However, improving on previous studies, we test respondents’ stated preferences against their actual moving behavior. While partisans differ in their residential preferences, on average they are not migrating to more politically distinct communities. Using zip-code-level census and partisanship data on the places where respondents live, we provide one explanation for this contradiction: by prioritizing common concerns when deciding where to live, Americans forgo the opportunity to move to more politically compatible communities.

2012
Bailey, Michael A., Jonathan Mummolo, and Hans Noel. “Tea Party Influence: A Story of Activists and Elites”. American Politics Research 40.5 (2012): , 40, 5, 769-804. Print.Abstract

Understanding how the Tea Party has affected congressional elections and roll call voting helps us understand not only an important political movement, but how movements affect politics more generally. We investigate four channels for the movement to influence political outcomes: activists, constituent opinion, group endorsement activity and elite-level self-identification. We find consistent evidence that activists mattered both electorally and for roll call voting on issues of importance to the movement. Constituent opinion had virtually no impact on either political outcome. Group endorsement activity had possible effects on elections, but mostly no effect on congressional voting. Self-identification among elites did not enhance—or harm—Republican electoral fortunes, but did affect congressional votes important to the movement. These divergent results illustrate how movement politics can influence outcomes through multiple channels and call into question the usefulness of the “Tea Party’’ moniker without important qualifiers.