Are Democrats’ Senate Chances In 2018 Overrated?

After Democrat Doug Jones won a stunning victory in Alabama’s special election for the U.S. Senate last month, lots of smart people whose work I read and follow, such as The New York Times’s Nate Cohn, declared that the battle for Senate control in 2018 was a “toss-up.” Prediction markets largely concur; after Jones’s win, the betting odds of Democrats taking over the Senate shot up to about 45 percent.

I think this might be premature. Winning in Alabama certainly makes the Democrats’ path easier: They could now gain control of the Senate by retaining all of their own seats, plus picking up the Republican held-seats in purplish Nevada and Arizona. But they’re probably still the underdogs.

Democrats face a really tough Senate map

It’s not that I’m pessimistic about the Democrats’ overall position next year. On the contrary, I think most political observers had, until recently, been slow to recognize just how bad things had gotten for Republicans. But the Senate map is really tough for Democrats, with 26 Democratic seats in play next year (including a newly opened seat in Minnesota after Al Franken announced his intention to retire) as compared to just eight Republican ones. Moreover, five of the Democratic-held seats — the ones in West Virginia, North Dakota, Montana, Missouri and Indiana — are in states that President Trump won by 18 percentage points or more.

Just how bad is this map for Democrats? It’s bad enough that it may be the worst Senate map that any party has faced ever, or at least since direct election of senators began in 1913. It’s bad enough that Democrats could conceivably gain 35 or 40 seats in the House … and not pick up the two seats they need in the Senate.

Don’t believe me? Check out the race-by-race ratings put forward by independent groups such as the Cook Political Report, Inside Elections¹⁸ and Sabato’s Crystal Ball. They suggest that Democrats are more likely to lose Senate seats next year than to gain them — and that while there’s a plausible path to a Democratic majority, it’s a fairly unlikely one.

These groups place their ratings into qualitative categories (e.g., “Likely Republican” and “Lean Democrat”) rather than assigning them probabilities. But it’s not too difficult to translate from qualitative to quantitative. I went back and looked at the ratings that Cook, Sabato and Inside Elections had assigned to Senate races at about this point¹⁹ in the 2010, 2012, 2014 and 2016 election cycles. About 95 percent of the races these groups rated as being “safe” were in fact won by the party they listed as the favorite (most of the exceptions were when an incumbent unexpectedly retired). The favorite won in about 85 percent of the races they rated as “likely” and about 75 percent of the races they rated as “lean.”²⁰ We can use these probabilities as guides to estimate the likelihood of Democrats winning each Senate race in 2018, averaging the ratings from Cook, Sabato and Inside Elections together:

If you add the probabilities for each race together, you’ll find that these ratings have Democrats losing an average of about three Senate seats next year. An optimistic Democrat might note that all the races the party needs to win control of the Senate — that is, all of their own seats, plus the Republican-held ones in Arizona and Nevada — are nevertheless listed as toss-ups or better for Democrats. But that doesn’t mean their overall chances of winning the Senate are 50-50. Unless Democrats unexpectedly put another seat, such as Tennessee, into play, they’d have to win all of the four toss-ups (Arizona, Indiana, Missouri and Nevada), plus a number of other races in which they’re listed as only marginal favorites (such as West Virginia and Florida).

How you model the Senate makes a big difference to Democrats’ odds

What are the chances of that? This is actually a tricky question because the outcomes in each race are correlated. (Failure to account for these sorts of correlations was a big reason that some models underrated Trump’s Electoral College chances in 2016.) If Trump’s approval rating declines even further by November, for instance, that could make the whole map bluer — so all of the races that currently look like toss-ups could be “lean Democrat” by then. Alternatively, strong economic growth later this year could make voters more inclined to keep Republicans in office — which could enable the GOP to win all the toss-up races.

These correlations make a huge difference when forecasting the fate of the Senate. We’ll launch our “official” FiveThirtyEight Senate forecasts later this year, once we have more polling in each race. (We don’t have much polling now; that’s why we’re using these ratings.) But I built a simulation program in which — unlike in a traditional Monte Carlo simulation where each race is assumed to be independent from the others — I can crank the correlations up or down as much as I want. If I assume that the races are totally uncorrelated — how the Democratic candidate does in Nevada has nothing to do with how the Democrat fares in Arizona — Democrats’ chances of taking over the Senate are only about 1 percent, according to the simulation. If I instead assume the races are perfectly correlated — if you win one “toss-up,” you win ’em all — Democrats’ chances are 50 percent, by contrast.

But neither of those assumptions is realistic. Although it’s important to account for some correlation, Senate races are a long way from being perfectly correlated. Sometimes the candidates can matter, as we saw with Jones and Roy Moore in Alabama. And the most competitive races this year are a somewhat eclectic mix of vulnerable Democratic incumbents (such as Missouri’s Claire McCaskill), vulnerable Republican incumbents (such as Nevada’s Dean Heller), and open seats (such as in Arizona).

A good rule of thumb for Senate races is that roughly half the uncertainty stems from local factors and half comes from national factors. If I encode that assumption into the simulation, it comes up with a 22 percent probability of Democrats taking over the Senate based on the race ratings. That isn’t nothing, but it’s a long way from the even-steven battle that conventional wisdom now seems to assume.

Of course, this logic is a little bit circular. The only thing I’ve “proven” here — it’s also been demonstrated by other analysts such as Dean Strachan — is that there’s an incongruity between how people²¹ are looking at individual Senate races (let’s call this the “micro” view) and how they’re looking at the overall Senate picture (the “macro” view). But It could be that the macro view is right and that the assessment of individual Senate races is off. Maybe Heller’s seat in Nevada should be thought of as “lean Democrat” or even “likely Democrat” rather than a toss-up, for instance.

I’ll return to this question in a moment — but first, there’s one more slightly unpleasant complication.

More Senate seats could come open by November

The 34 races I listed in the chart above are not necessarily the only ones that will be contested this year. As was the case with Franken’s seat in Minnesota, other senators could unexpectedly retire, or they could pass away, leaving a vacancy that would be filled by special election. Bettors assessing Democrats’ overall chances of winning the Senate are no doubt accounting for these possibilities, even if groups such as Sabato, Inside Elections and Cook are not.

Two seats involving ailing Republican senators could particularly affect the Senate calculus. One is Arizona, where Sen. John McCain has been diagnosed with an aggressive form of brain cancer; he hasn’t yet announced any plans to retire, but Republicans are preparing for the possibility that he’ll end his term prematurely. The other is Mississippi, where Sen. Thad Cochran, age 80, is reportedly considering retirement after a series of medical complications.

Another way for Democrats to gain seats would be for a current Republican to switch parties. Empirically, this happens fairly often after wave elections (two Democratic senators switched to the GOP after the 1994 Republican wave, and Republican Arlen Specter switched to the Democratic Party after 2008). It also happens when control of the Senate is up for grabs (as when Jim Jeffords defected from Republicans to caucus with Democrats in 2001). The most likely Republicans to switch parties — based on their relatively high rate of disagreement with Trump and their cross-partisan appeal in their respective states — are probably Maine’s Susan Collins and Alaska’s Lisa Murkowski.

So suppose we add the following contingencies to the simulation:

• There’s a 50-50 chance of a second Senate race in Arizona. If such a race occurs, it rates as a toss-up, just like the other Arizona race does (which is also an open seat since Sen. Jeff Flake is retiring).
• There’s also a 50-50 chance of a second Senate race in Mississippi. If such a race occurs, it rates as “likely Republican” — slightly less certain for Republicans than the other Mississippi race, which features an incumbent (Sen. Roger Wicker).
• There’s a 25 percent chance that Collins switches parties, conditional upon her vote either swinging the Senate to Democrats or Democrats already having won the Senate.²²
• And there’s a 10 percent chance that Murkowski does the same.

Include these possibilities, and the Democrats’ chances of winning the Senate improve to 29 percent, up from 22 percent before. So they help at the margin but aren’t enough on their own to turn Senate control into a toss-up.²³

What a true Senate toss-up would look like

As I mentioned, however, it’s possible that the macro view of the Senate is right in treating control as a 50-50 proposition and that the ratings of individual races are miscalibrated. As reliable as they usually are, ratings from groups such as Cook can sometimes be slow to account for the overall political environment, which currently seems to strongly favor Democrats.

Take Flake’s seat in Arizona, for example. Cook, Sabato and Inside Elections all rate the race as a toss-up. But there’s an argument that Democrats should be considered favorites there. In 2016 — a roughly neutral political year — Arizona voted for Trump by only about 4 percentage points. But as we gear up for 2018, the generic congressional ballot currently favors Democrats by about 10 points, which would be enough to outweigh that. Plus, the GOP has some potentially problematic candidates in the state, such as Kelli Ward. You wouldn’t want to rate Democrats as especially heavy favorites, but it might be reasonable to describe Arizona as leaning Democrat instead of as a toss-up.

Rather than nitpick the ratings in individual states, suppose we shift all the races slightly toward Democrats. I programmed the simulation to keep adding to Democrats’ win probability in every race until their overall chances of winning the Senate reached 50 percent. (The adjustment²⁴ is nonlinear: Races that were initially rated as toss-ups shift more than those where one party was considered almost certain to win.) Essentially, we’re working backward: If we insist that Senate control is a toss-up, what would the ratings in individual races have to look like to justify that claim?

There are several interesting things going on here:

• The “toss-ups” aren’t really toss-ups. The four races that were originally rated as pure toss-ups — Arizona, Indiana, Missouri, Nevada — now rate as having a two-in-three chance of going for Democrats instead.
• Republicans have fewer targets. Seemingly competitive Midwestern states such as Wisconsin, Michigan, Ohio and Pennsylvania, where there are potentially vulnerable Democratic incumbents, become real long shots for Republicans.
• And Democrats have more targets. The blue team has some realistic prospects of picking up Republican seats beyond their primary targets of Arizona and Nevada, with Tennessee or a second seat opening up in Arizona being the most likely possibilities.

So this is what a true 50-50 battle for the Senate would look like — but are these revised ratings realistic? You’d have to go through on a case-by-case basis. As I mentioned, for instance, I’d have no problem with treating Arizona as leaning Democratic. And the revised ratings for some of the Democratic incumbents in purple states, such as Wisconsin’s Tammy Baldwin and Pennsylvania’s Bob Casey, seem more realistic than the original ones to me; they shouldn’t be all that vulnerable in this sort of political climate. But I’m not sure I buy that Democrats have a 42 percent chance of a pickup in Tennessee or that McCaskill is a two-to-one favorite in Missouri, where Republicans are fielding some strong opponents.

Personally, I’d probably split the difference between the macro and the micro views and put Democrats’ chances of winning the Senate somewhere in the range of 35 percent to 40 percent. That’s a lot better for Democrats than it was before Alabama, and it’s higher than it probably “should” be given how favorable the Senate map is for Republicans. But it’s still a fairly steep hill to climb.

CORRECTION (Jan. 10, 2:40 p.m.): A previous version of this article incorrectly listed the probability of Democrats’ winning the Tennessee Senate race. According to the race ratings, Democrats’ implied chances are 27 percent, not 23 percent. The article has been updated.