Sunny Side Up

Ross E. Burkhart

School of Public Service

Boise State University

14 December 2019

            Yesterday, the aftermath of a consequential election in a critical Western democracy, the United Kingdom, came upon an important day for our Political Science students: the capstone course project showcase. A brief recap of both events follows, and then the theme of this short essay emerges: reflecting on a poor election forecast.

            The election process is simple in Britain: voters tick a box for their favored candidate and place the paper in a box, to be counted by the returning officer's staff in their geographic constituency. The winners in the 650 constituencies, decided by plurality (meaning winning at least one more vote than their closest opponent), are thus duly elected Members of Parliament. The party winning the most seats gets to form the next government. In this case, the Conservatives won a smashing majority of 80 in the House of Commons. This is the Conservative Party's largest majority since 1987, and suggests a fundamental realignment in British politics, where working-class voters crossed over to the Conservatives who promised a speedy Brexit to address working-class grievances over being ignored by globalized Brussels bureaucrats.

            Meanwhile, the capstone Political Science students presented their projects in poster format, on public display in the Student Union Building. They conducted research on an impressive array of topics: Title IX funding and transgender rights in schools, policing reform and civil liberties, increasing the intelligence budget, Twitter followers and campaign spending, balancing abortion politics, how reporters cultivate relationships with politicians, the prospects for a hard or a soft Brexit, the effects of door-to-door campaigning in state legislative elections, water pollution in Costa Rica, state-level abortion restrictions and the Supreme Court, and human rights violations in Burma and China. The students also had an eye in their poster presentations toward highlighting findings and shortcomings in their research. As critics, they were most effective evaluators in pointing out the shortcomings in their research and suggestions for future refinement. Such impressive self-reflection behooves me to follow their example with one of my recent research pieces.

            Two weeks ago I posted "A Snap Elections Forecasting Model of British Elections" in which, using data that reflected the snap election environment (defined as an election occurring well before the scheduled end of the Parliament), I gave a forecast of the Conservatives winning 290 seats in the House of Commons. It was a mere 75 seats off!  I need to understand what went wrong with the model. The forecasting model was as follows:

S = 20.50 + .73 P + 1.61 J – 3.78 T     (Eq.1)

    (1.45)   (3.03)   (2.74)   (1.82)

R² (adj.) = .68    S.E.E. = 5.42    D-W = 1.84   N = 15

Where S = percentage of House of Commons seats won by the incumbent governing party, P = approval of the performance of the Government at either the time of the dissolution of Parliament during a snap election or six months prior to dissolution during a non-snap election, J = jobless rate at the time of dissolution during a snap election or six months prior to dissolution during a non-snap election, T = the number of terms the incumbent party has been in office assuming a fatigue factor takes place. The figures in parentheses are absolute t-ratios, S.E.E. is the standard error of estimate, and D-W is the Durbin-Watson statistic after the Cochrane-Orcutt procedure.

            I noted in my forecast that the R-squared statistic is on the low side for a forecasting model, and that the standard error of estimate is rather high, suggesting a fair bit of unexplained variance in the seats forecast, though certainly nothing like the 75 seat error in forecast. Of course, that much error is unacceptable in a forecasting model, hence this postmortem.

            We are left with several possible paths toward enlightenment. One is that the election was an outlier, a freak of nature that could not be forecasted well. We reject this path because other forecasters were rather close to the mark (Murr, Stegmaier, and Lewis-Beck 2019), but there were prominent forecasts that were wide of the mark (Lebo and Fisher 2019), suggesting there was no real unanimous view as to the forecasted result (Greenwood 2019).

            A second possibility is that British elections are inherently unpredictable, with those who claim to make a forecast very close to the final result just having been lucky in their forecasting. After all, forecasting is an informed probability exercise, and even a broken clock is correct twice a day, goes the old adage, so anyone has a chance of getting it right. While there are critics of the election forecasting enterprise (van der Eijk 2005), this possibility of unpredictable elections seems unlikely as well. Many elections are well forecasted (Burkhart 2018). More broadly speaking, the forecasting literature has successfully forecasted election results in many countries over a lengthy time period, too well to ignore (Bélanger and Trotter 2017).

            A third possibilithy is that the model itself is rather faulty. There are three pieces of evidence that point in this direction.  First, the R-squared is rather low for forecasting purposes at .68. Ideally, we seek an R-squared of at least .75 in election forecasting models. Second, the unemployment independent variable is oddly signed in the positive direction, suggesting that incumbent governments that raise unemployment rates are more likely to win more seats. (The positive sign is most likely influenced by Margaret Thatcher's three strong majorities during the highest unemployment rates in British history since the Great Depression.)  Third, the statistical significance of the terms in office variable is pretty marginal at approximately the .09 level. The preferred significance level is .05.

            Further examination reveals that the next election forecast, presumably taking place in 2024 given the substantial Conservative majority, erodes in model performance. The forecast equation for the 2024 election is as follows (Cochrane-Orcutt estimation):

S = 7.16 + .95 P + 1.68 J – 1.37 T     (Eq. 2)

    (.56)   (4.10)   (3.45)   (.78)

R² (adj.) = .61    S.E.E. = 5.60    D-W = 2.06   N = 16

All model performance indicators decline in 2024 compared to 2019. The y-intercept is no longer significant, indicating that the baseline percentage of seats for the Conservatives, while estimated to be 7.16% of all seats, could actually be 0%. The popularity function is elevated in importance, almost equating a one-to-one impact on the Conservative vote. The jobless independent variable remains steady in impact, while the trend variable drifts into statistical insignificance.

            In fact, simply dropping the trend variable yields the following result:

S = -.63 + 1.07 P + 2.35 J     (Eq. 3)

       (.27) (5.53)  (2.95)

R² (adj.) = .62    S.E.E. = 5.51    D-W = 2.15   N = 16

This model performs better than Eq. 2 in that all variables are statistically significant and the adjusted R-squared rises, if just a smidgen. Yet the y-intercept suggests that the baseline for Conservative seats is effectively zero, which is nonsensical.

            To be sure, more work is needed in understanding a proper structural model of British election forecasting. In the face of the strong performance of the citizen forecasting models and the multilevel YouGov model, it is tempting to proclaim this the elegy of the structural equation model. However, we must remember another statistical issue with this literature, and that is small sample sizes. The Central Limit Theorem is a good rule of thumb for regression analyses upon which these forecasts rely. The 2019 election is one data point out of the small sample of 17 British general elections in this model, so perhaps the only elegy worth making is that it is too soon to tell its demise.

REFERENCES

Bélanger,  Éric, and David Trotter. 2017.  "Econometric Approaches to Forecasting." In

            The SAGE Handbook of Electoral Behavior, eds. Kai Arzheimer, Jocelyn Evans,

            and Michael S. Lewis-Beck. Beverly Hills, Calif. Sage Publications.

Burkhart, Ross E. 2018. "Was Anyone Right? Assessing the 2016 U.S. Presidential

            Election Forecast." The Blue Review, 15 January.
Greenwood, Joe. 2019. "Experts predict that the general election will be tighter than

            expected." LSE British Politics and Policy, 10 December.

Lebo, Matthew, and Stephen Fisher. 2019. "Forecasting the 2019 General Election using

            the PM and Pendulum Model". LSE British Politics and Policy, 20 November.

Murr, Andreas, Mary Stegmaier, and Michael Lewis-Beck. 2019. "Citizen forecasting

            2019: a big win for the Conservatives." LSE British Politics and Policy, 4

            December.

van der Eijk, Cees. 2005. "Election Forecasting: A Sceptical View." British Journal of

            Politics and International Relations 7:210-4.