Donald Trump’s approval rating is one of the key news stories of 2017. That’s true not just for political junkies. Approval — or the lack of it — can give the president more or less power as he negotiates with Congress.
How do we know a president’s approval rating? From polls. But how the poll data are processed makes a difference in what the results are. That becomes clear when we look at the graph above, prepared by Alexander Agadjanian, which shows Trump’s 2017 support this year according to two different kinds of polls:
- Adjusted for party. The red dots come from YouGov, a survey organization that adjusts its estimates for party identification. That means it estimates approval among Republicans, independents and Democrats, and then averages that over the proportion of these groups in the population. In doing so, it adjusts for any problems that result when a disproportionate number of Democrats or Republicans respond to a particular survey.
- Not adjusted for party. The black dots come from various other pollsters who did not adjust their polls for party ID.
As you can see, the general time trend is the same for both groups, but the YouGov results are much more stable. The variation in the unadjusted polls must be coming from two sources: first, from polling house effects (i.e., differences in how particular pollsters work); and second, variation in who’s responding to the polls.
Here’s what’s key: While the poll aggregates appear to be capturing the trend in Trump’s approval, if you’re following individual polls or short groups of polls you’ll be getting jerked around by artificial differences. Agadjanian wrote to me to explain:
Over the last year, I’ve gotten really interested in differential partisan nonresponse bias and considering the idea of weighting by party identification. I wanted to check for partisan nonresponse bias effects in Trump approval rating polls (similar to the way done in this post on your blog), and I came up with the below graphs about a week ago. I explain details here, but interestingly there appears to be a strong relationship (R2 = .45) between the unweighted partisan distribution of a poll and Trump evaluation if you only look at approval rating polls that don’t weight by party or past vote. It feels like a crude way of measuring partisan nonresponse, but it’s still interesting that 1) a fairly strong relationship already exists and 2) it’s occurring outside of just a campaign season context. For the latter point, I’ve been contemplating whether this suggests partisan nonresponse is a systemic issue that polls of any kind must always deal with.