More evidence that public opinion polls don’t measure policy preferences

I frequently argue that public opinion polls on complex policy issues are almost meaningless.  (Although polls can be useful for predicting election outcomes.)  It all depends on the framing.  Here’s another study that reached the same conclusion:

We presented respondents with two different education plans, the details of which are unimportant in this context. What is important is that half the sample was told A was the Democratic plan and B was the Republican plan, while the other half of our national sample was told A was the Republican plan and B was the Democrats’ approach.

The questions dealt with substantive policy on a subject quite important to most Americans “” education “” and issues that people are familiar with “” class size, teacher pay and the like.

Nonetheless, when the specifics in Plan A were presented as the Democratic plan and B as the Republican plan, Democrats preferred A by 75 percent to 17 percent, and Republicans favored B by 13 percent to 78 percent. When the exact same elements of A were presented in the exact same words, but as the Republicans’ plan, and with B as the Democrats’ plan, Democrats preferred B by 80 percent to 12 percent, while Republicans preferred “their party’s plan” by 70 percent to 10 percent. Independents split fairly evenly both times. In short, support for an identical education plan shifted by more than 60 points among partisans, depending on which party was said to back it.

Most polls on policy questions report little more than mood affiliation.

Update:  Here’s how Yahoo describes the charges against Dennis Hastert:

Hastert pleads not guilty in hush money case

The former House Speaker is accused of agreeing to paying $3.5M to hide past misconduct claims

Interesting that the American press is so ashamed of our country that they refuse come right out and say that it can be illegal to withdraw cash from your own bank account, and instead feel a need to make up lies about Hastert being charged with paying hush money.

Update#2:  Et tu, Vox?


Tags:

 
 
 

29 Responses to “More evidence that public opinion polls don’t measure policy preferences”

  1. Gravatar of Jason Smith Jason Smith
    9. June 2015 at 07:02

    The point of political parties is to produce this exact effect. I have a job and so can’t evaluate in detail every policy. Putting representatives in office saves time. And letting them make decisions saves time. And voting them out of office if they screw up? Also a time saver.

    “Mood affiliation” is the point of representative democracy.

  2. Gravatar of Ray Lopez Ray Lopez
    9. June 2015 at 07:38

    Ho-hum, another boring Sumner read. K. Arrow back in the days proved that there’s no fair way of voting, even with preferential voting, but Sumner’s trite rehashing of this truth bores rather than informs.

  3. Gravatar of Tom Brown Tom Brown
    9. June 2015 at 08:18

    Lol!… Afterwards I hope the respondents were informed of the results of this study, and that somebody put that on video somewhere and that it’s now on youtube.

  4. Gravatar of E. Harding E. Harding
    9. June 2015 at 08:20

    “Other experiments elicit similar results. A study by Yale psychologist Geoffrey Cohen found that, absent partisan information, liberal students favored a welfare plan that was more generous than any existing policy, while conservatives favored a plan more stringent than any in existence.”
    -So voters’ policy preferences exist, voters just don’t read the details of the policies.

  5. Gravatar of Patrick R. Sullivan Patrick R. Sullivan
    9. June 2015 at 08:46

    Speaking of evidence;

    http://www.voxeu.org/article/inflation-expectations-spur-consumption

    ‘In a recent paper, we use novel and unique German micro-level data from January 2000 until December 2013 (D’Acunto, Hoang, and Weber, 2015) and find that inflation expectations are positively associated with the readiness to spend on durable goods.’

    Essentially, an announced increase in the VAT led Germans to think that they should buy durables before their prices increased.

  6. Gravatar of ssumner ssumner
    9. June 2015 at 10:29

    Jason, That’s a reasonable claim. Of course it’s consistent with what I wrote.

    Ray, Only you would think this has anything to do with Arrow’s theorem. Still the perfect idiot.

    E. Harding, You said:

    “So voters’ policy preferences exist, voters just don’t read the details of the policies”

    I don’t agree—what they favor depends how a question is asked. There is no black slate baseline from which to ask preferences, as the policies are far too complex for average people to contemplate.

    Patrick, Yes, and that’s what happened in Japan early last year.

  7. Gravatar of Bill Reeves Bill Reeves
    9. June 2015 at 12:22

    In another study the name of the bathroom for males was changed to women and that for females was changed to men. The details of each bathroom (a topic that is important to everyone) were provided up front. Yet virtually every male went into the room marked men’s and vice versa – except one pervy looking guy that kept snickering until the study coordinator threw him out. This study definitively proves that academics will burn other people’s money doing the silliest things. Which of course was definitively proven by a scholarly study at Harvard last year.

  8. Gravatar of Gordon Gordon
    9. June 2015 at 13:30

    “Afterwards I hope the respondents were informed of the results of this study, and that somebody put that on video somewhere and that it’s now on youtube.”

    Tom, there was something similar to this before the 2012 election in which Obama’s policies were presented as Romney’s policies to Obama supporters. The reactions of people upon learning the truth is priceless.

    https://www.youtube.com/watch?v=Skw-0jv9kts

  9. Gravatar of benjamin cole benjamin cole
    9. June 2015 at 15:19

    In one way this study about party affiliation gives me hope. If the GOP puts forth a “monetary growth policy” that is actually just a NGDP LT, then the GOP rank and file will accept it.

    I wonder if the GOP-hysteria about gold and inflation is just an Obama-era artifact.

    The Donks are just as callow, btw.

  10. Gravatar of Major_Freedom Major_Freedom
    9. June 2015 at 18:06

    Except when the public votes in politicians who vote on and then enforce an end to the gold standard.

    Then the public is shrewd and foresighted.

    Hahaha

  11. Gravatar of Edward Edward
    9. June 2015 at 18:30

    I thought you weren’t in favor of the gold standard, but “free market money”.

    More evidence that you conflate a kind of
    statism(gold standard) with the free market in a ridiculous and unsupportable way!

  12. Gravatar of Ray Lopez Ray Lopez
    9. June 2015 at 21:11

    @ssumner – “Ray, Only you would think this has anything to do with Arrow’s theorem. Still the perfect idiot” -?? wow, you really are that stupid? Not ignorant, but stupid. Can you not see the relationship between your post and Arrow’s paradox? (http://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem)

    What else can’t you see?

  13. Gravatar of Steve Steve
    9. June 2015 at 23:39

    So, would you support a ban on dihydrogen monoxide pollution in our reservoirs? California is leading the initiative, but Texas and Oklahoma have recently taken a large step backwards. Dihydrogen monoxide is also the number one most potent greenhouse gas, and once again, Texas is flooded with it.

    P.S.

    Why are you wasting our time with framing trivialities, when the NYT is exposing Marco Rubio, his irresponsible student loan debt, and his cover-up scandal to repay his own debt.

  14. Gravatar of Major_Freedom Major_Freedom
    10. June 2015 at 00:05

    Edward:

    “I thought you weren’t in favor of the gold standard, but “free market money”.”

    You thought right. Really, in thing I said contradicted that.

    My comment was not an advocacy. It was a sardonic comment.

    “More evidence that you conflate a kind of
    statism(gold standard) with the free market in a ridiculous and unsupportable way!”

    That comment right there is evidence that the concept of evidence can be, and in your case often is, shaped and formed by preconceptions, assumptions, beliefs and prejudices.

    Nothing I said suggests I “conflate” statism with a free market. I recommend that you learn the difference between an advocacy, and not an advovacy.

  15. Gravatar of Major_Freedom Major_Freedom
    10. June 2015 at 00:07

    Ray is right on this one about Arrow’s impossibility theorem.

    Sumner is just being sloppy on his soapbox and doesn’t like it when the responses are something other than rah rah rahs.

  16. Gravatar of Major_Freedom Major_Freedom
    10. June 2015 at 00:11

    Quite a bit of sloppy analysis on this post.

    Probably because the topic tends to be an emotionally charged one. Hating on majority voters whom you disagree with.

    For democratic minded, it especially stings. The majority is sullying the pristine ideal of democracy.

  17. Gravatar of Dan W. Dan W.
    10. June 2015 at 04:53

    Representative democracy as exists currently in the USA is a political illusion. When federal politicians respected the Constitution there was little pretense that representation was democratic, but at the same time the federal government was restrained in interfering in the affairs of the citizenry. In the 20th century federal officials rejected Constitutional restraints but at the same time they increased the illusion of democracy (electing Senators by popular vote, lowering the voting age, the voting rights act, etc). In the 21st century federal officials routinely violate the Constitution as they legislate and enforce the law. But no one is supposed to notice and instead we are supposed to buy into the hype of poll driven policy.

    Political illusion, as with money illusion, works until the dissonance between the illusion and reality grows too large to ignore. The hubris of the elite is their assumption they can “fake it” forever, or at least until past the next election or past the next margin call.

  18. Gravatar of collin collin
    10. June 2015 at 06:12

    Of course polls show mood afflilation or polls how the questions are asked? Look at the polls on foreign policy which shows Americans want less focus on foreign countries, military invasions and foreign aid. But when the questions are asked more directly on specific issues (Should we give more aid to Israel?, Should we use more military forces against ISIS?, Should we aid the Ukraine government?), most polls show voters want more foreign intervention.

    Again, I find the whole Dennis Hashert great irony in the long run. He spent most of his career voting for most anti-privacy laws (ie Patriot Act) and now he is paying for these votes. Didn’t he vote for banking law provisions to cover terrorist actions? So instead complaining about ‘government’ why not complain about the politicians that passed these laws?

  19. Gravatar of Ray Lopez Ray Lopez
    10. June 2015 at 07:01

    @MF – thanks MF, good to see that Great Minds think alike, and/or there’s a few people on this site who can still think critically. What Sumner calls “framing” and dismisses it as a sort of behavioral economics phenomena is in fact pivotal to Arrow’s Paradox, in the so-called “preference order”, which determines who wins what (simple example is the game ‘Rock-Scissors-Paper’). So in fact the way questions are framed are rationally related to how one will answer. See more here: http://en.wikipedia.org/wiki/Preference_%28economics%29

    To go back to Sumner’s example: if a voter is presented with a question that says: “The Blue Party backs this proposal ZYX, and here is the proposal”, the voter is presented with two questions: (1) do you support the Blue Party, and, (2) do you support this proposal? Rationally, a voter who is “Blue” would be expected to favor the proposal more so than a voter who is “Red Party”. It’s not behavioral economics at work, nor silliness as Sumner suggests, but rationality, preferences, and Arrow’s Paradox at work.

    I’ll say it again: I’m a legal and business guy, not the economist. I’m not supposed to be the one lecturing here, but you’re welcome reader.

  20. Gravatar of ssumner ssumner
    10. June 2015 at 07:13

    Ray, You said:

    “What else can’t you see?”

    Nothing of what you say. Arrow’s theorem is not about framing issues.

    Steve, You said:

    “Why are you wasting our time with framing trivialities, when the NYT is exposing Marco Rubio, his irresponsible student loan debt, and his cover-up scandal to repay his own debt.”

    Yeah, you are right, I need to start focusing on the important stuff. But at least I covered his traffic tickets, ignored by other bloggers.

    Collin, You said:

    “So instead complaining about ‘government’ why not complain about the politicians that passed these laws?”

    I’ve been doing that for 6 years.

  21. Gravatar of Morgan Warstler Morgan Warstler
    10. June 2015 at 10:17

    What is Sumner IS Ray Lopez?

  22. Gravatar of Tom Brown Tom Brown
    10. June 2015 at 12:58

    Morgan, was that supposed to read “What if Sumner is Ray Lopez?”

  23. Gravatar of TravisV TravisV
    10. June 2015 at 15:05

    Dear Commenters,

    What are the primary reasons for this headline? Saudi Arabia (negative global supply shock)?

    “Government Bond Yields in U.S., Europe, Japan Hit 2015 Highs”

    http://www.wsj.com/articles/government-bond-yields-hit-2015-highs-across-globe-1433943736

  24. Gravatar of TravisV TravisV
    10. June 2015 at 15:09

    Here’s Lars Christensen with the Saudi Arabia negative supply shock theory (3:36 in): https://www.youtube.com/watch?v=nCCIrHehSd8&feature=youtu.be&t=3m36s

  25. Gravatar of Major_Freedom Major_Freedom
    10. June 2015 at 18:13

    “Arrow’s theorem is not about framing issues.”

    The “framing” debate is a sideshow to the more important voting and monetary policy.

    That is all Ray is saying.

  26. Gravatar of bignurse bignurse
    10. June 2015 at 20:15

    Ray,

    Arrow’s theorem is not about “fairness”; it simply states that there are a set of criteria that are not met by rank-ordering methods. Despite your cited wiki page’s use of the “fairness”, Arrow never used this word to describe the criteria, much less say that any system that is fair requires them. In fact, that same wiki page states “Although Arrow’s theorem is a mathematical result, it is often expressed in a non-mathematical way with a statement such as “No voting method is fair,” [….] These statements are simplifications of Arrow’s result which are not universally considered to be true.”

    What Sumner is saying in this post does not map onto Arrow’s theorem at all, and your criticism of him for not seeing what is not there makes you the idiot with no credibility.

  27. Gravatar of Ray Lopez Ray Lopez
    11. June 2015 at 07:06

    @bignurse (big idiot): type CNTRL + F and “fairness” and you’ll see you’re the first person to use this term, not I. Who’s the idiot now, big mouth?

  28. Gravatar of Don Geddis Don Geddis
    11. June 2015 at 08:10

    @Ray Lopez: type CNTRL + F and “fair”, and you may reach greater enlightenment. (Although I wouldn’t bet on it.)

    My favorite part: Ray appears to believe that the connection between this post and Arrow’s theorem is “there’s no fair way of voting“, but ironically that’s a poor misrepresentation of both the post and the theorem.

  29. Gravatar of ssumner ssumner
    11. June 2015 at 12:40

    Big nurse, Just ignore him, he’s always like this.

Leave a Reply