Wayne Scott and the Non-Exhaustive Feel-Gooder
The Merc is doing an amusing but instructive thing these days; they're using the DefCon color code system to note the wonk potential of forthcoming entries. For instance, the latest in their great series on Portland's sit/lie ordinance was rated yellow, for elevated. This one might hit orange, but I've already made up a new phrase to boil it down into something resembling English--the non-exhaustive feel-gooder--so we're off to a good start.
Today's topic is polling and the release is GOP-serving Moore Information, trying to help Wayne Scott and the House Republicans sell their plan to use 1% of budget to fund the Oregon State Police. As you may have been following, the GOP's tack in this debate has been to hold fast to the 1% solution, arguing that there was plenty of money to go around, and there's no reason not to bring troopers to full strength. It sounds like a reasonable argument, except that there's no real suggestion that OSP actually needs or deserves 1% of revenue as opposed to a normally based budget, one on need and balance of other priorities. And it rests entirely on the premise that there's plenty of money all of a sudden--which conveniently ignores the fact that the last two budgets have cut to the bone and left ALL services wanting. (And when they tell you the budget is 20% bigger, that's BS too, because about half of it is going back in kicker rebates. The true total is about 11%).
So that's their shtick, conveniently ignoring much of the current reality. And as if to mimic their clients' pattern of decision making and loyalty to a position less on facts than appealing marketing, Moore's interviewers asked a question tailor made to give Wayne Scott the numbers he was looking for. It's Question 7 of their "data presentation" (which conveniently skips questions 1-5 and also any potential items after #7), and it asks:
Mutually exclusive means that you can't logically choose more than one option. For this question, if we assume the routine "don't know" and refusal options were included but not shown here, the choice offered is indeed mutually exclusive. Either the OSP is funded by new taxes, or it is not. Even if the best answer is to use both existing revenues AND new taxes, that's still a new tax and thus option 2 is the best choice. A question is mutually exclusive if only one response is true for you, making all other options automatically false. A good example is "own vs. rent"--you do one or the other, can't be both.
So it passes that test, but fails on exhaustiveness. An exhaustive response set means no options are left unrepresented. Say you want to know what people's favorite flavors of ice cream are. You might offer the choices of vanilla, chocolate, and strawberry. That's all well and good, and you might get most of your answers among those three, but you've left people who have a different favorite--or don't like ice cream at all--without an option. In this case if you don't care about anything beyond the big three, an "other" category is a good catch all, with a "don't like ice cream" option added to make it even stronger.
So let's look at those choices in our survey again: you can either fund it with existing revenues or make new taxes. But wait--you can also pass bonds. Or you could cut services. Or you could contract out some services and save money while improving service levels. There are a number of ways you could solve the "OSP needs money to do stuff" problem--the same problem, incidentally, that every other agency has every budget cycle. But OK, public safety should be near the top. So why are these the two options? It's not because some scientist wanted to create a rigorous hypothetical scenario; the options simply represent the two legislative points of view as the Republicans see them.
How do we know this? Because if you're going to try to find out whether your position is popular or not, you'd pose the actual options at hand, right? In this case that would be using current revenues for 100 troopers, or 1% of the budget for 140 troopers. Those are the two proposals on the table. The Democrats have offered to fund 140 troopers as well, but would seek additional revenue for it because of other priorities in the budget. (Incidentally, all of this may become moot if momentum builds for dedicating beer tax money to the OSP...a new tax, obviously, but one with some bipartisan support, apparently).
But did the GOP present the two options honestly? Hell no--that might have caused respondents to say "hmmm....well, I don't want new taxes, but I do want to make sure we have all our bases covered, like for education and health care, which are just as bad off from the last few budgets as OSP. So let's fund 100 troopers with what we've got now." See, that would mean people actually favor the Democratic plan to balance everything out and bring OSP strongly up from previous levels, if not to peak levels all at once. And Wayne Scott doesn't want to hear anything about that, because then he'll just have wasted a bunch of money on the survey.
The non-exhaustiveness of the question is a huge problem for Wayne, however, because it eliminates the possibility that they can claim with any accuracy that Oregonians favor the GOP's plan for the OSP. The reality is that BOTH bills under strongest consideration use existing revenues rather than new taxes, so how can the Republicans suggest that the results of the question mean it's THEIR plan people favor? Obviously if given a choice, people would like to fund things out of existing money rather than raising more...
...which is where the second part of our title phrase comes in. One of the best ways to skew a result to your favor is to make the pollster-preferred response sound so good and so much better than the other options that you'd have to have a screw loose to answer differently. For instance: "Would you rather be A) Rich, or B) Stupid?" You might as well write up the press release before the results come back--"Oregonians Overwhelmingly Prefer Wealth to Idiocy."
By creating a question where one answer seems logically obvious and preferable, you insure that everybody "feels good" about their answer. A very common feel-gooder tactic is to refer to children, senior citizens or animals where appropriate, and often to pit them against less universally loved groupings, such as "Should the government keep children from dying unnecessarily by covering their health care, or should it preserve the right for smokers to afford cigarettes?" Even if you're wary about a tobacco tax as a way to pay for health care, it's hard for most people to say they are rooting for the smokers instead of the kids.
So to ask people "would you rather pay for troopers out of current funds, or be charged extra money for them" pits not spending more of your money against spending more of it, and who's going to pick getting taxed more? (Well, Moore got 21% to pick it, which to me is actually an indication that most of those 21% are actually following the OSP debate and are explicitly trying to pick the Democratic plan for full coverage). By making the non-preferred option sound really bad or illogical, one is able to artificially pump up the response for the preferred option.
Another good way to do that is to prime the pump, and Moore does that in Question 6 by asking a more generalized version of the same concept before getting specific. Once you have heard yourself tell the interviewer that you think government has enough money, you'd look like an idiot (you imagine) to say in the very next question that the government needs to raise taxes. The danger of pump-priming is also why it's of concern that Questions 1-5 are not shown in the presentation; it's entirely possible some of those questions are also intended to lead the respondent down the primrose path to the answer Wayne Scott is looking for.
An honest pollster looking for a valid answer to the OSP question would ask respondents if they favor adding some troopers with existing money, more troopers with additional tax revenue, or more troopers with existing money but at the expense of other previously shortchanged programs. That is the question being debated, and it would indeed be of great interest and value if someone were to ask it in a survey. But Wayne Scott's caucus doesn't give a rip which of those ideas you like best; that would be acting in the interest of the state constituency. What is obvious from the polling questions in this survey is that their goal is to create the impression that the Republicans' OSP plan is the one to go with--and they're not afraid of resorting to the non-exhaustive feel-gooder to do it. Boooo.
Today's topic is polling and the release is GOP-serving Moore Information, trying to help Wayne Scott and the House Republicans sell their plan to use 1% of budget to fund the Oregon State Police. As you may have been following, the GOP's tack in this debate has been to hold fast to the 1% solution, arguing that there was plenty of money to go around, and there's no reason not to bring troopers to full strength. It sounds like a reasonable argument, except that there's no real suggestion that OSP actually needs or deserves 1% of revenue as opposed to a normally based budget, one on need and balance of other priorities. And it rests entirely on the premise that there's plenty of money all of a sudden--which conveniently ignores the fact that the last two budgets have cut to the bone and left ALL services wanting. (And when they tell you the budget is 20% bigger, that's BS too, because about half of it is going back in kicker rebates. The true total is about 11%).
So that's their shtick, conveniently ignoring much of the current reality. And as if to mimic their clients' pattern of decision making and loyalty to a position less on facts than appealing marketing, Moore's interviewers asked a question tailor made to give Wayne Scott the numbers he was looking for. It's Question 7 of their "data presentation" (which conveniently skips questions 1-5 and also any potential items after #7), and it asks:
“Which of the following more closely reflects your opinion about funding for Oregon State Police? The Legislature should dedicate funding for the Oregon State Police out of existing revenuesFor every question a pollster asks--even the basic demographic questions--he or she needs to make sure the question is sound enough to yield an accurate result, and there's kind of a mental checklist you can go through to make sure it doesn't hit any of the pitfalls. One quick rule of thumb is to make sure the question is both exhaustive and mutually exclusive.
OR
The Legislature should establish a new tax to dedicate funding for the Oregon State
Police?”
Mutually exclusive means that you can't logically choose more than one option. For this question, if we assume the routine "don't know" and refusal options were included but not shown here, the choice offered is indeed mutually exclusive. Either the OSP is funded by new taxes, or it is not. Even if the best answer is to use both existing revenues AND new taxes, that's still a new tax and thus option 2 is the best choice. A question is mutually exclusive if only one response is true for you, making all other options automatically false. A good example is "own vs. rent"--you do one or the other, can't be both.
So it passes that test, but fails on exhaustiveness. An exhaustive response set means no options are left unrepresented. Say you want to know what people's favorite flavors of ice cream are. You might offer the choices of vanilla, chocolate, and strawberry. That's all well and good, and you might get most of your answers among those three, but you've left people who have a different favorite--or don't like ice cream at all--without an option. In this case if you don't care about anything beyond the big three, an "other" category is a good catch all, with a "don't like ice cream" option added to make it even stronger.
So let's look at those choices in our survey again: you can either fund it with existing revenues or make new taxes. But wait--you can also pass bonds. Or you could cut services. Or you could contract out some services and save money while improving service levels. There are a number of ways you could solve the "OSP needs money to do stuff" problem--the same problem, incidentally, that every other agency has every budget cycle. But OK, public safety should be near the top. So why are these the two options? It's not because some scientist wanted to create a rigorous hypothetical scenario; the options simply represent the two legislative points of view as the Republicans see them.
How do we know this? Because if you're going to try to find out whether your position is popular or not, you'd pose the actual options at hand, right? In this case that would be using current revenues for 100 troopers, or 1% of the budget for 140 troopers. Those are the two proposals on the table. The Democrats have offered to fund 140 troopers as well, but would seek additional revenue for it because of other priorities in the budget. (Incidentally, all of this may become moot if momentum builds for dedicating beer tax money to the OSP...a new tax, obviously, but one with some bipartisan support, apparently).
But did the GOP present the two options honestly? Hell no--that might have caused respondents to say "hmmm....well, I don't want new taxes, but I do want to make sure we have all our bases covered, like for education and health care, which are just as bad off from the last few budgets as OSP. So let's fund 100 troopers with what we've got now." See, that would mean people actually favor the Democratic plan to balance everything out and bring OSP strongly up from previous levels, if not to peak levels all at once. And Wayne Scott doesn't want to hear anything about that, because then he'll just have wasted a bunch of money on the survey.
The non-exhaustiveness of the question is a huge problem for Wayne, however, because it eliminates the possibility that they can claim with any accuracy that Oregonians favor the GOP's plan for the OSP. The reality is that BOTH bills under strongest consideration use existing revenues rather than new taxes, so how can the Republicans suggest that the results of the question mean it's THEIR plan people favor? Obviously if given a choice, people would like to fund things out of existing money rather than raising more...
...which is where the second part of our title phrase comes in. One of the best ways to skew a result to your favor is to make the pollster-preferred response sound so good and so much better than the other options that you'd have to have a screw loose to answer differently. For instance: "Would you rather be A) Rich, or B) Stupid?" You might as well write up the press release before the results come back--"Oregonians Overwhelmingly Prefer Wealth to Idiocy."
By creating a question where one answer seems logically obvious and preferable, you insure that everybody "feels good" about their answer. A very common feel-gooder tactic is to refer to children, senior citizens or animals where appropriate, and often to pit them against less universally loved groupings, such as "Should the government keep children from dying unnecessarily by covering their health care, or should it preserve the right for smokers to afford cigarettes?" Even if you're wary about a tobacco tax as a way to pay for health care, it's hard for most people to say they are rooting for the smokers instead of the kids.
So to ask people "would you rather pay for troopers out of current funds, or be charged extra money for them" pits not spending more of your money against spending more of it, and who's going to pick getting taxed more? (Well, Moore got 21% to pick it, which to me is actually an indication that most of those 21% are actually following the OSP debate and are explicitly trying to pick the Democratic plan for full coverage). By making the non-preferred option sound really bad or illogical, one is able to artificially pump up the response for the preferred option.
Another good way to do that is to prime the pump, and Moore does that in Question 6 by asking a more generalized version of the same concept before getting specific. Once you have heard yourself tell the interviewer that you think government has enough money, you'd look like an idiot (you imagine) to say in the very next question that the government needs to raise taxes. The danger of pump-priming is also why it's of concern that Questions 1-5 are not shown in the presentation; it's entirely possible some of those questions are also intended to lead the respondent down the primrose path to the answer Wayne Scott is looking for.
An honest pollster looking for a valid answer to the OSP question would ask respondents if they favor adding some troopers with existing money, more troopers with additional tax revenue, or more troopers with existing money but at the expense of other previously shortchanged programs. That is the question being debated, and it would indeed be of great interest and value if someone were to ask it in a survey. But Wayne Scott's caucus doesn't give a rip which of those ideas you like best; that would be acting in the interest of the state constituency. What is obvious from the polling questions in this survey is that their goal is to create the impression that the Republicans' OSP plan is the one to go with--and they're not afraid of resorting to the non-exhaustive feel-gooder to do it. Boooo.
posted by Anonymous at 11:30 AM
|
<< Home