A Haven For Straight Talk: <b>Bad Surveys Are Worse Than No Surveys</b>

Last month I wrote about the importance of measurement. Many firms do little or no marketing measurement and are surprised when their programs yield no definitive results. I've said it before (but not, I think, in this column), and I'll say it again ' marketing without measurement is like basketball without baskets; lots of dribbling, but not much point to it.

Marketing measurements generally come in two flavors; quantitative and qualitative. We'll talk next month about some quantitative measurements relative to marketing at your firm. This month, though, we'll discuss the ever-popular survey. On a scale of 1-7, how likely are you to read the rest of this article?

Why do you ask?

Surveys are great. I love surveys. Good surveys, that is. Bad surveys are worse than no surveys, though. Why? Because they are a waste of time, steer you in wrong directions and give you a false sense of having accomplished something. Want an instant rule to help you determine if you've been asked to take a badly designed and/or useless survey? Look to see if they ask for your gender or age. There are almost no circumstances under which those two pieces of information are useful or appropriate. Health, fashion and lifestyle issues are about it. If your marketing doesn't touch those areas, don't ask me if I'm a boy or girl and how old I am. It's none of your business.

Look at all of your questions that way. Are they not only appropriate, but useful. What are you going to do with the information once you have it? Pretend you got 100% participation on the survey and that everyone answered the question in a revealing manner; is there action you can take based on your new information? If not, it's a useless question. Don't ask it. You only have a short amount of time before your participants become bored or irritated. Don't waste their time and yours on questions that simply don't matter.

Ugly Surveys Yield Ugly Results

Believe it or not, a study was done, years ago, on whether or not the attractiveness of a survey affects its results. Not surprisingly, the answer is “Yes.” People are more inclined to be peevish and negative when holding a sloppy, ill-prepared document. The same holds true for questions that are long, unclear or rambling. Your participants are doing you a favor. Don't abuse that gift by offering up an ugly or badly laid-out survey. Before actually going live, do a test run with some employees. Ask them to critique the survey's “look and feel.” Ask if any of the questions were unclear or ambiguous.

On A Scale Of 1 to 2…

Depending on what you plan to do with your information, you should either give people a wide range of responses, or only two choices. When you will be using the survey to rank a choice against several other options it's good to give people a range of at least five answers. I like to use seven. For example:

“Was the location for the event convenient for you? Please rate your satisfaction from 1-7 with 1 being 'totally dissatisfied' and 7 being 'totally satisfied.'”

When you get all the answers back, you'll have a good range to compare to surveys taken for other event locations. What you'll look for is a location that tilts high, but with no real volume of 1s or 2s. A good average score isn't enough; a location with all 4s is much better than one with half 7s and half 1s.

If, however, you are deciding whether or not to do something at all, or selecting between one of two choices, give your participants the same deal. If you've held a seminar and want to know if you should do it again, ask that question. People are often hesitant to provide a strong negative response to a question if there's a way out. Give then the choice of 7, and they'll choose 3. If everyone chooses 3, you may still think it's an OK idea to do the event again. Nope. They were just too embarrassed to answer “1.” If you really want to know, ask. If you don't want to know, don't ask.

Probing Ain't Just For NASA

Sometimes you know what you don't know. And sometimes you don't even know what you don't know. If you've managed to get someone to sit still long enough to take your survey, use that opportunity to ask at least one “probing” question. By “probing” I mean that the question should allow the participant to get outside the box a bit. These can't be numerical or check-box questions. They need a couple of lines of space.

If you're surveying attendees at a seminar, ask them what topics they'd like covered in the future. Ask who their “dream speaker” would be. Ask where they'd love or hate to attend an event. Ask what the best and worst “gimmes” they ever received at an event were. Ask if there are other events they attend. Ask who should have been invited but wasn't. There are all sorts of neat questions that can get inside folks' heads and provide useful information for clever marketing people to leverage.

Go In With The Exit In Mind

As I said earlier, every question should have a purpose. So should the whole survey. Before you start putting one together, take the time to clearly state why you are creating the survey and what you will do with the information. If you are surveying clients about their satisfaction with your firm, will you take steps to address any issues that surface? If not, don't do the survey. Because ' guess what? ' when you ask someone if they're satisfied, and they say, “No,” they expect you to do something about it. If you don't, you've just moved the ball … backwards.

Surveys are great marketing tools. Done right they can provide critical data to help you improve your programs. Done wrong, they can irritate the very people you're trying to impress. Take time, pay attention and do them right.

Last month I wrote about the importance of measurement. Many firms do little or no marketing measurement and are surprised when their programs yield no definitive results. I've said it before (but not, I think, in this column), and I'll say it again ' marketing without measurement is like basketball without baskets; lots of dribbling, but not much point to it.

Marketing measurements generally come in two flavors; quantitative and qualitative. We'll talk next month about some quantitative measurements relative to marketing at your firm. This month, though, we'll discuss the ever-popular survey. On a scale of 1-7, how likely are you to read the rest of this article?

Why do you ask?

Surveys are great. I love surveys. Good surveys, that is. Bad surveys are worse than no surveys, though. Why? Because they are a waste of time, steer you in wrong directions and give you a false sense of having accomplished something. Want an instant rule to help you determine if you've been asked to take a badly designed and/or useless survey? Look to see if they ask for your gender or age. There are almost no circumstances under which those two pieces of information are useful or appropriate. Health, fashion and lifestyle issues are about it. If your marketing doesn't touch those areas, don't ask me if I'm a boy or girl and how old I am. It's none of your business.

Look at all of your questions that way. Are they not only appropriate, but useful. What are you going to do with the information once you have it? Pretend you got 100% participation on the survey and that everyone answered the question in a revealing manner; is there action you can take based on your new information? If not, it's a useless question. Don't ask it. You only have a short amount of time before your participants become bored or irritated. Don't waste their time and yours on questions that simply don't matter.

Ugly Surveys Yield Ugly Results

Believe it or not, a study was done, years ago, on whether or not the attractiveness of a survey affects its results. Not surprisingly, the answer is “Yes.” People are more inclined to be peevish and negative when holding a sloppy, ill-prepared document. The same holds true for questions that are long, unclear or rambling. Your participants are doing you a favor. Don't abuse that gift by offering up an ugly or badly laid-out survey. Before actually going live, do a test run with some employees. Ask them to critique the survey's “look and feel.” Ask if any of the questions were unclear or ambiguous.

On A Scale Of 1 to 2…

Depending on what you plan to do with your information, you should either give people a wide range of responses, or only two choices. When you will be using the survey to rank a choice against several other options it's good to give people a range of at least five answers. I like to use seven. For example:

“Was the location for the event convenient for you? Please rate your satisfaction from 1-7 with 1 being 'totally dissatisfied' and 7 being 'totally satisfied.'”

When you get all the answers back, you'll have a good range to compare to surveys taken for other event locations. What you'll look for is a location that tilts high, but with no real volume of 1s or 2s. A good average score isn't enough; a location with all 4s is much better than one with half 7s and half 1s.

If, however, you are deciding whether or not to do something at all, or selecting between one of two choices, give your participants the same deal. If you've held a seminar and want to know if you should do it again, ask that question. People are often hesitant to provide a strong negative response to a question if there's a way out. Give then the choice of 7, and they'll choose 3. If everyone chooses 3, you may still think it's an OK idea to do the event again. Nope. They were just too embarrassed to answer “1.” If you really want to know, ask. If you don't want to know, don't ask.

Probing Ain't Just For NASA

Sometimes you know what you don't know. And sometimes you don't even know what you don't know. If you've managed to get someone to sit still long enough to take your survey, use that opportunity to ask at least one “probing” question. By “probing” I mean that the question should allow the participant to get outside the box a bit. These can't be numerical or check-box questions. They need a couple of lines of space.

If you're surveying attendees at a seminar, ask them what topics they'd like covered in the future. Ask who their “dream speaker” would be. Ask where they'd love or hate to attend an event. Ask what the best and worst “gimmes” they ever received at an event were. Ask if there are other events they attend. Ask who should have been invited but wasn't. There are all sorts of neat questions that can get inside folks' heads and provide useful information for clever marketing people to leverage.

Go In With The Exit In Mind

As I said earlier, every question should have a purpose. So should the whole survey. Before you start putting one together, take the time to clearly state why you are creating the survey and what you will do with the information. If you are surveying clients about their satisfaction with your firm, will you take steps to address any issues that surface? If not, don't do the survey. Because ' guess what? ' when you ask someone if they're satisfied, and they say, “No,” they expect you to do something about it. If you don't, you've just moved the ball … backwards.

Surveys are great marketing tools. Done right they can provide critical data to help you improve your programs. Done wrong, they can irritate the very people you're trying to impress. Take time, pay attention and do them right.