Hart Business Research

Articles

A Dozen Ways to Improve Readership Survey “Hearing”

Enthusiast magazines do well when they listen to their readers' needs, interests, and passions. However, some ways of listening are like a hearing aid with a dead battery. The dead battery in a survey can be badly-worded or boring questions, intimidating length, or an unclear purpose. The following 12 tips should help you hear.

1. Start by writing down the purpose of the survey, to stay focused

Surveys are most effective when you are doing them for a specific purpose, such as a remaking editorial content and design, informing advertisers, evaluating an acquisition, or exploring new product ideas. This keeps your questions focused, and gives you criteria for evaluating which questions to keep when the inevitable time comes to shorten the survey. Consult with your editorial, design, and advertising departments as to their needs, but stick to the specific purpose of the survey.

2. Phrase the questions in your customer’s language

Avoid using terms that you often use in the company, but that customers may not know, such as magazine “departments” or “features”. Ask questions from a market-wide perspective, not from your company’s market niche. A good way to understand your customer’s language and perspective is to individually interview 6 – 8 subscribers, in person preferably. A new employee (not yet steeped in company thinking) or consultant will be able to conduct the most neutral and useful interviews. You’ll use the interviews to explore the issues you plan to cover in your survey, then build your survey using the interviewees' perspectives and language.

3. Use fill-in-the-blank (not always multiple-choice) to facilitate fresh ideas

Multiple-choice answers make data entry easy. However, fill-in-the-blank answers give you a richer and more accurate picture. If you aren’t completely sure of the full range of possible answers, give respondents a blank to write in their answer. For example, the question, “What is your favorite astronomy Web site?” could yield some great new competitor info if you give respondents a blank line. If you give them your pre-set choices, you get less information back. A good compromise for important questions is first to ask a fill-in-the-blank question, and then later ask a multiple-choice version. You can also do the “D. Other:________” multiple-choice option, but you’ll get a less objective response, since the other predefined options tell respondents what they are supposed to think of as “competitors”.

4. Make sure you use a consistent set of data values

For example, when asking what someone did in the last year, always use the same time period, such as “in 2003” or “in the last 12 months”, but not both. This way you can compare and cross-tab data reliably across questions. Also consider the data values you have used in past surveys, and what your competitor publications have used. Income brackets are a good example. If data values are consistent between questions, between surveys over time, and versus competitor’s surveys, you then have more powerful data from which to draw conclusions.

5. Run a field pre-test of the “final” survey to make sure all questions work as planned

Mail the draft out to 30 – 40 subscribers along with a letter asking them to fill out the survey, note any questions which are confusing, and mark how long it took them to take the survey. Then shorten and clarify your survey accordingly. The pretest will save possible future headaches from poor response rates, misunderstood questions, and unhappy subscribers.

6. Keep the survey as brief as possible

Respondents should be able to complete it in less than 10 minutes. However, the more passionate someone is about your magazine, the longer you can make the survey (up to a point!).

7. Pick your sample carefully

Most of the time you want a sample which is representative of your subscriber base — pick every Nth (every 50th, 500th, etc.) subscriber on your list. However, if you want the more passionate, involved subscribers in your sample, do the following: 1) Offer an incentive that appeals particularly to them, such as a raffle for a specialized product. 2) Mail the survey to more subscribers than you think you’ll need, and use the first surveys you get back. Hard-core enthusiasts send theirs in first. 3) Make the survey longer, as only the most committed will respond.

8. If you're doing a Web survey, offer a paper option if not everyone uses the Web regularly

If you want a representative sample, use a Web survey only if your subscriber base is equally comfortable with the internet and paper. An internet-based survey will disproportionately attract respondents who are comfortable with the internet; a paper-based survey will appeal to those who are comfortable with paper. Since magazines are paper, a paper-based survey of subscribers will attract a more representative sample. However, a Web-based survey costs significantly less to conduct; if cost is an issue, try doing one tandem survey of your subscribers by mail/print and by e-mail/Web, and see how different the responses are in each medium. You can then use these results to help interpret future online surveys.

9. Make subscribers feel needed and knowledgeable

Send out a letter ahead of time, announcing the survey is coming and how important their response is. When you send out the survey include $1 plus a nice note saying how much you appreciate and value their thoughts and how you’ll use their input to improve their magazine.

10. Reassure them about survey anonymity and purpose — build trust

Insert a brief sentence or two at the start of the survey to reassure the respondent you will keep their responses anonymous, and explain how you will use the information and ideas the respondent gives.

11. Think through your results, don’t just run some statistics

Look through the data for larger patterns and themes. Look at what people have written in. Make sure the responses make sense, and discard data for that question if not. Compare this survey’s results with surveys from the past, from competitors, and even from unrelated markets. Creative ideas will emerge from the sea of numbers.

12. Finish the story

Note in a future magazine issue how helpful the survey responses were, and how you used them to improve something for your readers. Let them know you heard them.