Chapter 10 Notes

Página 1

Chapter 10 Data Collection Through Surveys

Survey provide a “point in time” snapshot of respondents’ attitudes, beliefs, and behaviors.

Advertising and marketing researcher typically utilize one of four survey data collection methods:

Personal (face-to-dace) interviews

Telephone interviews

mail or paper self-completed surveys

Online surveys (Hybrid methods, which combine one or more of the prior methods, are also increasing in usage)

Methods of collection survey information

Personal Intervies

Personal interviews occur when an interviewer administers a survey to a respondent in a face-to-face setting.

There are two main types of personal interviews: intercept and pre-recruited

An Intercept interview recruits respondents “on the spot”

For example: as an individual is walking through a shopping mall, grocery store, airport, or train station. All intercept interviews are generally conducted in the same way regardless of the location.

Mall intercept interview are probably the most common setting for intercept interview.

Individual in these cases are initially contacted first by mail, online, or telephone at which time arrangements for a personal interview are made

Personal interviews VS. other forms of surveys techniques, generally provide the highest level of data quality due to personal administration of the survey by a trained interviewer and the face-to-face contact between then interviewer and interviewee.

In addition to data quality, personal interviews have several advantages over other data collection techniques:

They work extremely well when the questionnaire is very long and/or complex

They permit the use of visual stimuli, such as the view of television commercials

They permit the interviewer to make very certain that the respondent understands individual survey questions or complicated instructions

Disadvantage:

High cost

High amount of time required for completion

Personal Interviews can also be conducted in-home through door-to-door interviews. However, due to the very high coat and time, as well as safety concerns, this technique is rarely used in advertising research.

Telephone Interview

Telephone Interview are typically conducted by a team of trained interviewers telephoning from a central location

Almost all telephone interviews use Computer Assisted Telephone Interviewing (CATI) where a telephone interviewer reads the survey questions off a computer and types in the responses as they are provided

CATI telephone interviewing offers many advantage over paper-and pencil telephone interviews: Decrease error and helps not inconsistencies in respondent answers

Telephone interviews, similar to personals, provide an opportunity for an interviewer to explain complicated instructions and questions, although explanations in these areas do tend to be more difficult over the telephone vs. a personal setting.

Advantage:

Especially those that are assisted by CATI, are able to administer interviews with very complex skip patterns. This is because the computer automatically calls up the next appropriate question based on the respondent’s current answer.

Have lower marginal costs because individuals in the sample frame can be contacted several times if they are not available during the first contact

Can more efficiently obtain a true random sample, increasing the generalizability and representativeness of the results.

Disadvantage:

Questionnaire length must be relatively short. It is very difficult to keep respondent on the telephone for an exit did period of time.

Question complexity must be relatively simple. complex or detailed questions and scales are difficult for respondents to remember and answer accurately

It is difficult to collect sensitive data as individuals are often unwilling to give this information to strangers

Refusal rates are high

It is extremely difficult to use visual stimuli, for example, having a respondent view a television commercial

Care must be taken when selecting the sample for a telephone survey.

A researcher providing or purchasing a list of names and telephone numbers must make very certain that the list is comprehensive and without bias.

Mail Survey

Maile surveys entail mailing each potential respondent a package containing a cover letter, the survey questionnaire, instructions for completion and return, and a stamped envelope addressed to the research company conducting the research. (an incentive - money or small gift)

Mail surveys thus differ from personal and telephone interviews because there is no personal interaction between the respondent and the interviewer.

Respondents simply fill out and return the questionnaire at their convenience.

Mail surveys may be conducted in one of two ways:

First, mailings to a selected sample of the population can be conducted - similar to telephone surveys, the list of names and addresses can be generated internally or purchased

Second, a mail panel can be used - A mail panel is a continuing group of individuals who have agreed to participated in survey research studies

panels are created and maintained by independent marketing research organizations such as Market Fact, National Family Opinion, and the National Panel Diary (NPD) Group.

Panel sizes, which range from 100,000 individuals to over 1 million, provide several advantages over mailings to the general population:

Response rates tend to be significantly higher. Panel members have agreed to answer questionnaires mailed to them.

Cost efficiencies tend to be greater. Higher levels of response lower the per-interview cost

The selection of individuals with certain demographic or product usage characteristics can often be made without prescreening. This information has already been connected on all panel members by the marketing research company

Prescreening of low-incidence respondents can be efficiently conducted. A postcard with screening characteristics can be mailed to a large segment of the panel. Those with the appropriate characteristic are then mailed the main questionnaire.

Strengths of mail surveys:

Cost efficiency. The coast per completed interview of properly designed mail surveys can be considerably lower than comparable personal or telephone interviews.

Respondent convenience. Respondents may be more willing to participate because they can complete the questionnaire at their convenience.

Weaknesses of mail surveys:

Low response rate. Respondents may not complete to return the questionnaire, causing problems with the integrity of data.

Limited questionnaire length and complexity. In theory, one can develop mail questionnaires of extreme length and complexity increase.

Extended timing. Response is mediated by mail delivery and thus data collection takes much longer than other methods.

this survey, which generally ask a limited number of questions, are most commonly used to measure customer satisfaction and the quality of a customer’s brand-related experience.

Online Surveys

Online surveys are administered over the Internet or more recently, through one’s mobile phone.

These surveys are typically posted on a central Web site that allows individuals to respond on their computer or phone at a time of their choosing.

3 components: the invitation, questionnaire design and posting, and data collection

Strengths:

First, many online survey sites allow for random assignment of individuals to different groups and for the random presentation of stimulus materials. - this strength is of special importance when one uses these sites to collect experimental data.

Second, question bias can be significantly reduced or eliminated through randomization of question response options.

Third, data collection is typically very quick, as questionnaires from multiple members of the sample can be processed simultaneously

Fourth, costs are typically lower vs. other forms of data collection

The primary caution regarding online surveys related to sample composition. - Since internet access is not yet universal and comfort levels of internet usage differ across age groups, there is always the question of sample representativeness. Some individuals in the sample frame may simply be unavailable through this data collection option

Hybrid (Mixed-Mode) Surveys

Researchers are increasingly taking the position that in some circumstances, the “best” way to obtain data is one that is actually multi-modal, that is, The use of more than one approach to data collection.

Multi-mode sampling is the process of accessing respondents using more than one mode for the same study

This could happen two ways:

Different modes may be used only at the contact stage, while the sample itself remains on one mode-for example, contacting someone via e-mail to request that they do a study via phone or recruiting people via phone for an online or mail study.

Multi-mode may also apply to the questionnaire, where some portion of the sample answers online and another portion may answer via telephone.

Multi-mode is important because 30% of the US population does not have internet access

Order, less affluent, rural populations are less likely to be online. But conducting an entire study via phone, mail, or face-to-face could be cost prohibitive.

A multi-mode solution could be employed to conduct as much as possible online, then supplementing with a phone or mail sample to fill in the population gaps.

Hybrid design have the potential to increase response rate and as a consequence increase data quality

2 limitation:

First, they are complex to put into practice, with different sample frames required for each mode of data collection

Second, hybrid designs assume that the same person ( or a very similar one) would “give the same response to the same question irrespective of mode, sampling frame and sampling method.”

Criteria for selecting a data collection method

Six factors should be considered when determining which survey method is the best of ra particular research study:

Cost

Timing requirements

Sample, Interview, and administrative control

Informational needs

Complexity of the topic and questionnaire

Interview length

Cost

cost is an extremely important consideration in the conduct of survey research.

The cost of any particular research study reflects questionnaire length, required response rate, geographic coverage, and sample characteristics

Study cost is the initial 首要的 starting point in evaluating the suitability of alternative data collection methods

Timing requirements

telephone and online surveys generally require the least amount of time to collect required data

Telephone survey is really quick for two reason:

all training and coordination can be performed at a central facility

teams of interviewers can work simultaneously

mail surveys tend to take a great amount of time

personal interviews are almost always the slowest from of data collection - especially simple sizes increase

Sample, Interview, and Administrative Control

Sample Control refers to the extent to which a researcher can control who responds to the survey questionnaire

it is important that the individual who meets the sample definition is the person who responds to the survey

Personal and telephone interviews provide the greatest degree of sample control - the personal contact and immediate nature of personal and telephone interviews help to ensure that the target respondent is the one providing responses

Mail and online interviews provide considerably less control

Interview control refers to the extent to which a researcher has control over the circumstances in which a respondent provides his or her answers to the survey.

Personal and telephone interviews provide the greatest degree of interview control

the immediate nature of personal and telephone interviews permits an interviewer to address ambiguities or errors in response at the time they occur

online data collection techniques provide a relatively lower degree of interviewer control

Mail surveys provide the lease about of interview control

Administration Control refers to the degree to which a researcher is able to monitor interviewer quality, which can be done only in telephone or personal interviews - Supervisors at the telephone facility can monitor the quality of each telephone interviewer and immediately correct any problems with interviewer tone, style, or questionnaire administration

Personal interviews provide the least amount of administrative control.

Informational Needs

Some survey techniques work better than others when the survey topic is of a sensitive or personal nature

Respondents tend to be more honest and candid in telephone and especially mail interviews vs. personal interviews.

Telephone interviews tend to elicit fewer “socially acceptable” responses

Mail vs. telephone surveys were the better way to collect sensitive, personal information, in the case reports of substance abuse

personal interviews appear to be a better means for collecting complete and considered responses

Complexity of the topic and questionnaire

A skip pattern instructs the respondent or interviewer to jump from one point to another on the questionnaire depending on an individual’s response

personal interviews are most appropriate when both the topic an questionnaire are complex

Telephone interviews word well when the issue is complex and questionnaire complexity is entirely due to skip patterns

Telephone interviews are not appropriate when questionnaire complexity is due to verbal or question complexity

Online interviews are appropriate for relatively simple topics with or without questionnaire complexity

Interview length

Interview length refers to the amount of time required for a typical respondent to complete the survey

Interview length tends to increase as the absolute number of questions,question complexity, and the number of open-end question rises

Personal contact is essential of long surveys - the personal,face-to-face nature of the interview permit an interviewer to keep the respondent interested, attentive, involved, and a continuing participant in the research

mail, telephone, and interactive data collection techniques are generally appropriate only for shorter interviews

Response rate, Nonresponse rate, and nonresponse error

Response rate is an important way to evaluate the integrity pf survey data

Response rate, a crucial 重要的 element of a research study, refers to the percentage of the valid sample who participate in the research by completing an interview or survey. - a low response rate can severely decrease the validity and generalizability of collected data, thereby reducing the ability of the data to positively and accurately contribute to the decision-making process.

A response rate is calculated by dividing the number of respondents completing the survey by the number of respondents in the valid sample

Response rate + nonresponse rate = 100%

A high response rate, and thus a low nonresponse rate, generally indicates that there are no meaningful differences between those who responded to the survey and those who did not

A low response rate, and thus a high nonresponse rate, generally indicates a significant problem with the source of the data - it indicates that there is probably bias in the data

nonresponse error - the overall averages increased due to the over-representation of women in the final sample

Causes of nonresponse errors

The two main sources of nonresponse errors are refusals and not-at-home not-at-home nonresponse bias occurs when a respondent is unavailable at the time of potential contact

Improving response rate to reduce nonresponse error

The primary approaches to reducing nonresponse bias due to refusals and not-at-homes are advance notification 预先通知, incentives, callbacks, and recontacts

Reducing nonresponse in personal interviews and telephone surveys

Refusals are a problem in both personal interviews and telephone surveys

Increased cooperation in personal interviews and telephone surveys:

Explicitly motion the interview topic and describe why the topic is important and relevant to the respondent

Describe the purpose and goal of the research, noting the importance of the individual’s response to achieving the goal

making certain that respondents do not feel threatened

Use incentives at levels appropriate to the demand of the survey - gift

Use advance notification

Increase the response rate in mall intercept interviews:

Keep the screener short, concise, and focused only on characteristics of primary importance

Minimize the skip patterns on the screener and make it easy for the recruiter to distinguish qualified from non-qualified respondents

wants to reduce refusal rates, it is important to make certain that interviewers are:

successful trained to establish quick rapport with respondents.

Carefully monitored with regard to their individual refusal rates. - Interviewers who have exceptionally high refusal rates should be immediately retrained or replaced

Increasing the success rate of repeated contact:

Attempt recontact between three and six times

Vary the time of day during which calls are made - use weekday evening and weekend days to contact individuals who work full-time outside the home

Reducing nonresponse in mail surveys

Increase response rate:

Use a cover letter that clearly states the interview topic, describe why the topic is important and relevant to the respondent, Describe the purpose and goal of the research, noting the importance of the individual’s response to achieving the goal

Postage-paid return envelope

Prepare an sprint the questionnaire to maximize visual simplicity and professionalism

Incentives with questionnaire - gift

Reducing nonresponse in online surveys

First, similar to mail surveys, well-written, motivating invitation tends to encourage click-though and participation

Second, Incentives

Third, use of personal salutations in the email － change “Dear student” to “Dear Lisa”

In addition:

Make sure your sieve link is clickable

Reassure people about their privacy and confidentiality

Take care with the “from”, “to”, and “subject”

Keep it short