Research instruments

Research instruments

1. Questionnaires

   • The questionnaire is the favoured tool of many of those engaged in research, and it can often provide a cheap and effective way of collecting data in a structured and manageable form. Questionnaires can be difficult to design and analyze. The questions asked may be misleading or ambiguous ; they may need to be targeted to specific hard to reach groups; and they can generate hours, days or weeks of work in analysis. If managed correctly, they can require fewer resources than many other research instruments, and can help gather views and opinions from many individuals, or "respondents" as they are more commonly referred to.
   • Why use a questionnaire? Why might they be useful?
     • Questionnaires can be designed and used to collect vast quantities of data from a variety of respondents. They have a number of benefits over other forms of data collection: they are usually inexpensive to administer; very little training is needed to develop them; and they can be easily and quickly analysed once completed. An effective questionnaire is one that enables the transmission of useful and accurate information or data from the respondent to the researcher
   • Types of questionnaire
     • There are three broad types of questionnaire
       • The mail survey
         • This instrument is addressed to respondents and delivered by mail, and can be an effective way to collect large amounts of data. However, the mail survey is sometimes considered impersonal and may have a low response rate.
       • The group-administered questionnaire
         • The group-administered questionnaire is a useful instrument for collecting data from a sample of respondents who can naturally be brought together for the purpose. This type of instrument allows each member of the group to complete his or her own questionnaire and return it to the researcher on completion. Response rates using group-administered questionnaires can be higher than those for mail surveys, as the group is often assembled specifically for the purpose of assisting with the research and the respon-dents feel personally involved with the work by being handed the questionnaire by a member of the research team.
       • The household drop-off survey.
         • The household drop-off survey is a hybrid of the mail and the group-administered survey. Using this approach, the researcher delivers the questionnaire by hand to a member of an identified household for collection at some later date. Among the advantages of this approach are that the drop-off and subsequent collection affords the opportunity for those completing the instrument to clarify questions posed with the researcher.
   • Design issues and other considerations when using questionnaires
     • The provision of short and clear instructions to the respondent, at relevant places in the question-naire, can generate more useful and reliable data. If zero is a meaningful answer at some point in your questionnaire, make this clear to those completing.
     • Maximising response rates
       • To maximise response rates consider the use of a covering letter for your questionnaire. This should be short and should explain the research in a clear and understandable way. If data are to be used anonymously, you should state this in your letter, along with information on the accessibility to third parties of your results following completion of the work. A simple and effective covering letter. Other strategies to increase response rate include offering something to the respondent – such as entry to a prize draw or documentation of the findings from the research.
   • Ideal questionnaire length and time to complete
     • As a general rule of thumb, a questionnaire should take no more than about twenty minutes to complete. If a respondent is asked to give up more time than this, he or she may abandon part or all of the questionnaire. Respondents may be expected to complete a lengthy questionnaire if there is some identifiable payback for them – but even here a completion time of no more than thirty minutes should be the aim. If the respondent is particularly interested in the subject, he or she might complete the questionnaire over a period of time, working through it one section at a time. However, this increases the amount of time spent in completing the questionnaire; moreover, the respondent’s train of thought may be broken and may provide responses that seem disjointed.

2. Interviews

   • Define interviews in a research setting
     • They are often likened to a conversation between two people, though a competent researcher knows that he or she are more than this: he or she require orches-trating, directing and controlling to varying degrees. Interviews involve a set of assumptions and understandings about the situation which are not normally associated with a casual conversation. Interviews have long been used in research as a way of obtaining detailed information about a topic or subject. In many situations the use of a research interview rather than, say, a questionnaire can be an indicator of the greater importance attached to the research topic. The research interview is far more resource-intensive. It requires the researcher to elicit information from respondents on a one-to-one basis. Interviews can last for longer than an hour and can produce vast amounts of data.
   • Three interview types
     • The unstructured interview
       • The unstructured interview is a very flexible approach. Areas of interest are established by the researcher but the discussion of issues is guided by the interviewee. This allows some control over the interview for both interviewer and interviewee. However, unstructured interviews can be difficult to plan (in terms of the time to be given to the event), they are difficult to ‘steer’ if the discussion gets away from the key subject matter, and they can prove extremely difficult to analyse.
     • The semi-structured interview
       • There is less flexibility with the semi-structured interview. The interviewer directs the interview more closely. More questions are predetermined than with the unstructured interview, though there is sufficient flexibility to allow the interviewee an opportunity to shape the flow of information.
     • The structured interview
       • Some see the structured interview as no more than a questionnaire that is completed face-to-face. The interviewer has control over the order of questions, all of which are predetermined. There is an element of predictability to the structured interview which allows the event to be timetabled with some precision (an interview scheduled to last an hour will usually do so). Unlike the other models for interviews, the structured interview may provide an easier framework for analysis.
   • Procedures for conducting interviews
     • Pilot your questions
       • No research instrument is perfect. You can begin to identify and correct imperfections (before its too late) by piloting, or testing out, your questions with a select few people in order to establish their clarity. Many researchers carefully craft their interview questions, structuring the instrument around key themes or subject areas – but they fail to pilot the questions they intend to use. Piloting is crucial. It assists in eliminating ambiguous questions as well as in generating useful feedback on the structure and flow of your intended interview. As with other research instruments, interview questions should be easy to understand.
     • Select your sample interviewees
       • The sample of interviewees must be representative - if generalizations are to be made from the data they provide - and sensible. The central research question(s) should help you decide how many interviewees you will need and whom you should interview. If you conduct more than one interview in the same organization, it is acceptable for these to be conducted "top-down," i.e., interviews with department or section heads usually precede those with employees who report to them.
   • Examining interview analysis techniques
     • Using this approach allows themes, issues and concerns to be easily identified and quantified. The final stage of the interview process begins by drawing together the data collected and structuring them in such a way as to make ready for analysis. In small-scale work, this would typically involve grouping the responses to each question from all interviewees to make comparison between respondents easy (the cut-and-paste function of most wordprocessors is ideal for this). When analysing a large number of interview transcripts it may be necessary toutilise the functions of computer-based tools. However, they rely on a coding structure that has to be developed by the researcher before any meaningful analysis can take place. We conclude this discussion by outlining the advantages and disadvantages of using interviews, followed by a checklist for planning and conducting interviews

3. Content analysis

   • Define content analysis
     • Content analysis is an extremely broad area of research – its coverage includes both quantitative and qualitative approaches to analysis. As a research technique, content analysis has been used in a variety of ways and within a number of contexts. It has been successfully used to analyse text and solve issues of disputed authorship of academic papers: the techniques used have included an examination of prior writings and a frequency count of nouns or commonly occurring words to help determine the probability of authorship. Content analysis can be used as a powerful research tool to determine, from the content of a message, sound inferences concerning the attitude of the speaker or writer. Often the message is delivered as a text, or converted to one. Other examples of texts suitable for content analysis include essays, journal articles, books and chapters in books, discussions, newspaper articlles or stories, speeches, conversations and advertisements.
   • Conceptual analysis
     • Of the two main approaches to content analysis, conceptual analysis is by far the more popular. This approach examines either the incidence or the frequency of concepts (themes/issues, words, phrases, etc.) in a text. It quantifies the occurrences of the concepts you have chosen for examination. Conceptual analysis is also referred to as thematic analysis because it is the themes or issues in the text that the researcher intends to analyse. At this level, conceptual analysis is relatively straightforward and simple. However, the themes for conceptual analysis may be implicit rather than explicitly stated, like those in the example discussed above. Often, content analyses are conducted on a number of text pieces in order to explore the coverage of particular themes. Before textual data are analysed, a decision has to be made concerning the unit(s)/the level of analysis. In some cases, and in an attempt to provide more useful context, sentences or strings of words are used as levels.
   • Analysis models
     • In essence, the stages of the content analysis process, just outlined, focus on the number of times a word, theme or issue emerges, and meaning is applied to that numerical data. Qualitative researchers argue that because some form of quantification occurs, the real meanings of the data are overlooked in favour of frequencies. Qualitative research is concerned with capturing the richness, and describing the unique complexities, of data. It is often the case that themes and issues can be established quickly using a qualitative approach, rather than the more resource-intensive and time consuming quantitative approach. In the works defending the usefulness of qualitative content analysis, it has been argued that quantitative content analysis should be used only when three factors are present: i.e., when: 1 You are interested in very precise results. 2 There is the possibility of the collected data being biased. 3 The data collected will be statistically related to numerical data.

4. Focus groups

   • Focus-group research is a form of qualitative method used to gather rich, descriptive data in a smallgroup format from participants who have agreed to ‘focus’ on a topic of mutual interest. Focus-group interviews have enjoyed consistent popularity over many years as an effective and economical instrument of data collection. The information they produce has been used to develop better products and to encourage consumers to use or buy them.
   • What is distinctive about focus groups?
     • Focus-group interviews are organised along altogether different lines. Not only do they permit exactly the kind of interaction which would be inappropriate in other interview situations, but they positively encourage it. It is generally agreed that individuals have their own thoughts, feelings or opinions about a certain issue which are either ‘brought to the table’ or formed and developed during a discussion. The resulting combined group effort may produce a wider range of information, insight and ideas than that likely to be revealed by any single member of the group in a one-to-one interview. Stimulation is a situation in which the group setting works to spur members on to express their own ideas.
   • Procedure
     • There are many things to consider in the planning stages of your focus-group interviews. These include: who should participate in your focus groups; how you should approach them; what kinds of questions you should ask, and how many; how the focus groups should be conducted; where to hold the focus groups; and how to collect and analyse your data.
       • Establishing your focus group The group is made up of participants. It need not contain a fixed number of participants (though fewer than four may jeopardise the valuable group dynamic you seek, and more than twelve may make the group unwieldy).
       • Developing your questions Your participants should feel he or she are taking part in a free-flowing discussion, exploring a variety of related issues and covering a range of topics on their own terms and in their own time, rather than being exposed to a series of probing questions. Nevertheless, you will have to take the time to construct your questions carefully, so that they can be embedded neatly, and non-threateningly, in your group discussion when the time comes. You should need far fewer questions than you would develop for one-to-one interviews as in a focus-group situation each question can lead to a substantial amount of discussion and debate.
       • Conducting your focus group It is likely that you, alone or together with one or more friends, colleagues or coresearchers, will act as moderator during the focus-group meeting. It is the responsibility of the moderator to ensure the meeting is a successful one – by organising a seating plan for the participants in as comfortable and convivial a setting as possible, welcoming them to the discussion and putting them at their ease, allowing each participant the time to introduce himself or herself to the rest of the group and to make comments when they wish, ensuring that all participants can make eye contact with one another as well as hear each other, arranging for the discussion to be recorded (on audio-tape or mini-disk, or even video), outlining the purpose of the meeting to the group, and so on.
       • Analysing your data There is no avoiding the first task involved in analysing your data – transcribing your tapes in order to produce an accurate record of everything which was said in each of your focus-group interviews. This may appear a daunting and unappealing chore. After all, you may be conducting a whole series of discussions, each lasting an hour or more. In order to make the prospect slightly more palatable, we recommend that you transcribe each recording as soon as possible after each meeting, when participants’ comments are still fresh in your mind.

5. Observation

   • Merely asking about or reporting the activities people carry out in different social settings and situations will no doubt give you a flavour of what is involved, but in order to understand fully what these activities mean to people, how they themselves perceive them and what their perspective is on them, it is necessary to see those people in action, to experience what it is they do, even to wade in and have a go yourself. This is the research method called observation, and one of the clearest basic definitions we have found of observation as a research method tells us that it is ‘research characterised by a prolonged period of intense social interaction between the researcher and the subjects, in the milieu of the latter, during which time data, in the form of field notes, are unobtrusively and systematically collected
   • Why choose observation?
     • Observation is an extremely handy tool for researchers in this regard. It can allow researchers to understand much more about what goes on in complex real-world situations than they can ever discover simply by asking questions of those who experience them (no matter how probing the questions may be), and by looking only at what is said about them in questionnaires and interviews.
   • Planning and conducting your observation
     • Before embarking on the planning stages of your observation you will find it helpful to be reasonably confident of two things: the focus of your study, i.e. the topic of your research, what it is you wish to explore and learn more about; and the research questions you intend to address. Of course, both your research focus and your research questions may change over the course of your inquiries in light of your experiences. Although they may well be provisional in the beginning they are still vital.
   • Observation is not a research method which suits everyone. It can consume a great deal of a researcher’s time and energy, and the reward for investing all that effort can be a long time coming. Yet observation can be one of the most professionally gratifying experiences a researcher can have. To see what is actually going on in the social setting you are researching – which may not at all be what you are accustomed to seeing or what you expected or presumed you would see is both the challenge and the reward of observational research. With careful planning and a little practice there is no reason why you should not be up to that challenge.