Field Methods recent issues

Q Method and Surveys: Three Ways to Combine Q and R

Danielson, S. — Tue, 21 Jul 2009 22:21:51 PDT

This article describes three techniques for combining a Q method analysis with a traditional survey. Q method is an effective way of identifying the various shared perspectives that people may hold about a given topic. The ability to measure those viewpoints in a later survey would allow the researcher to generalize the findings to a larger population and to construct explanations in which respondents' adherence to those perspectives is related to other variables. "Scale creation" involves presenting distinctive Q statements as Likert items that can be used to construct a traditional psychometric scale to measure each perspective. "Profile correlation" involves correlating respondents' overall pattern of responses with prototypes representing the perspectives. Finally, "narrative evaluation" asks the respondent to give his or her opinion about a narrative description of each perspective.

Q Technique, Method, and Methodology: Comments on Stentor Danielson's Article

Brown, S. R. — Tue, 21 Jul 2009 22:21:51 PDT

Recognizing Common Ground: A Reply to Steven R. Brown

Danielson, S. — Tue, 21 Jul 2009 22:21:51 PDT

Coding Issues in Modality Analysis

Popping, R., Roberts, C. W. — Tue, 21 Jul 2009 22:21:51 PDT

Ambiguous phrases are the bane of researchers' attempts to ensure acceptably high interrater agreement in the encoding of texts. When modal usages and their associated rationales are encoded as part of a text analysis, ambiguities arise in characteristic (and thus identifiable) ways. This article illustrates the typical sources of disagreement among coders involved in encoding data during a modality analysis and provides concrete strategies for improving interrater agreement.

Not Quite Crisp, Not Yet Fuzzy? Assessing the Potentials and Pitfalls of Multi-value QCA

Vink, M. P., Van Vliet, O. — Tue, 21 Jul 2009 22:21:51 PDT

This article assesses the strengths and shortcomings of multi-value qualitative comparative analysis (mvQCA), a comparative technique for small- to medium-sized data sets that has been integrated in the TOSMANA software developed by Lasse Cronqvist. The main difference with "crisp-set" QCA is that in mvQCA, the conditions can have more values than just the Boolean values 0 and 1, whereas the main difference with "fuzzy-set" QCA is that mvQCA conditions remain discrete. The major advantage of nondichotomous categorization, according to its proponents, is that it reduces the likelihood of contradictory configurations because of a more homogeneous grouping of cases. We give an overview of existing mvQCA applications, with a detailed discussion of two recent publications, and argue that crisp-set and fuzzy-set alternatives should be less easily discarded, as the mvQCA solution comes with substantial set-theoretical costs.

Participatory Photography as a Qualitative Approach to Obtain Insights into Farmer Groups

Gotschi, E., Delve, R., Freyer, B. — Tue, 21 Jul 2009 22:21:51 PDT

Participatory methods and visual tools are increasingly popular as qualitative approaches for enriching and complementing quantitative survey tools for understanding livelihoods and assessing and documenting impacts of development projects. However, the use of visual tools for analyzing and documenting social processes is still in its infancy. This article reports on an innovative attempt in adopting existing methods of using disposable cameras to stimulate interviews and focus group discussions with groups of smallholder farmers and to obtain new insights into farmer group dynamics and social capital of groups.

Demonstrating the Merits of the Peer Research Process: A Northern Ireland Case Study

Burns, S., Schubotz, D. — Tue, 21 Jul 2009 22:21:51 PDT

This article discusses the benefits and challenges of involving peer researchers in social research projects. A research project on pupil participation in policy making on school bullying in Northern Ireland's schools was commissioned by the Office of the Northern Ireland Commissioner for Children and Young People and undertaken by the National Children's Bureau in conjunction with researchers from Queen's University Belfast in fourteen schools across Northern Ireland, utilizing a mixed methods approach. We trained and employed nine 15—18-year-old peer researchers to support them in this project. After the project's completion, we conducted interviews with six of the peer researchers to investigate how they experienced their involvement in the research. We discuss the findings from these interviews and contextualize in a review of literature on research involving children and young people.

Respondent Debriefings Conducted by Experts: A Technique for Questionnaire Evaluation

Nichols, E., Hunter Childs, J. — Thu, 04 Jun 2009 03:31:12 PDT

This article explores the use of expert respondent debriefings to evaluate the quality of survey data. In this case study, subject matter experts observed 169 interviews and conducted qualitative respondent debriefings on select cases in a field test of a census coverage survey. By comparing "true" residence status for each person as determined by the debriefing against the residence status obtained by the questionnaire alone, the authors determined whether the questionnaire was collecting accurate information. For the 473 people for whom survey data was available, the questionnaire failed only five times to assign the correct residence status code in a way that would have detrimentally affected coverage estimates. The respondent debriefing technique helped pinpoint specific problems in the questionnaire, as well as confirm that the questionnaire performed adequately in most cases. This article describes the expert respondent debriefing methodology that was used in a face-to-face interview setting and discusses how it could be adapted to telephone interviewing.

Participant Observation and Phased Assertion as Research Strategies in the Canadian Arctic

Collings, P. — Thu, 04 Jun 2009 03:31:12 PDT

Participant observation is the basic and defining research strategy for cultural anthropologists, a useful tool for building rapport, establishing trust, and gaining an understanding of culture as experienced by its members. This article uses the author's experience working in an Inuit community in Canada to explore another use of participant observation: the acquisition of communicative competence. In small, bounded communities such as those in the Canadian Arctic, the development and display of cultural and communicative competence is necessary to overcome apathy and sometimes hostility toward researchers. Furthermore, establishment of these abilities allows for the use of phased assertion as an interview probe. Phased assertion works not only as a data collection technique, it reinforces communicative competence and improves informant rapport.

From Interviews to Social Network Analysis: An Approach for Revealing Social Networks Embedded in Narrative Data

Mckether, W. L., Gluesing, J. C., Riopelle, K. — Thu, 04 Jun 2009 03:31:12 PDT

This article describes the process of discovery used to convert interview data into a format readable into MultiNet for social network analysis. Based on the 2005 doctoral dissertation research of Willie McKether, the authors describe the steps used to collect and store interview data in Microsoft Word, the preparation process for exporting the interviews to ATLAS.ti for coding, the coding process, and the conversion path that allowed them to export the coded qualitative data from ATLAS.ti to SPSS and ultimately to MultiNet for social network analysis. This study is significant because it describes a replicable conversion technique that can be used by experienced scholars and those unfamiliar with the range of network analysis programs and conversion possibilities.

The Field Site as a Network: A Strategy for Locating Ethnographic Research

Burrell, J. — Thu, 04 Jun 2009 03:31:12 PDT

Through the work of constructing a field site, researchers define the objects and subjects of their research. This article explores a variety of strategies devised by researchers to map social research onto spatial terrain. Virtual networked field sites are among the recent approaches that are challenging conventional thinking about field-based research. The benefits and consequences of one particular configuration, the field site as a network that incorporates physical, virtual, and imagined spaces, will be explored in detail through a case study. The author focuses in particular on the logistical issues involved and practical steps to constructing such a field site. This article includes suggestions for ways of studying social phenomena that take place on a vast terrain from a stationary position.

Design of Web Questionnaires: The Effects of the Number of Items per Screen

Toepoel, V., Das, M., Van Soest, A. — Thu, 04 Jun 2009 03:31:12 PDT

This article analyzes the effects of an experimental manipulation of the number of items per screen in a Web survey with forty questions aimed at measuring arousal. The authors consider effects on survey answers, item nonresponse, interview length, and the respondents' evaluation of several aspects of the survey (such as layout). Four different formats are used, with one, four, ten, and forty items and headers on a screen. The authors find no effect of format on the arousal index, but nonresponse increases with the number of items appearing on a single screen. Having multiple items on a screen shortens the duration of the interview but negatively influences the respondent's evaluation of the questionnaire layout. Grouping effects are generally similar for different demographic groups, though there are some differences in magnitude and significance level.

"They'll Change What They're Doing If They Know that You're Watching": Measuring Reactivity in Health Behavior Because of an Observer's Presence-- A Case from the Peruvian Amazon

Harvey, S. A., Olortegui, M. P., Leontsini, E., Winch, P. J. — Sat, 24 Jan 2009 18:41:49 PST

Social scientists often employ direct observation to study human behavior, but a health researcher who proposes it may face considerable skepticism from colleagues. Concerns about reactivity lead many to question the validity of observational data. Because few studies have measured reactivity, evidence to evaluate this concern is limited. The authors report results from their systematic measurement of reactivity during a Peruvian malaria prevention study. In sixty observations over nine months, observers recorded all behavior they perceived as potentially reactive. The authors then assessed reactivity using iterative coding and analysis. Although they documented 339 reactivity episodes, only two involved behaviors related to study objectives. These findings are consistent with prior research and provide additional evidence that reactivity, though common, need not bias study results. The authors suggest strategies for assessing reactivity that can help reassure skeptics and reinforce the validity of observational data.

Using Nonmedically Trained Interviewers to Collect Biomeasures in a National In-home Survey

Jaszczak, A., Lundeen, K., Smith, S. — Sat, 24 Jan 2009 18:41:49 PST

A key operational component of the National Social Life, Health, and Aging Project (NSHAP) is the use of nonmedically trained interviewers to collect biomeasures in a national in-home survey of health and aging. Few studies have integrated in-home biomeasure collection using nonmedically trained interviewers on a large scale. In this article, the authors discuss their approach to using nonmedically trained interviewers to collect biomeasures in the home. The article focuses on activities that impact the ability to integrate biomeasures into survey research, including developing field methods, recruiting and training interviewers, and monitoring data collection activities. In addition, cooperation rates and measures of interviewer productivity and data quality are provided to evaluate the authors' approach.

Estimating Household Water Use: A Comparison of Diary, Prompted Recall, and Free Recall Methods

Wutich, A. — Sat, 24 Jan 2009 18:41:49 PST

Studies of household water use are often based on retrospective behavioral reports, which are vulnerable to threats to informant accuracy. This article compares three methods for collecting household water use data: a diary, prompted recall, and free recall. The analyses are based on data from seventy-two randomly selected households in Cochabamba, Bolivia. Per capita water use estimates based on the three methods were significantly different. When compared against known parameters, the diary provided the most accurate estimate of household water use. The prompted recall method yielded similar results to the diary method on hygienic and food preparation tasks, but there were significant differences on household cleaning tasks. The free recall method significantly underestimated water use. There were not significant differences in water use assessments within households, but there was variation in misestimation across households. The article discusses which participants (e.g., heads of less water-consumptive households) provided more reliable water use estimates.

Nonresponse in the Dutch Time Use Survey: Strategies for Response Enhancement and Bias Reduction

Ingen, E. V., Stoop, I., Breedveld, K. — Sat, 24 Jan 2009 18:41:49 PST

This article describes attempts to enhance response rates in the Dutch Time Use Survey (DTUS). The authors examine what kind of nonresponse bias exists in the DTUS—with special concern for "busyness"—and they analyze whether enhanced response rates result in lower nonresponse bias. Results partly confirm the outcomes of earlier studies. First, time pressure does not seem to stop people from cooperating in the DTUS. Second, people who are participating actively in other domains (work, sports, volunteer work) and people who are more involved (political interest, reading newspapers) tend to cooperate more often in the survey than others. Surprisingly, the authors conclude that an increase in response rate does not necessarily lead to less selectiveness, which is a counterintuitive and worrisome finding. Additional field efforts resulted in more of the same kinds of people cooperating.

Excel as a Qualitative Data Analysis Tool

Meyer, D. Z., Avery, L. M. — Sat, 24 Jan 2009 18:41:49 PST

Qualitative research seeks to examine the interconnections in rich, complex data sources. The statistical tools of quantitative methods separate out pieces of data in a manner that defeats the purpose. But, like quantitative researchers, qualitative researchers often still find themselves overwhelmed by the amount of data and equally in need of tools to extend their human senses. This has led the development of a number of software packages designed for this purpose. An often overlooked option, however, is Microsoft Excel. Excel is generally considered a number cruncher. However, its structure and data manipulation and display features can be utilized for qualitative analysis. In this article, the authors discuss data preparation, analysis, and presentation, including discussion of lesser known features of Excel.

When There Is More than One Answer Key: Cultural Theories of Postpartum Hemorrhage in Matlab, Bangladesh

Hruschka, D. J., Sibley, L. M., Kalim, N., Edmonds, J. K. — Tue, 11 Nov 2008 13:44:45 PST

Individuals can acquire cultural knowledge from many sources, including personal experience, informal learning, and schooling. Identifying these distinct source models and describing personal variation in their use present ongoing theoretical and methodological challenges. Three questions are of particular importance: (1) how to determine if there is more than one cultural model, (2) how to characterize the differences between models, and (3) how to assess the degree to which individuals draw from these different models. This article addresses these questions by analyzing the theories endorsed by women and their maternal care providers about the causes, signs, and treatments of postpartum hemorrhage in rural Bangladesh. Two cultural models are identified, each associated with traditional birth attendants or professionally trained "skilled" birth attendants. More broadly, the article discusses the statistical issues involved in determining the existence of multiple cultural models in a population.

Reflecting on the Experience Sampling Method in the Qualitative Research Context: Focus on Knowledge Production and Power during the Data-Collection Process

Koro-Ljungberg, M., Bussing, R., Williamson, P., M'Cormack-Hale, F. — Tue, 11 Nov 2008 13:44:45 PST

In this conceptual article, we discuss how the conventional experience sampling method (ESM) was applied to a qualitative research project to increase participants' agency and empowerment during data collection. Specifically we outline the conceptual and methodological tensions, complexities, and power shifts that emerged during our data-collection process. Research examples illustrate the location of knowers (researchers and study participants), the knowers' relationship to various tangible objects of research (e.g., protocols, digital devices), and how these notions shape power and data that is constructed within a research study. We conclude that it is important to analyze and reflect on how researchers conduct themselves during the data-collection process, and how research can be adapted to reflect the needs and social circumstances of the participants. Our experiences suggest that greater awareness of power shifts during the data-collection process will help researchers better understand the nuances and situated meanings within their data.

Real or Virtual? Response Behavior in Video-Enhanced Self-Administered Computer Interviews

Gerich, J. — Tue, 11 Nov 2008 13:44:45 PST

This article compares response behavior in video-enhanced self-administered questionnaires with three standard methods: paper-and-pencil, face-to-face, and audio-enhanced self-administered questionnaires. Although its scope is limited, and only small mode differences were observed, preliminary results suggest that video-enhanced interviews may produce more valid responses in that they are less contaminated by social desirability. Knowledge of video elements integrated in self-administered questionnaires has so far been limited. This study attempts to build a tentative theoretical framework for the interpretation of the mode differences established in the course of this research.

The Use of Client-side Paradata in Analyzing the Effects of Visual Layout on Changing Responses in Web Surveys

Stern, M. J. — Tue, 11 Nov 2008 13:44:45 PST

Recent research has shown that even minor changes in the visual layout of survey questions can affect the way in which respondents answer. Still, little is known about how these changes affect the actual answering process. The recent development of client-side paradata allows us to better understand the way respondents construct their answers by providing new types of data, including time taken to answer a question and whether respondents change their answers. Although a number of studies have used this technology to examine response latency, few have taken advantage of its potential to collect data on the types of changes respondents make while answering survey questions. This study uses client-side paradata to analyze changes in multiple versions of three types of Web survey questions. The results suggest that the visual layout of survey questions not only affects the number but also the types of changes respondents make.

On the Validity of Cross-Cultural Social Studies Using Student Samples

Flere, S., Lavric, M. — Tue, 11 Nov 2008 13:44:45 PST

Student samples have become a widely used resource in the study of not only particular phenomena and problems within individual environments but also of their study within a cross-cultural context. A number of such studies, most often generalizing their results to the level of cultures, are illustrated in this article. In addition, the authors carry out a number of empirical tests of the generalizability of results attained on student samples in cross-cultural research. Based on the World Values Survey data, mean values of four sociologically and psychologically relevant measures are compared between national and student samples of twenty-three countries. Results suggest that the findings attained on student samples can be viewed cautiously as a good indicator of national sample rankings in cross-national comparisons.

Book Review: Method and Meaning in Polls and Surveys, by Howard Schuman. Cambridge, MA: Harvard University Press. 2008. 230 pp

Smith, T. W. — Tue, 11 Nov 2008 13:44:45 PST