Wednesday Seminar: With bilingual eyes. Throwing away the “ideal” participant (or: is my data collection reliable?)
Hanna Solberg Andresen, a doctoral candidate at MultiLing, will give a talk on methodological issues regarding participant selection.
In my PhD I work on the conceptualization of motion events in English-Norwegian bilingual children (age 7-8). A set of 60 video-clips is used to elicit verbal data, eye-tracking data and memory data. In this presentation I will discuss aspects of the data collection, especially those related to the question of reliability and replicability. As Mertins (2016:19) points out when it comes to the comparability of the experimental settings, “it is very important to keep the experimental protocol across individual recordings as consistent as possible and to minimize variations in the experimental procedure, incl. the instruction and interaction between participants and investigator” (my emphasis).
But working with children is different from collecting data from adults, and some children need more encouragement than others to endure the whole session. How much variation in the instruction/interaction can be tolerated? Recruiting the right participants is a long process, and hence the participants are very valuable (“not an endless pool of participants”, chf. Mertins 2016:24). What do you do in cases where you will be loosing data because the child “falls out” (loss of eye gaze/verbal data)? To what degree does extra promting influence the results (in my case the eye-tracking data (attention to endpoints) and linguistic data (mentioning of endpoints))? And what if the alternative is breakdown?
I will show some examples of the “ideal” vs. the “close to breakdown”-participant and discuss whether the latter will have to be excluded or if it’s possible somehow to keep him/her (or do we only want “ideal” participants?) – alternatively how to decide on the flexibility vs consistency of the experimental protocol and how to avoid breakdowns. I welcome a discussion on how to find a solution to this challenge (please feel free to bring in your own experience and questions!).
Cohen Louis, Lawrence Manion and Keith Morrison 2011: “Validity and reliability” (Chapter 10, p. 179-215) Research methods in education, Routledge.
Mertins, Barbara 2016: “The use of experimental methods in linguistic research: Avantages, problems and possible pitfalls” in Anstatt, T., Clasmeier, C & A. Gattnar (eds.): Slavic languages in Psycholinguistics. Chances and Challenges for Empirical and Experimental Research. Tübingen: Narr Francke Attempto, 15-33.