Introduction

Today it is an essential part of any pre-service and in-service teacher training course to talk about ways of improving students’ reading comprehension, the different stages of a reading lesson, and the reading strategies that students should be taught. Novice and expert teachers are all familiar with (or at least have heard of) techniques such as pre-reading, gist reading, scanning and skimming, etc. Little do they know, however, about the research that has informed practitioners of students’ reading habits and provided them with evidence of the usefulness of certain techniques and strategies. 

Knowing about the findings of latest research is one thing, doing research is quite another. Those of us who have had the chance of carrying out some research in our own school environment know how beneficial it is for everybody involved: the teacher-researcher may recognise the problematic areas in her students’ learning and work out possible solutions, whereas the students themselves may learn from the process of research as well as from the improved techniques of their teacher. This is why teachers should be encouraged to carry out small-scale investigations themselves so that they have a better idea of their students’ learning habits and the problems the students encounter while grappling with the intricate system of the foreign language. 

“Please keep talking” is the sentence that one keeps repeating when using think-aloud as a research method. The aim of this article is to familiarise readers with this approach, which has been successfully used in second language (L2) reading research. Apart from describing the way the method can be used to explore the processes in L2 reading comprehension, I intend to make suggestions as to how the method can be employed by researchers as well as practising teachers who wish to move beyond the everyday routine of teaching. 

Think-aloud as a research method

One of the methods researchers use to get a clearer picture of what learners generally do while reading in a foreign language is think aloud. This is one type of verbal reports, obtained from the readers during reading (Cavalcanti, 1987). Think-aloud means that readers report their thoughts while reading, but they are not expected to analyse their behaviour as in introspection (Cohen, 1987). By means of asking their subjects to say out loud whatever goes through their minds, researchers hope to get a more direct view of the mental processes readers are engaged in while reading (Rankin, 1988).

Verbal reports and think-aloud protocols have been widely used in both L1 and L2 reading research with the aim of tapping the mental processes of readers in different situations. The purpose of the different studies has been to 

• develop a taxonomy of reading strategies (e.g. Anderson, 1991; Olshavsky, 1977), 
• compare first and foreign language reading and find evidence of strategy transfer from the native to the foreign language (e.g. Sarig, 1987), 
• identify the reading strategies of ‘good’ and ‘poor’ readers (e.g. Block, 1986), 
• investigate the effects of prior knowledge on reading comprehension (e.g. Pritchard, 1990), and 
• describe strategies used in taking reading comprehension tests (e.g. Anderson, Bachman, Perkins & Cohen, 1991). 

Reading in a second language is a problem-solving activity per se, because it involves considerable efforts on the reader’s part to make sense of a text written in an unfamiliar code. Thus the cognitive processing required to comprehend a text written in a foreign language can easily become the subject of verbalisation in a think-aloud experiment. This is especially true for reading with a specific purpose, for example, reading for a test. The test questions pose an additional problem that readers need to solve, thus making the activity - reading and thinking in order to find the correct response - suitable for think-aloud. 

Think aloud as a research method has a number of advantages over other types of methods. Olson, Duffy and Mack (1984, p. 256) regard ‘thinking-out-loud’ as a tool for collecting “systematic observations about the thinking that occurs during reading”, that is, for collecting data about the otherwise unseen, unobservable processes, such as inferencing or the use of prior knowledge. They also find the method promising for the study of individual differences, for example, in the level of reading skills or the amount of background knowledge activated. 

In a study (Elekes, 1998), which intended to find out to what extent readers’ background knowledge (or the lack of it) facilitates or hinders comprehension, think-aloud appeared to be a suitable research method. Although the participants had to read and summarise two linguistically quite complex newspaper articles - one on a familiar and the other on an unfamiliar topic, the aim of the research was not to see how well the participants were able to do the task, it was rather intended to tap the underlying thought processes instead. The protocols were expected to shed light on the difficulties L2 readers might encounter when carrying out the reading task, mainly due to the lack of prior knowledge or the inability to activate existing knowledge. The protocols revealed that, contrary to expectations, the presence of certain background knowledge did not only facilitate, but in certain cases hindered comprehension, and that the lack of background knowledge did not always result in failure to understand the text. It was also found that the reading process was largely influenced by the readers’ idiosyncratic knowledge and individual way of applying that knowledge. Without the protocols it would not have been possible to see how individual readers arrived at an interpretation and how they decided what to include in the resulting summary. 

The very strength of the method is that it is the closest possible way to get to the cognitive processes of readers. This appears to be one of the most serious weaknesses, as well, which any researcher or teacher who wishes to embark on using think-aloud protocols in their research should keep in mind. Only the conscious processes are available for verbalisation, that is, much of what is going on in the readers’ mind remains hidden. Thinking also needs to be slowed down to allow for the additional time that is required for verbalising thoughts, which may add to the incompleteness of the reports. Researchers should be aware of this inherent incompleteness of verbal reports and take it into consideration when designing the experiment and analysing data (Ericsson & Simon, 1993). 

Guidelines for design and analysis

There is a general agreement that think-aloud requires careful setting up and preparation on the researcher’s part (e.g. Seliger & Shohamy, 1989). Above all, the purpose of the research should be in harmony with what a think-aloud protocol can reveal. If, for example, research intends to find out about the effect of pre-teaching vocabulary on reading test scores, think-aloud may not be the most reliable measure to be used. On the other hand, if a researcher is interested in the amount of background knowledge individual readers bring to the reading task and how background knowledge is activated in the reading process, think-aloud will most probably provide the required data. 

Instructions

The method is sensitive to several variables: the instructions research participants receive, the text types used in the experiment, the context in which the texts are read and the participants’ ability to verbalise their thoughts. Olson et al. (1984) claim that instructions for any think-aloud task should be focused in relation to the research aims. By giving the subjects general instructions and asking them to say whatever comes into their mind while reading, it can be ensured that they report on various things and do not restrict themselves to specific strategies, recommended by the rubrics. This way the effect of researcher expectancy can be minimised. This is also the only way to carry out the research successfully when the aim is to identify the strategies a particular age-group reading particular texts uses to promote their comprehension. 

Selecting and training participants

Selecting and training the participants for the experiment also needs careful consideration. Participants may vary in their ability to verbalise, and even for those who find it easy to read, think and speak at the same time, think-aloud is an unusual task. Think-aloud experts (e.g. Ericsson & Simon, 1993; Olson et al. 1984) note the importance of training research subjects before the recording session. One way of doing it is to organise a group training session, first of all, to familiarise the participants with the purpose of the study, which is to be kept fairly broad “to avoid biasing the subjects’ responses” (Rankin, 1988, p. 127); then, to show subjects what they are expected to do; and also to ease the subjects’ initial reservations and make them aware of the difficulties everybody else experiences doing a think-aloud for the first time in their lives (Elekes, 1998). A second short practice session can be arranged immediately before the experiment to help subjects to recall the nature of the task as suggested by Rankin (1988). Ericsson and Simon (1993) suggest the use of simple warm-up tasks in which it is comparatively easy to think aloud, for example, a simple arithmetic exercise. 

Age of research population

Another crucial question is the age of the research population. Most studies involving verbal reports had tertiary level students, that is, educated adults as their research participants. Cohen (1986) cites several studies that were conducted with different population (e.g. young readers, women without formal education) and showed that verbal reports potentially had a wider application than had previously been suggested. In several studies (Sarig, 1987; Wijgh, 1995), for example, secondary school students aged 16 to 18 were found articulate enough and capable of verbalising their thought processes after a short training session. 

Language of verbalisation

During the preparation stage, the researcher should decide what language the participants will be expected to use when doing their think-aloud. On the one hand, some complication might arise from the think-aloud procedure if subjects are told to read in L2 and talk in L1: “Requiring subjects to switch back and forth between languages while reading and verbalising would seem to encourage translation ...” (Rankin, 1988, pp. 122-123). On the other hand, there is a danger that participants will worry more about speaking out loud and concentrate less on the reading itself if they are required to verbalise in the foreign language. This is why, in order “to avoid the problem of limited L2 production abilities” (Lee, 1986, p. 204) and to avoid cognitive processes not in the focus of the study, subjects should be instructed to verbalise in their mother tongue. Another alternative is to let the participants decide which language they would feel comfortable with when doing verbalisation. 

Eliciting verbalisation

Eliciting verbalisation is often considered problematic. Researchers need to find out how to elicit verbalising without interfering too much in the reading process. The literature gives a number of solutions as summarised by Rankin (1988), for example, letting the subjects verbalise at will or placing some kind of signals at certain points of the text to remind subjects to talk. The simplest version seems to be the ‘sentence-by-sentence talking’, with the sentences presented one by one to the reader (Olson et al., 1984). The problem with all these techniques is that “by predetermining the junctures at which the subjects must verbalise, those thoughts that occur between the dots may be lost” (Rankin, 1988, p.125). 

Recording

Ericsson and Simon (1993) argue that during the recording session both the recording device and the researcher should be out of sight - this may enhance the reliability of the data gained. When the researchers are present, even if they are invisible to the research participants, they can maximise the amount of verbalisation by prompting the subjects to keep talking whenever a period of silence, longer than expected, occurs. Too much or too direct intervention on the researchers’ part, however, should be avoided (Rankin, 1988). On the other hand, it must be borne in mind that the researcher’s presence may have a negative effect on the content of verbalisation. If the subjects are too much aware of the presence of the researcher, they may try to produce ‘socially acceptable data’ (Cohen, 1987). It seems true that people generally feel less frustrated and less desperate to provide a positive image of themselves when they are alone with a tape recorder than in the presence of an animate listener. This is why, it should be considered an acceptable option to leave subjects on their own, especially, if during the training sessions they appear to provide adequate data without continuous prompting (Elekes, 1998). 

Other sources of information

One of the main reservations about the method is that it may interfere with the reading process, so it is important to find evidence for readers’ behaviour in other ways, as well. To complement the think-aloud protocols, in my background knowledge study (Elekes, 1998) I employed two additional data collection methods: summary writing as evidence of comprehension of the texts; and semi-structured interviews to confirm the findings of the protocols, following Rankin’s (1988) suggestion, “as an additional safeguard, it may be advisable to have subjects do a retrospective analysis of the research passage after the thinking-aloud session” (p. 125). The beneficial effect of such a follow-up on students’ learning should not be overlooked, which will be discussed in more detail below. 

Data analysis

The data obtained through think-aloud is difficult and time-consuming to analyse, and if only a small amount of data is gathered this way, the generalizability of the findings is questionable. Also, protocol data can easily become elusive so the researcher should take special care of recording and keeping the data. It is advisable to make a copy of the tapes as quickly as possible, preferably before doing the transcription. It is also a good idea to do the transcribing shortly after the recording session when everything is still fresh in the researcher’s memory. This way some of the parts otherwise missing due to some background noise, unintelligible speech or too low voice may not get lost. 

In the process of analysing think-aloud protocols Rankin’s advice must be kept in mind, “subjects’ comments should be taken in the context of the situation, and care should be taken not to ascribe meaning to them except at face value” (1988, p.129). The protocols naturally lend themselves to some qualitative analysis, which means that patterns of behaviour are looked for and categorised according to a coding scheme. There are lists of strategies in the literature (e.g. Anderson, 1991) which could be applied, but personal experience suggests that existing lists always require some modification, simplification or expansion as new data emerges from the protocols. Teachers may prefer open-ended data collection when categories are defined on the basis of the recorded material, but as Oxford (1990) warns, such data collection requires skilful interpretation. 

Pedagogical implications

Research participants have every right to know what the researcher has found in the data produced by them. It is especially true in the case of research carried out by the teacher of the participants. In post-recording interviews students report to have become aware of both positive and negative things they do while reading and admit to have learned a lot from the process of doing the think-aloud. This is why I consider it worthwhile to go through the protocol (or tapescript) once again with the students – if time permits – and discuss what they found out about their own reading habits and what the teacher/researcher discovered through the analysis of the data. 

Think-aloud appears to be especially useful when a teacher has students with reading difficulties and decides to find out what exactly causes trouble for them. What else could the teacher do? He or she could observe her students and draw some conclusions on the basis of their answers to reading comprehension questions. This, however, would yield only superficial information. The teacher could also enquire about the students’ weaknesses either in a face-to-face conversation or through a checklist. The problem is that students are rarely capable of realising by themselves what difficulties they have and checklists are not too helpful either as was proven in a study by Allen (1992). A think-aloud session with a follow-up interview seems to be a more reliable and effective option. 

It could also be a tempting idea to compare what poor and good readers do, however, the results may not be easily applicable to the teaching situation. As Cohen (1986) warns “strategies may not be inherently good or bad for a given reader. Rather, they may or may not promote successful comprehension of a text, depending on the particular reader, the particular text, the context in which the reading is going on, and the choice of other strategies in conjunction with the chosen one” (pp. 132-133). His suggestion is that the aim of the research should not be describing the ‘ideal’ reader and prescribing to all students how to read. Instead, research using some mentalistic measure, such as think-aloud, should aim to describe “reading behaviour that promotes or deters comprehension” (p.133). 

Finally, in our test-centred world, it could be another useful aim of the research to see how potential test-takers deal with the typical reading tasks of tests that at present are popular with Hungarian learners. As was suggested above, there is every likelihood that the method is particularly suitable for examining test-taking strategies as readers can only report on what is going on in their conscious mind and test questions require such conscious processing to be solved. The results of such research could help language teachers to design a series of reading sessions as part of their test preparatory course. 

Conclusion

Some 20 years ago Ericsson and Simon noted that verbal reports were considered to “provide possibly interesting but only informal information to be verified by other data” (1980, p. 216). Since then, verbalisation has proved to be a valuable data collection procedure, nevertheless, it is still not used extensively due to the limitations process methodologies are usually claimed to have. 

In the present paper I have described one type of verbal report, the think-aloud technique, and the way it can be used in second language reading research. I have argued that this research method has the potential of becoming part of the language teacher’s repertoire. My experience is that think-aloud is a fascinating, enjoyable and rewarding way of doing research. First of all, it enables you to get a glimpse of how the human brain works. Secondly, by analysing students’ behaviour and needs, you can discover things (important but so far hidden) about them. Finally, a research involving think-aloud invites you to reconsider your own reading habits (especially if you yourself try doing it first), your expectations of the students and the role of reading comprehension in language teaching. 

References

Allen, A. I. C. G. (1992). EFL reading test validation: Investigating aspects of process approaches. Unpublished PhD thesis. University of Lancaster. Anderson, N. J. (1991). Individual differences in strategy use in second language reading and testing. Modern Language Journal, 75, 460-472. 
Anderson, N.J., Bachman, L., Perkins, K., & Cohen, A. (1991). An exploratory study into the construct validity of a reading comprehension test: triangulation of data sources. Language Testing, 8, 41-66. 
Block, E. (1986). The comprehension strategies of second language readers. TESOL Quarterly, 20, 463-494. 
Cavalcanti, M. C. (1987). Investigating FL reading performance through pause protocols. In: C. Færch & G. Kasper (Eds.), Introspection in second language research (pp. 230-250). Clevedon: Multilingual Matters. 
Cohen, A. D. (1986). Mentalistic measures in reading strategy research: Some recent findings. English for Specific Purposes, 5, 131-146. 
Cohen, A. (1987). Using verbal reports in research on language learning. In: C. Færch & G. Kasper (Eds.), Introspection in second language research (pp. 82-95). Clevedon: Multilingual Matters. 
Cohen, A. (1997). Towards enhancing verbal reports as a source of insights on test-taking strategies. In: A. Huhta, V. Kohonen, L. Kurki-Suonio, & S. Luoma (Eds.), Current developments and alternatives in language assessment. Proceedings of LTRC 96 (pp. 339-366). Jyväskylä: University of Jyväskylä. 
Elekes, K. (1998). The role of background knowledge in second language reading comprehension. Unpublished MA dissertation. The University of Reading.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215-251. 
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Verbal reports as data. (Rev. ed.). Cambridge, MA: MIT Press. 
Lee, J. F. (1986). On the use of the recall task to measure L2 reading comprehension. Studies in Second Language Acquisition, 8, 201-212. 
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall. 
Olshavsky, J. E. (1977). Reading as a problem solving: An investigation of strategies. Reading Research Quarterly, 12, 654-674. 
Olson, G., Duffy, S., & Mack, R. (1984). Thinking-out-loud as a method for studying real-time comprehension processes. In: D. E. Kieras & M. A. Just (Eds.), New methods in reading comprehension research (pp. 253-286). 
Hillsdale, NJ: Lawrence Erlbaum. Oxford, R. L. (1990) Language learning strategies: What every teacher should know. Boston, MA: Heinle & Heinle Publishers.
Pritchard, R. (1990). The effects of cultural schemata on reading processing strategies. Reading Research Quarterly, 25, 273-295. 
Rankin, J. M. (1988). Designing thinking-aloud studies in ESL reading. Reading in a Foreign Language, 4, 119-132. 
Sarig, G. (1987). High-level reading in the first and in the foreign language: Some comparative process data. In: J. Devine, P. Carrel, & D. E. Eskey (Eds.), Research in reading in English as a second language (pp. 105-120). Washington D.C.: TESOL. 
Seliger, H.W., & Shohamy, E. (1989). Second language research methods. Oxford: Oxford University Press. Wijgh, I. F. (1995). A communicative test in analysis: Strategies in reading authentic texts. In: A. Cumming & R. Berwick (Eds.), Validation in language testing. (pp. 154-170). Clevedon: Multilingual Matters Ltd.