When analysing test results, teachers tend to focus on the actual test-scores reflecting the acquisition of certain vocabulary or grammar items, and seldom do they look into the strategies their learners employ while taking the tests. However, it must be remembered that performance on written tests can be improved if both language teachers and test designers have a better insight into the different strategies that students apply. While participating in a major educational project in southern Hungary, I became aware of certain tendencies in the way learners approach test tasks. The aim of the study described in this article is to identify and describe the strategies that 7th and 11th grade students use when taking written tests in English. In addition, I also intend to show that looking at the written papers, it is possible to trace some of the thinking processes that test takers employ when solving tasks in writing.

The data presented here were collected as a part of a large-scale project whose aim was to identify the role of school instruction on learners’ performance across all school subjects. The results of the first part of the research, which focused on science subjects, have already been published (Csapó, 1998). That study attempted to identify the influence of school instruction on useful knowledge, which one can utilise in life later. Several different aspects of pupils’ knowledge and ways of thinking were examined. The aim of the second part of the project is to evaluate the effect of school instruction on performance in the humanities, including foreign languages, based on different written tests. I joined the project after piloting the data collection instruments was complete. After analysing the pilot results, I redesigned one of the instruments and compiled new test booklets. After administering the two booklets to a large number students, I conducted an informal investigation of the test-taking strategies, the results of which are presented in this article.

In the following, a brief overview of test-taking strategies is provided, then the method of data collection is described in detail. In the second part of the article, the observed strategies are described and reflected on.

Test-taking strategies

The recent focus on different strategies in education has changed our approach to the teaching and learning process. Learners are no longer perceived to be passive, they actively contribute to language acquisition during the whole process. The identification of various kinds of strategies has led us to realise the importance of learner training and has emphasised the role of the teacher in providing sufficient support in widening the repertoire of possible learner strategies. Some of these strategies such as problem solving, grouping, matching and other operations are common in everyday life. McDonough (1999) points out that the strategies learners apply are sometimes not directly related to language learning but are characteristic features of the human brain. Although, within learner strategies, considerably more attention has been paid to oral communication strategies, test-taking strategies can also be identified.

Bialystok (1990) has distinguished conceptual and linguistic communication strategies. These two categories are related to test-taking by Bachman (1991), who warns not to burden test-takers with tasks that are cognitively demanding. He posits that task type familiarity can facilitate achievement, but results might be misleading if a task can be solved by merely using common sense. Moreover, there may be a mismatch between the tester’s and test-taker’s intentions. As Cohen (1984) points out, it sometimes happens that the test provides information on different language areas to those expected. Finally, the assumption is that a lot depends on test-taking mechanics, that is, if testees cannot identify what is expected of them, the language generated by the tasks will not reflect their level of proficiency. On the other hand, if students have appropriate test-taking strategies, they will be able to achieve improved results by utilising their skills such as grouping, inferencing and evaluating.

As regards training in testing skills, material is available to support preparation for standardised tests in the United States (e.g. Ronald, 1988). Most of these documents are available on the Internet and are written primarily for teachers, but interested parents can understand them easily and thus can help their children.

Calkins and her colleagues (1998) identify some techniques essential for children when taking tests. Their aim is to exclude all disturbing factors and to train students to be at ease on the day of the test, concentrate on the tasks and perform at their best. The authors also present a variety of activities for answering questions and dealing with problem-solving tasks. In his advice to teachers, Scrivener (1994) also emphasises the importance of allocating some classes to training in examination preparation, and he makes a clear distinction between language skills and the necessary test-taking techniques. He also warns teachers to find an appropriate balance and not to be too concerned about test-taking, as it can hinder real learning. The issue of the so-called ‘washback’ effect, that is, the impact of tests on language teaching, has been discussed in detail by Alderson and Wall (1993). The need for training in test-taking strategies is apparent, but teachers should balance the amount of skill-training in their syllabus properly. If they devote too much attention to solving different test tasks, students will not be able to communicate properly outside a testing situation.

Most studies that discuss the issue of strategy use while taking tests were conducted making use of the think-aloud technique. This sheds light on the students’ way of thinking while they are working on different tasks. The researcher gets data directly, and in case of misunderstandings, can ask participants for clarification. For example, Wijgh (1988) reports on research carried out among 13 Dutch school leavers. The test consisted of communicative reading tasks, in which the texts are carefully chosen with the central aim of resembling real-life situations as much as possible. Findings showed a range of strategies used by students during taking the test such as reading the question, reading the whole text and translation.

The analysis of think-aloud protocols on reading comprehension tests conducted by Anderson (1991) resulted in a list of 47 different strategies such as support, paraphrase and strategies for establishing coherence in the text. As the researcher pointed out, not all of them were used, there were considerable individual differences in the type and number of strategies participants applied.

Classroom teachers are usually not in the position, nor may they intend to conduct research based on scientific considerations such as the ones described in the literature. However using informal techniques, such as observation and reflection can also lead to useful findings which can guide teachers when preparing students for taking tests.

Method

The data for this research came from the large-scale project referred to in the introduction (Csapó, 1998). My role in the project was to compile an English language test and administer it to 7th and 11th grade students in southern Hungary. While evaluating the test results, I realised several apparent tendencies in the way learners approached the tasks. This made me think about the strategies students used while taking the tests. I started taking notes of the patterns and tried to relate my assumptions to the available literature. The examination of written products suggests the existence of strategies mentioned in the literature and may reveal some new ones. These assumptions may further be confirmed with think-aloud protocols in a possible later phase of this research.

Participants

The data for this project were collected in two groups of adolescents, that is 7th and 11th grade students (13-14 and 16-17-year olds). The 1221 participants were chosen from nine primary, six vocational secondary and five grammar schools in Szeged. Participants in the research represent different schools in a big town and were chosen according to the principles described in Csapó (1998, pp. 30-35). The distribution of learners is shown in Table 1.

Test booklets

The research instrument was designed and piloted for a large-scale school project (Csapó, 1998) investigating the effects of formal instruction on cognitive processes and learning school subjects, among them English as a foreign language. There were two test booklets used for data collection. Booklet One included a traditional grammar and vocabulary test, resembling those used in schools. This booklet was designed in two parallel versions: A and B. Both versions contained the same task types with different words and prompt sentences. Booklet Two consisted of skills tasks, which I designed using authentic materials to test reading comprehension and writing. The two booklets were designed using the same layout principles. All rubrics were given in English in bold type, and clear examples were provided. In Booklet One the same examples were used both in the A and B versions. Tables 2 and 3 provide a brief summary of the content of the two tests.

Each testee took both tests towards the end of the school year 1997/1998. The timing took account of the general policy of assessing performance in all subjects towards the end of the school year when substantial revision is taking place. Owing to the lengthy nature of the tests (each booklet took 45 minutes to complete), Booklet One and Two were administered on different occasions.

Procedure

After trialling and revising, the test booklets accompanied with detailed instructions were sent to selected schools. Each learner had to take two written tests, each on a different day towards the end of the school year. The tests were administered in regular 45-minute classes by class teachers and were corrected centrally by a research assistant. The recognition type tasks were assessed according to an answer key. The evaluation of the open ended reading comprehension questions needed an expert to make consistent judgements, so I accomplished the task myself using sets of criteria, and another colleague double checked 60 papers to ensure consistent marking. It was during this process that I made several observations regarding test-taking strategies and reflected on them from a teacher’s point of view.

Observations and reflections

While evaluating the two test booklets of each of the participating 1221 students (altogether 2442 test booklets), I observed that the strategies identified greatly resemble the grouping and description used by Oxford (1990), so I will follow her terminology in my analysis.

Arranging and planning

Although the two test booklets were different, students seem to have employed some identical strategies. In general, there was no paper among the 2442, in which the last task or tasks were completed and the first one or first ones were skipped. Obviously, students worked on the paper in a linear fashion and did the tasks one after the other. In other words, nobody started at the end of the booklet moving backwards. A possible assumption is that hardly anybody planned timing or looked through the whole booklet carefully before starting to do the tasks. This may have been counter-productive for some students in the case of the skills booklet, where the last task was letter writing, as there are some students who are better at letter writing than performing other tasks. In fact, several carefully written letters demonstrated good writing ability. A thorough comparison of the booklets showed that if the letter was written, the previous tasks were also completed, whereas the reverse case did not occur. The same phenomenon could also be observed in Booklet One, which contained discrete point items with the tasks evenly spread as far as difficulty and length were concerned. Still, there were fewer tasks attempted at the end of the paper than at the beginning.

Another opportunity for applying the strategy of arranging and planning was offered in the vocabulary matching task. In order to reduce the chance for blind guessing and to make the tasks more difficult, the lists of words to match did not contain an identical number of items. Some, but not all students crossed out the words they had used in order to see more clearly which ones were left. My observation was that solving this task was harder for examinees unfamiliar with this matching task type.

Task avoidance

It was also interesting to observe which were the tasks that students did not complete. Table 4 shows the number of students in the three different groups who did not attempt the letter writing task in the skills test booklet.

The findings suggest that the task type is not really familiar in vocational secondary schools, whereas the ratio of the students who did not attempt the letter-writing task in the secondary grammar schools is relatively low (3.3%).

Reflecting on task avoidance in the grammar-vocabulary booklet leads to interesting observations. It seems that whether students attempted doing the tasks depended on whether the grammatical structure to be used was actually named in the instruction. For example, Tasks 6 and 9 in Booklet One contained two sentence transformation tasks: one involved making the active statements passive, whereas the other required transforming direct sentences into indirect ones. An example in Table 5 shows how the 190 7th grade students performed on Task 6 in Booklet One A, and it also gives the ratio of students who did not attempt the task. Some students left the lines blank, but 63 7th grade primary school students remarked that they had not learned this grammar point. They did not make a similar complaint where the grammar point was not named. Checking their overall performance, they turned out to be low-achievers. They attempted doing the tasks but failed.

Overcoming limitations: Switching to the mother tongue

This strategy can be observed in the letter writing task in which, according to the rubrics, the task was to write a letter of introduction to an English host family before going on an exchange programme to Britain. Students were required to write about their favourite food for example. In some cases the letters contained words such as: “palacsinta” (pancake), “sziámi” cat (Siamese cat) and “spenót’” (spinach) or sentences like: “I like hot dog, hamburger, pizza, sült krumpli, rántot szelet vagy rántot sajtot.” The latter sentence is interesting because the Hungarian words contain spelling mistakes and the word ‘sajtot’ is inflected whereas the other Hungarian items in the list are not.

Compensation: Copying from each other

Interestingly enough, some learners tried to copy answers from each other to compensate for missing knowledge. In the Hungarian educational context this type of cheating is unfortunately quite widespread, and it shows the lack of confidence on the part of the learner. This phenomenon is often difficult to trace in the case of recognition tasks, where the only thing the testee has to do is to circle or underline. In the open-ended reading comprehension task in Booklet Two, it was nearly impossible for two or more students to produce identical responses independently. It was observed that students giving identical answers to the questions attended the same school and the same class. Traces of copying from each other could be found in Booklet One as well. In the first task the synonym of a given word had to be selected from the list provided. Some students might have been mislead by the fact that the examples given in the instructions were identical in Booklets One A and B, and they copied the answers of others. After writing the first ‘synonym’ on the line, some students realised that in their booklet that word was not even on the list, so they crossed it out and probably continued working on their own realising that versions A and B were different after all.

Compensation: Copying the example

Another way of overcoming limitations is copying the example into the gaps. Attempting the gap-filling task on prepositions, students found themselves in trouble if the task was not clear or if they had not yet acquired that area of linguistic knowledge. They simply copied the preposition provided in the example into all the gaps. They were lucky, though, as one out of the five sentences was correct with that preposition. The implication for the test designer is to avoid using the same structure in the example and the item.

The same phenomenon could be observed in the verb transformation task. The example sentence was in the Present Perfect Tense (“I haven’t finished my homework yet.”), and there were seven sentences with ten verbs in base form, which the students had to rewrite in the appropriate tense. Some testees, mostly low-achievers, rewrote all sentences in the tense given in the example, regardless of the content and other language clues. The result was not much better if the actual structure required was stated, for example, the instruction in a task asked for changing the sentences into Reported Speech. Some students used exactly the same tense as in the example, but it was not appropriate in the given sentence.

Compensation: Using ‘safe’ knowledge

In the skills booklet, one of the tasks was reading an authentic extract from a brochure and answering comprehension questions. To help the students, there was a dotted line to limit the length of responses; still, there were very long answers as testees copied whole chunks which contained some of the words that also appeared in the question itself. For example, instead of the required “This is an open top bus.” they put “The easiest way to see Plymouth is on the Discoverer open top bus.” Although natural communication required one-word responses, only few students dared to write short answers. Most of them attempted to produce long responses, which turned out to be grammatically incorrect. This phenomenon can also be explained with language teaching traditions, in which learners are always expected to react to questions in full sentences. In most schools students are taught to answer at length, so whenever a question is asked, the answer must be a complete sentence.

Evidence for using ‘safe’ knowledge can be traced in the letter writing task. Most primary-school students used very short, two- or three-word sentences starting with “I like...”, “I am...”, “My...” or “I have...”. A typical sample of this type of writing is as follows:

“My name is Andras. I’m threeten years old. My father job is truck driver. I go to Zrinyi Ilona elementary school. My mother is shop assistant. My sister is seventen years old. My dog is white and brown color. I like coke and chocolate. I don’t like vegetable. I like apple and banana,”

As the example shows, the student could communicate his ideas, but very simply using the same pattern all over. There are a few grammatical errors as well, which do not hinder understanding. I think teaching students risk-taking even in tests is an important aspect in foreign language teaching.

Conclusion

Good language learners use a range of strategies during the process of learning, among them test-taking strategies, and it would help testees to do well on exams if they received some training in test-taking skills. Teaching learners test-taking strategies could be included in language courses. Although most literature on strategy training does not mention test-taking strategies separately, it is an important issue to deal with, for example in case of written tests, as learners have to give evidence of their language proficiency on their own within a limited period of time, and without any help or immediate feedback on their achievement. Therefore, it seems to be useful to identify those general skills that help learners in a test-taking situation, such as planning, identifying and grouping. Wenden (1998) suggests that teachers should assess individual students’ strategies first and then provide training which enables them to develop their own individual strategies for successful learning.

Acknowledgements

I am gratefuil for Professor Csapó Benõ for providing the opportunity to take part in the data collection project at the Department of Education at the University of Szeged T 030555.

I would also like to thank Krolopp Judit for her efforts in designing the first version of the skills booklet.

References

Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115-129.
Anderson, N. J. (1991). Individual differences in strategy use in second language reading and testing. The Modern Language Journal, 75, 460-473.
Bachman, L. F. (1991). What does language testing have to offer? TESOL Quarterly, 25, 671-704.
Bialystok, E. (1990). Communication strategies: A psychological analysis of second language use. Oxford: Basil Blackwell.
Calkins, L., Montgomery, K., & Santman D. (1998). A teacher’s guide to standardized reading tests: knowledge is power. Oxford: Heineman.
Cohen, A. D. (1984). On taking language tests: What the students report. Language Testing, 1, 70-81.
Csapó, B. (Ed.). (1998). Az iskolai tudás. Budapest: Osiris.
McDonough, S. H. (1999). Learner strategies. Language Teaching, 32, 1-18.
Oxford, R. L. (1990). Language learning strategies: What every teacher should know. Boston, MA: Heinle & Heinle Publishers.
Ronald, B.T.C. (1988). Improving your test-taking skills. Practical Assessment, Research & Evaluation. (http://ericae.net/pare/getvn.asp?v=1&n=2).
Scrivener, J. (1994). Learning teaching. Oxford: Heineman.
Skehan, P. (1991). Individual differences in second language learning. Studies in Second Language Acquisition, 13, 275-298.
Wenden, A. L. (1998). Metacognitive knowledge and language learning. Applied Linguistics, 19, 515-537.
Wijgh, I. F. (1995). A communicative test in analysis: Strategies in reading authentic texts. In: A. Cumming & R. Berwick (Eds.), Validation in language testing (pp. 154-170). Clevedon: Multilingual Matters Ltd.