THE ASSESSMENT OF READING COMPREHENSION THROUGH MULTIPLE-CHOICE CLOZE TEST OF THE EIGHTH GRADE STUDENTS OF SMPN 2 SUSUT IN ACADEMIC YEAR 2019/2020

THE ASSESSMENT OF READING COMPREHENSION THROUGH MULTIPLE-CHOICE CLOZE TEST

OF THE EIGHTH GRADE STUDENTS OF SMPN 2 SUSUT

IN ACADEMIC YEAR 2019/2020

I Dewa Ayu Widarini

SMPN 2 Susut

Desa Abuan, Kecamatan Susut, Kabupaten Bangli, Provinsi Bali

e-mail: dwayuwidarini@gmail.com

ABSTRACT

Reading is an essential competence that should be acquired by the students. Considering that reading is a receptive skill that can only be assessed through measurement of comprehension, it is essential to choose the right test to assess students’ reading comprehension. The undertaking of this present study that made use of ex post facto research was to figure out the achievement of reading comprehension of the eighth grade students of SMPN 2 Susut in academic year 2019/2020 which was assessed through multiple-choice cloze test. The data needed for this present study were acquired by administering research instrument in the form of multiple-choice cloze test, which previously had been tried out in order to make the test reliable and valid, to the samples of the eighth grade students of SMPN 2 Susut. There were 32 samples that had been chosen previously by using random sampling with lottery system. The data of this present study which were analyzed by using norm reference measurement with five standard values clearly indicated that out of 32 samples: 4 samples attained excellent achievement; 6 samples, good achievement; 10 samples, sufficient achievement; 9 samples, insufficient achievement; and 3 samples, poor achievement. Hence, there were 62.50% samples could pass the assessment of reading comprehension through multiple-choice cloze test while 37.50% samples failed. To summarize, the eighth grade students of SMPN 2 Susut in academic year 2019/2020 attained sufficient achievement in reading comprehension.

Keywords: assessment, reading comprehension, and multiple-choice cloze test.

ABSTRAK

Membaca merupakan sebuah kompetensi penting yang harus dikuasai oleh siswa. Mengingat bahwa membaca adalah kecakapan yang bersifat reseptif yang hanya dapat dinilai melalui tes terkait pemahaman siswa terhadap teks yang dibaca, maka sangat penting untuk menentukan tes yang tepat untuk mengukur pemahaman membaca siswa tersebut. Pelaksanaan penelitian ex post facto ini bertujuan untuk mengetahui pencapaian pemahaman membaca siswa kelas delapan SMPN 2 Susut tahun pelajaran 2019/2020 yang dinilai melalui multiple-choice cloze test. Data yang diperlukan dalam penelitian ini diperoleh dengan cara memberikan instrumen penelitian dalam betuk multiple-choice cloze test kepada sampel dari siswa kelas delapan SMPN 2 Susut tahun pelajaran 2019/2020, dimana instrumen penelitian tersebut telah diujicobakan sebelumnya dengan tujuan agar hasil dari tes yang digunakan dapat dipercaya dan benar. Dalam penelitian ini, terdapat 32 sampel yang sebelumnya telah dipilih secara acak dengan menggunakan undian. Data dari penelitian ini, yang mana dianalisa dengan menggunakan lima standar pencapaian, secara jelas menunjukan bahwa dari 32 sampel: 4 sampel memperoleh pencapaian yang sangat baik; 6 sampel, pencapaiannya baik; 10 sampel, pencapaiannya cukup baik; 9 sampel, pencapaiannya kurang baik; dan 3 sampel, pencapaiannya tidak baik. Sehingga, 62.50% sampel mampu menyelesaikan dan mengerjakan multiple-choice cloze test dengan baik, sedangkan 37.50% sampel gagal dalam mengerjakan tes tersebut. Jadi, dapat disimpulkan bahwa siswa kelas delapan SMPN 2 Susut tahun pelajaran 2019/2020 memiliki pencapaian yang cukup baik dalam pemahaman membaca.

Kata kunci: penilaian, pemahaman membaca, dan multiple-choice cloze test.

INTRODUCTION

Reading, as one of four language skills, is an essential competence that should be acquired. It is a great source of language learning since language can be acquired from reading as the contrary of studying the language directly. Urquhart and Weir in Hedgcock and Ferris (2018:15) state that reading means “dealing with language messages in written or printed form”. From that definition we know that reading is the process results from a negotiation of meaning between the text and its reader as the reader must be able to translate the written words into a meaningful language.

Reading involves students interacting with visual input of language, which they need to process and understand (Pollard, 2008:44). The reader interacts with a text, predicts what will come next and brings his or her knowledge of the subject and language to the text. Therefore, reading is nothing without comprehension as comprehension is the act of understanding what we are reading. Comprehending a text is important because without it reading does not provide the reader with any information; it is nothing more than tracking symbols on a page with your eyes and sounding them out. Readers use many different text comprehension strategies to develop reading comprehension. These strategies include monitoring for understanding, answering and generating questions, summarizing and being aware of and using a text’s structure to aid comprehension.

Teaching process is closely interrelated with the assessment in order to give a feedback and reflection about the effectiveness of teaching-learning process. One of the assessment types that can be used to assess the students’ understanding of the materials given by the teachers is test. Brown (2019:4) states that tests are a subset of assessment that can be useful devices to assess the students’ ability. A good test will help to locate precise areas of difficulty encountered by the students. English teachers, especially in SMPN 2 Susut, however, usually use traditional technique in assessing students’ reading comprehension that is conventional multiple-choice test. The researcher believes that this assessment cannot truly reflect the extent of students in understanding and comprehending the passage as it does not cover all of the specifications in assessing reading comprehension.

Knowing the fact that reading is a receptive skill that we can only assess through measurement of comprehension, it is essential to choose the right test to assess students’ reading comprehension. The test that is suitable to be used in assessing students’ comprehension and can truly reflect their extent in understanding texts is cloze test, in which the test takers are presented text that some of the words in the text have been left out and replaced with blanks. The students will use their background knowledge, vocabulary, grammatical knowledge, and other strategies to help them understand the text. Hence, the students will not be able to fill the blank spaces with the correct words if they do not understand the text first and vice versa.

According to the statements above, reading is very important skill that can give many advantages to the students; moreover, through administering test, teachers will know the students’ achievement in comprehending texts. Therefore, the teacher can evaluate the strength and weakness of teaching-learning process as it reflects teacher’s successfulness in teaching the students. To sum up, the researcher is favorably motivated to do a research entitled “The Assessment of Reading Comprehension through Multiple-Choice Cloze Test of the Eighth Grade Students of SMPN 2 Susut in Academic Year 2019/2020”.

Research Problem

A research is conducted because a significant problem happens and it should be solved straight away. Selecting, analyzing, and focusing the research on specific problem is essential since the researcher will be able to identify the answer of the research problem accurately. Thus, in accordance with the background of the study mentioned above, the research problem of this present study is formulated as follows: to what extent is the achievement of reading comprehension of the eighth grade students of SMPN 2 Susut in academic year 2019/2020 which is assessed through multiple-choice cloze test?

Objective of the Study

In accordance with the research question above, the researcher is able to write the objective of the study to be attained that is to figure out the achievement of reading comprehension of the eighth grade students of SMPN 2 Susut in academic year 2019/2020 which is assessed through multiple-choice cloze test.

THEORETICAL REVIEW

In this part there are three theories that are going to be discussed related to the present study; those are reading comprehension, assessment of reading comprehension, and multiple-choice cloze test.

Reading Comprehension

Reading is the process results from a negotiation of meaning between the text and its reader. Through reading, people can grasp many information and knowledge. (Harmer, 2007:99) states that reading is useful for language acquisition. The more they read, the better they get at it. Reading also has a positive effect on students’ vocabulary knowledge, spelling and writing.

Reading is about understanding written texts and it is a complex activity that involves both perception and thought (Pang et al., 2003:6). Reading consists of two related processes: word recognition and comprehension. The majority of good reader behaviors refer to comprehension, as well they should. Good readers use the words of the author of the text to construct a personal version of the text in their minds by supplying additional details and make inferences that are not stated in the text.

The problem in reading comprehension is making meaning from the text. The problem is solved by the intentional action of the reader, which includes the purpose for reading as well as the ability to draw upon prior knowledge that is relevant to the text. Snow (2002:11) defines reading comprehension as the process of simultaneously extracting and constructing meaning through interaction and involvement with written language. The use of words “extracting” and “constructing” are to emphasize both the importance and the insufficiency of the text as a determinant of reading comprehension.

Assessment of Reading Comprehension

Teaching process cannot be separated from the assessment in order to get feedback of the whole teaching-learning process that has been done by the teachers and it can be given in form of test. Bachman (2010:3) states that language tests have become a pervasive part of our education system and society. Scores from the language test are used to make inference about individual’s language ability and help us to collect useful information that will benefit a wide variety of individuals.

Achievement tests are widely used in educational research, as well as in school systems to measure what individuals have learned (Ary et al., 2013:201). There are two types of achievement test classified as standardized test and research-made test. Standardized test is published test that has resulted from careful and skillful preparation by experts while researcher-made test is the achievement test that is constructed by researchers to suit specific objectives of their research study.

Multiple-Choice Cloze Test

One of the most popular types of the assessment to evaluate students’ reading comprehension is the cloze test (Brown, 2004:201). The word “cloze” was coined by educational psychologists to capture the Gestalt psychological concepts of “closure”, that is, the ability to fill in gaps in an incomplete image (visual, auditory, or cognitive) and supply omitted details from background schemata.

Broughton et al. (2003:152) state that cloze test or cloze procedure consists of a written passage in which, after the first, introductory paragraph, every fifth (or sixth…or twelfth) word is left out for the student to supply. Cloze test is used to measure the readability of texts, reading and listening comprehension, and general proficiency in English. Cloze tests are global in nature demanding perceptive and productive skills and an integrating knowledge of the various linguistic systems.

Cloze tests can be constructed relatively easily as long as the specifications for choosing deletions and for scoring are clearly defined. In rational deletion exercises (alternatively known as modified cloze tasks), the tester decides, on some rational basis, which words to be deleted (Alderson, 2010:207). Rational deletion tasks might focus on specific lexical items or grammatical structures. This method allows for some flexibility as the tester can avoid deleting words that would be difficult to be predicted from the text; moreover, it gives more washback.

In order to reduce the scoring difficulties, cloze test can be arranged in the form of multiple-choice (Brown, 2004:203). Multiple-choice cloze test allows even more rapid scoring; hand scoring with an answer key or hole-punched grid or computer scoring using scannable answer sheets. Furthermore, in arranging cloze test in multiple-choice format, researcher must consider all guidelines for effective multiple-choice items, especially the choice of appropriate distracters.

RESEARCH METHOD

Research Design

The present study which dealt with the assessment of reading comprehension through multiple-choice cloze test of the eighth grade students of SMPN 2 Susut in academic year 2019/2020 used ex post facto research design with descriptive analysis. Ex post facto research design with descriptive analysis allows the researcher to describe the data and examine relationships between variables. Descriptive analyses are frequently used to summarize a study sample prior to analyzing a study’s primary hypotheses (Marczyk et al., 2010:209). This provides information about the overall representativeness of the sample, as well as the information necessary for other researchers to replicate the study, if they so desire.

Population and Sample

In any educational research study it is important to have an accurate description of the population of elements to form the focus of the study. The population was a fixed one that consists of elements which conform to some designated set of specifications that provide clear guidance as to which elements are to be included in the population and which are to be excluded. The selection of a sample from a defined target population requires the construction of a sampling frame (Ross, 2005:3). A well-constructed sampling frame allows the researcher to ‘take hold’ of the defined target population without the need to worry about contamination of the listing with incorrect entries or entries which represent elements associated with the excluded population.

According to the statements afore mentioned, the population of this present study was all of eighth grade students of SMPN 2 Susut in academic year 2019/2020 with the total of 209 students whereas the samples were 15% from the total of the eighth grade students. Hence, there were 32 samples who had been chosen by using simple random sampling technique with lottery system.

Research Instrument

The research instruments utilized in this present study was in the form of multiple-choice cloze test. As multiple-choice cloze test was in the form of objective test, the test should be tried out to make the test reliable and valid to collect the data. Hence, there were 80 test items of multiple-choice cloze test that had been tried out before. The multiple-choice cloze test was dichotomously scored in the following ways, by which one point was awarded for every correct answer, and zero point was credited for wrong answer.

After the research instrument had been tried out, out of 80 test items, there were 31 test items were unacceptable while the other 49 test items were considered acceptable. Hence, those 49 acceptable test items were considered as valid and reliable items that can be used to collect the data for this present research. However, there were only 40 test items out of 49 acceptable test items to be used to collect the data of the assessment of reading comprehension in this present research. Therefore, nine acceptable test items were removed from this research instrument of multiple-choice cloze test in order to make the instrument balanced.

Data Collection

The data for this present study was collected by administering the research instrument that consisted of 40 test items of multiple-choice cloze test to the 32 samples of the eighth grade students of SMPN 2 Susut in academic year 2019/2020 on January 29, 2020. The samples of this research were given forty five minutes to answer the test and they were not allowed to open dictionary or even to cheat to the others. The instrument in the form of multiple-choice cloze test was administered by the researcher herself. After the research instrument had been administered, the researcher scored the entire answers of the samples. Hence, the raw scores that indicated the achievement of the subjects’ reading comprehension were considered as the data of this present research.

FINDING

The data of this present study were collected by means of administering multiple-choice cloze test to the 32 samples of the research in this present on January 29, 2020. After the research had been conducted and research instrument had been administered, the researcher scored the entire answers of the samples to get the raw scores. The raw scores indicated the achievement of the samples' reading comprehension and the scores were considered as the data of this present study. Through the data of this present study, the real achievement of the samples in reading comprehension could be described. The data needed for this present study, which were obtained from the samples’ score, were analyzed descriptively in order to answer the research problem of this present study that had been clearly formulated previously. The required data that were needed in this present study could be tabulated in table 4.1 as follows:

Table 1 The Data Showing Samples’ Reading Comprehension Achievement

Sample Raw Score (X)

1 34

2 33

3 33

4 32

5 29

6 29

7 29

8 29

9 27

10 27

11 26

12 26

13 26

14 26

15 25

16 25

17 24

18 23

19 23

20 23

21 22

22 22

23 22

24 21

25 21

26 21

27 21

28 20

29 20

30 17

31 17

32 17

N = 32 ∑X = 790

The data obtained for this present study, as what had been tabulated above, would be analyzed by using norm reference measures with five standard values. Before computing the data by using norm reference measures with five standard values, first the researcher should find out the mean of scores that symbolized by “X” and standard deviation (SD) of raw scores. The mean was obtained by counting up all the scores of the samples and then they were divided by the number of samples. The formula of mean score was figured out as follows:

X =

X = 24.69

After the mean of the raw scores had been found out, the standard deviation (SD) of raw scores was calculated by using the following formula:

SD =

Table 2 The Computation of Mean Deviation (d) and the Estimated Squared Mean Deviation (d2) of the Raw Scores

Sample Raw Score (X) Mean Deviation (d) Squared of Mean Deviation (d2)

1 34 9.31 86.68

2 33 8.31 69.06

3 33 8.31 69.06

4 32 7.31 53.44

5 29 4.31 18.58

6 29 4.31 18.58

7 29 4.31 18.58

8 29 4.31 18.58

9 27 2.31 5.34

10 27 2.31 5.34

11 26 1.31 1.72

12 26 1.31 1.72

13 26 1.31 1.72

14 26 1.31 1.72

15 25 0.31 0.09

16 25 0.31 0.09

17 24 -0.69 0.48

18 23 -1.69 2.86

19 23 -1.69 2.86

20 23 -1.69 2.86

21 22 -2.69 7.24

22 22 -2.69 7.24

23 22 -2.69 7.24

24 21 -3.69 13.62

25 21 -3.69 13.62

26 21 -3.69 13.62

27 21 -3.69 13.62

28 20 -4.69 21.99

29 20 -4.69 21.99

30 17 -7.69 59.14

31 17 -7.69 59.14

32 17 -7.69 59.14

N = 32 ∑d2 = 676.96

SD =

SD = 4.59

So: X = 24.69

SD = 4.59

N = 32

After the mean (X) and Standard Deviation (SD) of the raw scores had been calculated, then the scores gained by the samples were able to be computed by using norm reference measures with five standard values. The computation of the scores using norm reference measures was as follows:

1. A (Excellent) = ≥ X + 1.5 SD

= ≥ 24.69 + 1.5 x 4.59

= ≥ 24.69 + 6.88

= ≥ 31.57

So A or excellent achievement was ≥ 31.57

2. B (Good) = X + 0.5 SD

= 24.69 + 0.5 x 4.59

= 24.69 + 2.29

= 26.98

So B or good achievement was in a range of ≥ 26.98  31.57

3. C (Sufficient) = X – 0.5 SD

= 24.69 – 0.5 x 4.59

= 24.69 – 2.29

= 22.40

So C or sufficient achievement was in a range of ≥ 22.40  26.98

4. D (Insufficient) = X – 1.5 SD

= 24.69 – 1.5 x 4.59

= 24.69 – 6.88

= 17.81

So D or insufficient achievement was in a range of ≥ 17.81  22.40

5. E (poor) = < X – 1.5 SD

=  24.69 – 1.5 x 4.59

=  24.69 – 6.88

=  17.81

So E or poor achievement was  17.81

Table 3 The Conversion of the Samples’ Raw Score Using the Basis of Five Standard Values

Sample Raw Score (X) Converted Score The Achievement of the Sample

1 34 ≥ 31.57 A (Excellent)

2 33 ≥ 31.57 A (Excellent)

3 33 ≥ 31.57 A (Excellent)

4 32 ≥ 31.57 A (Excellent)

5 29 ≥ 26.98  31.57 B (Good)

6 29 ≥ 26.98  31.57 B (Good)

7 29 ≥ 26.98  31.57 B (Good)

8 29 ≥ 26.98  31.57 B (Good)

9 27 ≥ 26.98  31.57 B (Good)

10 27 ≥ 26.98  31.57 B (Good)

11 26 ≥ 22.40  26.98 C (Sufficient)

12 26 ≥ 22.40  26.98 C (Sufficient)

13 26 ≥ 22.40  26.98 C (Sufficient)

14 26 ≥ 22.40  26.98 C (Sufficient)

15 25 ≥ 22.40  26.98 C (Sufficient)

16 25 ≥ 22.40  26.98 C (Sufficient)

17 24 ≥ 22.40  26.98 C (Sufficient)

18 23 ≥ 22.40  26.98 C (Sufficient)

19 23 ≥ 22.40  26.98 C (Sufficient)

20 23 ≥ 22.40  26.98 C (Sufficient)

21 22 ≥ 17.81  22.40 D (Insufficient)

22 22 ≥ 17.81  22.40 D (Insufficient)

23 22 ≥ 17.81  22.40 D (Insufficient)

24 21 ≥ 17.81  22.40 D (Insufficient)

25 21 ≥ 17.81  22.40 D (Insufficient)

26 21 ≥ 17.81  22.40 D (Insufficient)

27 21 ≥ 17.81  22.40 D (Insufficient)

28 20 ≥ 17.81  22.40 D (Insufficient)

29 20 ≥ 17.81  22.40 D (Insufficient)

30 17  17.81 E (poor)

31 17  17.81 E (poor)

32 17  17.81 E (poor)

The data of this present study that were analyzed by means of norm reference measurement with five standard values, as what had been shown on the table aforementioned, resulted the converted scores that (1) there were 4 samples attained excellent achievement of reading comprehension; (2) there were 6 samples attained good achievement of reading comprehension; (3) there were 10 samples attained sufficient achievement of reading comprehension; (4) there were 9 samples out of 32 samples attained insufficient achievement of reading comprehension; and (5) there were 3 samples attained poor achievement of reading comprehension. The following block graph clearly presented the findings of the samples’ achievement in reading comprehension in this present study:

Graph 4.1 Percentage of the Achievement in Reading Comprehension

DISCUSSION

The data obtained in this present study were clearly analyzed by using norm reference measurement with five standard values. The result of the data analysis clearly pointed out that (1) there were 12.50% samples who attained excellent achievement of reading comprehension; (2) there were 18.75% samples who attained good achievement of reading comprehension; (3) there were 31.25% samples who attained sufficient achievement of reading comprehension; (4) there were 28.13% samples who attained insufficient achievement of reading comprehension; and (5) there were 9.37% samples of this present study attained poor achievement of reading comprehension. As the findings of this present study were used to assign the grade of the eighth grade students of SMPN 2 Susut in academic year 2019/2020, then 20 samples (62.50%) out of 32 samples could pass the assessment of reading comprehension through multiple-choice cloze test whereas 12 samples (37.50%) out of 32 samples failed in passing the assessment of reading comprehension through multiple-choice cloze test. It clearly pointed out that in general the achievement of reading comprehension by the samples of the eighth grade students of SMPN 2 Susut in academic year 2019/2020 was sufficient.

CONCLUSION

This present study made use of ex post facto research design with descriptive analysis dealt with the assessment of reading comprehension through multiple-choice cloze test of the eighth grade students of SMPN 2 Susut in academic year 2019/2020. The obtained data, which were analyzed by using norm reference measurement with five standard values, clearly indicated that, out of 32 samples, there were 20 samples (62.50%) who could pass the assessment of reading comprehension through multiple-choice cloze test while 12 samples (37.50%) failed in passing the assessment of reading comprehension through multiple-choice cloze test. Therefore, generally the research finding of this present study was sufficient.

Even though the research finding of this present study pointed out sufficient achievement in reading comprehension, it had limited validity and reliability to a certain extent as this present study made use of ex post facto research design with descriptive analysis. The researcher had admitted some weaknesses, such as the lack of control and manipulation of the independent variable, the validity and reliability coefficient of the research instrument, and also the procedures of data collection might also affect validity and reliability of the obtained data in some extent.

SUGGESTION

In reference to the finding of this present study and to improve the students’ achievement in learning English, especially in reading comprehension, the researcher would like to suggest some practical suggestions.

The English teachers of SMPN 2 Susut, especially teachers of the eighth grade students, are suggested to be more active and creative in implementing different methodologies, techniques, and strategies in teaching process that depend on the particular classroom, atmosphere, objectives and circumstances. They are also suggested to give exercises not only in the form of conventional multiple-choice test, but also in other different form of tests, such as: cloze test that can be arranged in the form of multiple-choice, open-ended questions, or completion test; therefore, the students will get used to train and improve their comprehension in reading skill. Furthermore, the teachers also should consider validity and reliability in constructing the test items so they can fit the objectives of the test.

The eighth grade students of SMPN 2 Susut should be more active in learning, practicing, and exercising their reading skill, especially in comprehending the text they are reading, not only during teaching-learning process in the classroom, but also outside the classroom. They should get used to read any books in English, not only their English textbook, in order to train their comprehension on the text that they read. Since reading activity is not easy to be done for English as foreign language or second language students as they should deal with aspects of language, such as word recognition, comprehension, and fluency. It is essential to train the students to be accustomed to read many book as through reading they can get new knowledge and access various information.

REFERENCES

Alderson, J. C. (2010). Assessing Reading. United Kingdom: Cambridge University Press.

Ary, D., Jacobs, L. C., Sorensen, C., Razavieh, A. (2013). Introduction to Research in Education (Ninth Edition). California: Wadsworth, Cengage Learning.

Bachman, L. F. (2010). Statistical Analysis for Language Assessment. United Kingdom: Cambridge University Press

Broughton, G., Brumfit, C., Flavell, R., Hill, P., and Pincas, A. (2003). Teaching English as a Foreign Language (Second Edition). London: Routledge.

Brown, H. D. (2019). Language Assessment: Principles and Classroom Practices (Second Edition). New York: Pearson Longman.

Harmer, J. (2007). How to Teach English New Edition. England: Pearson Education Limited.

Hedgcock, J. S., and Ferris, D. R. (2018). Teaching Readers of English: Students, Texts, and Contexts (Second Edition). New York: Routledge.

Marczyk, G., DeMatteo, D., Festinger, D. (2010). Essentials of Research Design and Methodology. New Jersey: John Wiley & Sons, Inc.

Pang, E. S., Muaka, A., Bernhardt, E. B., and Kamil, M. L. (2003). Teaching Reading. Brussel: International Academy of Education (IAE).

Pollard, L. (2008). Lucy Pollard’s Guide to Teaching English. UK: Lucy Pollard.

Ross, K. N. (2005). Sample Design for Educational Survey Research. Paris: International Institute for Educational Planning.

Snow, C. E. (2002). Reading for Understanding: Toward an R&D Program in Reading Comprehension. USA: RAND.