Table 6 Multiple comparison test (Ryan’s method) (NV = 70)
Related Figures (8)
OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test; SHOAQ.-T, short-answer question test. The ordering of the test papers in each of the four groups There was a one-week interval between the testing sessions; this was assumed to be long enough for the participants to forget some of the story and thus decrease the negative order effect. The participants were not told the dates of sessions. Each session was conducted at the beginning of English classes by the instructor and his assistant. The participants were informed of the purpose of the tests and they were encouraged to answer as many questions as possible. OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test: SHOAQ-T, short-answer question test. Descriptive statistics (V = 70) reliability coefficient of SHOAQ-T is moderate (« = 0.597). In the four tests the SHOAQ-T had the highest mean (M = 22.500), OE-TRAN the second (M = 20.157), the MC-TRAN (M = 19.286), and the CLOZE-T the lowest (M = 11.786). The participants are divided into two proficiency groups (higher and lower) based on the results of the official scholastic achievement tests in their high school. Because of the test security code, the results of the tests are not open to the author. However, the author was told that there was a statistically significant dif- ference of test scores between the higher and lower learners at p < 0.01 from the t-test result. Descriptive statistics (higher learners) (N = 35) OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test; SHOAQ.-T, short-answer question test. OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test SHOAQ.-T, short-answer question test. Descriptive statistics (lower learners) (N = 35) Analysis of variance (NV = 70) he results indicated that each of the two factors was statistically significant (Pro- ciency level: F = 36.984, p < 0.01; test method: F = 82.218, p < 0.01), which means iat the participants’ test performance is sensitive to differences of proficiency level nd test method. However, the interaction of the two factors was not significant * = 1.974, p= 0.12). Put another way, the analysis revealed that test method — Le., hether or not translation was used — did not interact with proficiency level; that is ) Say, it is an independently significant statistical effect regardless of student ability. In order to clarify the effect of each factor on the participants’ test performance, \ultiple comparison tests using Ryan’s method (Ito, 2001) were then employed. First \ultiple comparison on (A) proficiency level (between-subject factor) reveals that the igher proficiency learners (M = 22.357) performed better than the lower learners = 14.507) (¢ = 6.081, p < 0.01). In addition, another multiple comparison test on 3) test method (within-subject factor) was conducted to examine where the statistical ifference lay among the six pairs of the four test scores. Table 6 shows that it existed in ich different pair of the four tests except for MC-TRAN (M = 20.071) and OE- RAN (M = 19.286). Therefore, we can argue that the following order of test scores nerged: SHOAQ-T > OE-TRAN = MC-TRAN > CLOZE-T. Table 7 displays the correlation coefficients among the four tests. There were \oderate correlations between OE- TRAN and MC- TRAN (r= = 0. 603), MC- TRAN Correlations among the tests (V = 70) OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test; SHOAQ.-T, short-answer question test. Table 7 OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test SHOAQ.-T, short-answer question test. Correlations among the tests (higher learners) (N = 35) Correlations among the tests (lower learners) (NV = 35) OE-TRAN, open-ended translation test; MC-TRAN, MC translation test; CLOZE-T, cloze test; SHOAQ.-T, short-answer question test.
