Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks

Yong-Won Lee
April 2006
Language Testing;Apr2006, Vol. 23 Issue 2, p131
Academic Journal
A multitask speaking measure consisting of both integrated and independent tasks is expected to be an important component of a new version of the TOEFL test. This study considered two critical issues concerning score dependability of the new speaking measure: How much would the score dependability be impacted by (1) combining scores on different task types into a composite score and (2) rating each task only once? To answer these questions, generalizability theory (G-theory) procedures were used to examine the impact of the numbers of tasks and raters per speech sample and of subsection lengths on the dependability of speaking scores. Univariate and multivariate G-theory analyses were conducted on rating data collected for 261 examinees for the study. The finding in the univariate analyses was that it would be more efficient to increase the number of tasks rather than the number of ratings per speech sample in maximizing the score dependability. The multivariate G-theory analyses also revealed that (1) the universe (or true) scores among the task-type subsections were very highly correlated and that (2) slightly larger gains in composite score reliability would result from increasing the number of listening � speaking tasks for the fixed section lengths.


Related Articles

  • Comparing PETS and GEPT in China and Taiwan. Mei Wu // English Language Teaching;Jun2012, Vol. 5 Issue 6, p48 

    This paper compares the Public English Test System (PETS) administered in mainland, China and the General English Proficiency Test (GEPT) administered in Taiwan, from the aspects of test levels, test contents and scoring weight. Compared with the PETS, the GEPT is found to value the English...

  • Evaluation of Two ESP Textbooks. Fraidan, Abdullah Al // English Language Teaching;Jun2012, Vol. 5 Issue 6, p43 

    This paper evaluated two ESP textbooks using the evaluation of McDonough and Shaw (2003) based on external and internal evaluation. The first textbook is Business Objectives (1996) by Vicki Hollett, and the second textbook is Business Studies, Second Edition (2002) by Alain Anderton. To avoid...

  • Effects of Negotiated Interaction on Mongolian-nationality EFL Learners' Spoken Output. Xueping Li // English Language Teaching;Jun2012, Vol. 5 Issue 6, p119 

    The present study examines the effect of negotiated interaction on Mongolian-nationality EFL learners' spoken production, focusing on the teacher-learner interaction in a story-telling task. The study supports the hypothesis that interaction plays a facilitating role in language development for...

  • Facilitating Autonomy and Creativity in Second Language Learning through Cyber-tasks, Hyperlinks and Net-surfing. Akinwamise, T. K.; Adedara, O. G. // English Language Teaching;Jun2012, Vol. 5 Issue 6, p36 

    The digitalization of academic interactions and collaborations in this present technologically conscious world is making collaborations between technology and pedagogy in the teaching and learning processes to display logical and systematic reasoning rather than the usual stereotyped informed...

  • The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Ockey, Gary J.; Koyama, Dennis; Setoguchi, Eric; Sun, Angela // Language Testing;Jan2015, Vol. 32 Issue 1, p39 

    The purpose of this study was to determine the extent to which performance on the TOEFL iBT speaking section is associated with other indicators of Japanese university students’ abilities to communicate orally in an academic English environment and to determine which components of oral...

  • The Relationship between Language Learning Strategies, Language Learning Beliefs, Motivation, and Proficiency: A Study of EFL Learners in Iran. Ghavamnia, Maedeh; Kassaian, Zohreh; Dabaghi, Azizollah // Journal of Language Teaching & Research;Sep2011, Vol. 2 Issue 5, p1156 

    Since the emergence of strategy research in the 1970s many issues have been examined. One of these areas which has been favored in recent years is related to answering questions like 'what variables are related to the choice and the use of learner strategies?' and 'How strong is the influence of...

  • SELECTIVE TRANSFER IN THE ACQUISITION OF ENGLISH DOUBLE OBJECT CONSTRUCTIONS BY BRAZILIAN LEARNERS. Vidigal ZARA, Júlia; Pereira de OLIVEIRA, Fernando Luiz; de SOUZA, Ricardo Augusto // Alfa: Revista de Lingüística;2013, Vol. 57 Issue 2, p519 

    The present study investigates the acquisition of the English double object constructions (GOLDBERG, 1995) by Brazilian learners. We hypothesize that, due to first language (L1) influences, the prepositional ditransitive construction (John gave a book to Mary) will be acquired earlier, while the...

  • English-for-Science-and-Technology Teaching Materials: Phase One Evaluation. Hui-Chuan Liao; Yeh-uh Hsueh Chen // English Language Teaching;Jun2012, Vol. 5 Issue 6, p89 

    The importance of English for the workplace, in addition to that of general English proficiency, is receiving increasing attention from educators and policy makers in the field of teaching English as a foreign language. In order to prepare college students for their professional development,...

  • Effects of Lexical Modification on Incidental Vocabulary Acquisition of Iranian EFL Students. Negari, Giti Mousapour; Rouhi, Mahdieh // English Language Teaching;Jun2012, Vol. 5 Issue 6, p95 

    The present article reports on the results of a study designed to investigate the effects of two types of lexical modification i.e., lexical simplification and elaboration, on incidental vocabulary acquisition of Iranian EFL learners.To this end, four versions of experimental texts containing 20...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics