This book explores the appropriateness of Item Response Theory (IRT) in language testing. It investigates the dimensionality of the reading tests of the Cambridge First Certificate of English Test (FCE) and the Test of English as a Foreign Language (TOEFL), and the relative fit of 1, 2, 3 parameter IRT models in which the Rasch model is closely examined. Finding that the Rasch model fails to provide an adequate fit for the data, the study recommends that its predominant use in language testing be re-evaluated. Moreover, the 2 and 3 parameter models fit the data much better than the Rasch model. Finally, it shows that moderate departures from unidimensionality do not necessarily lead to an unacceptable model fit, nor does the use of IRT in test development guarantee that the unidimensionality assumption will be satisfied.
Even though the application of IRT to language testing has recently attracted much attention, no model-data fit research has been conducted to explore the appropriateness of IRT modeling in language testing. The tenability of the strong assumption of unidimensionality has not been studied systematically, and little is known concerning the effects of departure from unidimensionality on the estimation of parameters and on model fit. Furthermore, no study has examined the adequacy of the Rasch model which has been predominant in language testing. The present study investigated the dimensionality of the reading and vocabulary sections of two widely-used English as a foreign language proficiency tests, the University of Cambridge First Certificate of English (FCE) and the Test of English as a Foreign Language (TOEFL). It also compared the relative model fit of three IRT models: 1, 2, and 3 parameter model. Dimensionality of the tests was investigated using Stout's method, factor analyses, and Bejar's method. Secondly, employing fit statistics, invariance check, and the residual analyses, the current study investigated the adequacy of the Rasch model, and the effects of multidimensionality on parameter estimation and model fit. The results of this study suggest the following: (1) Even the TOEFL reading subtest, developed using the three-parameter IRT model, was multidimensional. This appears to be due to underlying factors associated with the reading passages. (2) The FCE reading and vocabulary subtest, based on the traditional British examination system, was found to be essentially unidimensional. (3) Bejar's approach to checking dimensionality appears to be inadequate in that the results differ across the 1, 2, and 3 parameter models. (4) The finding that the Rasch model clearly fails to provide an adequate fit for these data suggests that the prevailing use of the Rasch model in language testing needs to be re-evaluated. (5) The 3 parameter model fit the data only marginally better than did the 2 parameter model. This suggests that for language tests, the discrimination parameter is more significant than is the guessing parameter. (6) A moderate departure from unidimensionality does not appear to invalidate IRT modeling with the data. This finding suggests the possibility of more justified implementation of IRT modeling in language testing.
Item response theory (IRT) has moved beyond the confines of educational measurement into assessment domains such as personality, psychopathology, and patient-reported outcomes. Classic and emerging IRT methods and applications that are revolutionizing psychological measurement, particularly for health assessments used to demonstrate treatment effectiveness, are reviewed in this new volume. World renowned contributors present the latest research and methodologies about these models along with their applications and related challenges. Examples using real data, some from NIH-PROMIS, show how to apply these models in actual research situations. Chapters review fundamental issues of IRT, modern estimation methods, testing assumptions, evaluating fit, item banking, scoring in multidimensional models, and advanced IRT methods. New multidimensional models are provided along with suggestions for deciding among the family of IRT models available. Each chapter provides an introduction, describes state-of-the art research methods, demonstrates an application, and provides a summary. The book addresses the most critical IRT conceptual and statistical issues confronting researchers and advanced students in psychology, education, and medicine today. Although the chapters highlight health outcomes data the issues addressed are relevant to any content domain. The book addresses: IRT models applied to non-educational data especially patient reported outcomes Differences between cognitive and non-cognitive constructs and the challenges these bring to modeling. The application of multidimensional IRT models designed to capture typical performance data. Cutting-edge methods for deriving a single latent dimension from multidimensional data A new model designed for the measurement of constructs that are defined on one end of a continuum such as substance abuse Scoring individuals under different multidimensional IRT models and item banking for patient-reported health outcomes How to evaluate measurement invariance, diagnose problems with response categories, and assess growth and change. Part 1 reviews fundamental topics such as assumption testing, parameter estimation, and the assessment of model and person fit. New, emerging, and classic IRT models including modeling multidimensional data and the use of new IRT models in typical performance measurement contexts are examined in Part 2. Part 3 reviews the major applications of IRT models such as scoring, item banking for patient-reported health outcomes, evaluating measurement invariance, linking scales to a common metric, and measuring growth and change. The book concludes with a look at future IRT applications in health outcomes measurement. The book summarizes the latest advances and critiques foundational topics such a multidimensionality, assessment of fit, handling non-normality, as well as applied topics such as differential item functioning and multidimensional linking. Intended for researchers, advanced students, and practitioners in psychology, education, and medicine interested in applying IRT methods, this book also serves as a text in advanced graduate courses on IRT or measurement. Familiarity with factor analysis, latent variables, IRT, and basic measurement theory is assumed.
Item Response Theory, though it has become a widely recognized tool in language testing research, is still not used frequently in practical language assessment projects. This book intends to provide a theoretical overview as well as to give practical guidance concerning the application of IRT in item bank building in a language testing context by presenting a particular project in a higher education setting.
In the decade of the 1970s, item response theory became the dominant topic for study by measurement specialists. But, the genesis of item response theory (IRT) can be traced back to the mid-thirties and early forties. In fact, the term "Item Characteristic Curve," which is one of the main IRT concepts, can be attributed to Ledyard Tucker in 1946. Despite these early research efforts, interest in item response theory lay dormant until the late 1960s and took a backseat to the emerging development of strong true score theory. While true score theory developed rapidly and drew the attention of leading psychometricians, the problems and weaknesses inherent in its formulation began to raise concerns. Such problems as the lack of invariance of item parameters across examinee groups, and the inadequacy of classical test procedures to detect item bias or to provide a sound basis for measurement in "tailored testing," gave rise to a resurgence of interest in item response theory. Impetus for the development of item response theory as we now know it was provided by Frederic M. Lord through his pioneering works (Lord, 1952; 1953a, 1953b). The progress in the fifties was painstakingly slow due to the mathematical complexity of the topic and the nonexistence of computer programs.
Introduction to measurement -- The one-parameter model -- Joint maximum likelihood parameter estimation -- Marginal maximum likelihood parameter estimation -- The two-parameter model -- The three-parameter model -- Rasch models for ordered polytomous data -- Non-Rasch models for ordered polytomous data -- Models for nominal polytomous data -- Models for multidimensional data -- Linking and equating -- Differential item functioning -- Multilevel IRT models.
Several decades of psychometric research have led to the development of sophisticated models for multidimensional test data, and in recent years, multidimensional item response theory (MIRT) has become a burgeoning topic in psychological and educational measurement. Considered a cutting-edge statistical technique, the methodology underlying MIRT can be complex, and therefore doesn’t receive much attention in introductory IRT courses. However author Wes Bonifay shows how MIRT can be understood and applied by anyone with a firm grounding in unidimensional IRT modeling. His volume includes practical examples and illustrations, along with numerous figures and diagrams. Multidimensional Item Response Theory includes snippets of R code interspersed throughout the text (with the complete R code included on an accompanying website) to guide readers in exploring MIRT models, estimating the model parameters, generating plots, and implementing the various procedures and applications discussed throughout the book.
