Statistical Methods for Combining Diagnostic Tests and Performance Evaluation Metrics

Statistical Methods for Combining Diagnostic Tests and Performance Evaluation Metrics

Author: Chengning Zhang (Ph.D.)

Publisher:

Published: 2022

Total Pages: 0

ISBN-13:

DOWNLOAD EBOOK

In biomedical studies, it is usually the case that several diagnostic tests can be performedon an individual or multiple disease markers are available simultaneously, and that many of them may be associated with the clinical outcome. In practice, a single test or marker often has limited diagnostic performance. Therefore, it is important to combine multiple sources of information available to achieve higher classification performance. This dissertation focuses on statistical methods for combining multiple diagnostic tests and the corresponding performance evaluation metrics. In the first project, we provide a survey of the current state of the art in methods for combining multiple tests. We categorize existing methods into three general groups and conduct extensive simulation studies to compare the performance of different combination methods. The reviewed methods serve as benchmark for developing new combination approaches in the following projects. In the second project, we consider the problem of combining multiple tests whose values are missing at random (MAR). In addition, we aim to exploit the known monotonicity relationship between the input variables and the disease outcome for gains in diagnostic accuracy. We develop a novel likelihood-based approach to monotone classification that accounts for missing inputs in a natural and principled way. The risk score function is obtained through the nonparametric maximum likelihood estimation (NPMLE). A novel expectation-maximization (EM)-type algorithm is devised to compute the NPMLE by treating the monotonicity-constrained risk score function as a cumulative distribution for a latent random vector. Through simulation studies and a real data example, we demonstrate that the proposed method outperforms state-of-the-art methods for combining multiple inputs under monotonic assumption, especially when the inputs contain missing data. We illustrate our approach with a dataset from a recent nonalcoholic fatty liver disease (NALFD) study. In the third project, our approach established in the second part is extended to the scenario where one covariate is randomly censored. The proposed approach consists of two steps. In step one, we use a Cox proportional hazards model for the distribution of the censored covariate given other covariates in the model, this conditional distribution is used for calculating the observed likelihood of data. In step two, a similar expectation maximization (EM)-type algorithm is devised, based on observed data likelihood from step one, to compute the NPMLE of the monotonicity-constrained risk score function. Through simulation studies, we demonstrate that the proposed method outperforms the simple but inefficient complete-case analysis as well as the substitution methods. We apply our method to the data set from a primary biliary cirrhosis (PBC) study conducted at Mayo Clinic. The proposed methods in part two and three can be extended to multi-class cases, where the labels have an inherent order but no meaningful numeric distance between them. A natural question arises as to how to evaluate the classification performance under such setting. Therefore, in the fourth project, we consider the problem of performance evaluation metrics for ordinal classification. We propose three novel performance evaluation metrics that better capture the ordinality of the outcomes. The first metric is adapted from the area under the receiver operating characteristic (ROC) curve (AUC), while the latter two are simple and interpretable generalizations of the Harrell's concordance index (C-INDEX). Moreover, we show the optimality of the AUC based metrics through Neyman-Pearson lemma. We conduct extensive simulation studies to confirm the usefulness of the proposed performance metrics for ordinal classification.


Statistical Methods for Performance Evaluation and Their Applications

Statistical Methods for Performance Evaluation and Their Applications

Author: Longzhuang Li

Publisher:

Published: 2002

Total Pages: 342

ISBN-13:

DOWNLOAD EBOOK

Statistical performance evaluation has many applications. In these applications, many alternative solutions or hypotheses exist and the ones performing the best in terms of predetermined measurements are sought. The performance measures of hypotheses are numerical numbers and have to be obtained based on examples and may contain noise. In addition, due to the time and resource constraints in real applications, it is often impractical or even impossible to evaluate all hypotheses. Thus, statistical metrics are used to evaluate the performance of hypotheses efficiently using a limited number of examples and tests. There are many statistical metrics available and their results depends on many factors, such as the number of test cases, whether or not the performance measurements are noisy, and the distribution of performance measurements of the hypotheses. Selecting the most appropriate statistical metrics is a challenging task. In this dissertation, we propose a general framework for statistical performance evaluation. The framework incorporates various statistical metrics and automatically selects the most appropriate one based on the characteristics of the application problem. We have identified the following important problem characteristics: the number of hypotheses, the size of sample data for each hypothesis, the distribution of performance measurements, and the distribution of noise in performance measurements. Then, we apply statistical performance evaluation methods to four applications: evaluation of search engine performance on the Web, analysis and improvement of HITS-based document ranking algorithms, optimization design of filter banks for image compression, and optimization design of filter banks for signal denoising. In the first application, we apply statistical methods to evaluate the precision of search engines. We have performed extensive experiments using real search engines on the Web and obtained promising results. In the second application, we statistically analyze the performance of the combination of HITS-based algorithms and relevance scoring methods, and develop a adaptive weighting method which achieves better results without any content analysis. In the third application, we develop an optimization-based approach to design biorthogonal filter banks for image compression, in which statistical performance evaluation methods are used to select the solutions that are more generalizable to other images unseen in the optimization design stage. Similarly, in the fourth application, we develop an optimization-based method for designing orthonormal filter banks for signal denoising and apply statistical performance evaluation methods in selecting more generalizable solutions. In these two applications, our methods have obtained filter banks that perform better than the benchmark existing filter banks.


Evaluation of Diagnostic Systems

Evaluation of Diagnostic Systems

Author: John Swets

Publisher: Academic Press

Published: 1982-01-28

Total Pages: 280

ISBN-13:

DOWNLOAD EBOOK

Evaluation of Diagnostic Systems: Methods from Signal Detection Theory addresses the many issues that arise in evaluating the performance of a diagnostic system, across the wide range of settings in which such systems are used. These settings include clinical medicine, industrial quality control, environmental monitoring and investigation, machine and metals inspection, military monitoring, information retrieval, and crime investigation. The book is divided into three parts encompassing 11 chapters that emphasize the interpretation of diagnostic visual images by human observers. The first part of the book describes quantitative methods for measuring the accuracy of a system and the statistical techniques for drawing inferences from performance tests. The subsequent part covers study design and includes a detailed description of the form and conduct of an image-interpretation test. The concluding part examines the case study of a medical imaging system that serves as an example of both simple and complex applications. In this part, three mammographic modalities are used: industrial film radiography, low-dose film radiography, and xeroradiography. The case study focuses on the overall reliability of accuracy indices made by its main components, that is, the variabilities across cases, across readers, and within individual readers. The supplementary texts provide study protocols, a computer program for processing test results, and an extensive list of references that will assist the reader in applying those evaluative methods to diagnostic systems in any setting. This book is of value to scientists and engineers, as well as to applied, quantitative, or experimental psychologists who are engaged in the study of the human processes of discrimination and decision making in either perceptual or cognitive tasks.


Diagnostic Tests Toolkit

Diagnostic Tests Toolkit

Author: Matthew Thompson

Publisher: John Wiley & Sons

Published: 2011-09-29

Total Pages: 114

ISBN-13: 1119951801

DOWNLOAD EBOOK

Diagnostic Tests Toolkit Diagnostic Tests Toolkit Finding the evidence for diagnostic tests Establishing an evidence-based methodology to assess the effectiveness of diagnostic tests has posed problems for many years. Now that the framework is in place health professionals can find and appraise the evidence for themselves. With Diagnostic Tests Toolkit clinicians and junior researchers can interpret the evidence for the effectiveness of different types of diagnostic tests, or develop their own research using the successful ‘step-by-step’ format of the Toolkit series. Written by renowned clinical researchers, this is the first basic guide to evidence-based diagnosis. It is equally valuable to starters in clinical research and those needing a quick refresher on the core elements of evidence-based diagnosis.


Finite Mixture Models

Finite Mixture Models

Author: Geoffrey McLachlan

Publisher: John Wiley & Sons

Published: 2004-03-22

Total Pages: 419

ISBN-13: 047165406X

DOWNLOAD EBOOK

An up-to-date, comprehensive account of major issues in finitemixture modeling This volume provides an up-to-date account of the theory andapplications of modeling via finite mixture distributions. With anemphasis on the applications of mixture models in both mainstreamanalysis and other areas such as unsupervised pattern recognition,speech recognition, and medical imaging, the book describes theformulations of the finite mixture approach, details itsmethodology, discusses aspects of its implementation, andillustrates its application in many common statisticalcontexts. Major issues discussed in this book include identifiabilityproblems, actual fitting of finite mixtures through use of the EMalgorithm, properties of the maximum likelihood estimators soobtained, assessment of the number of components to be used in themixture, and the applicability of asymptotic theory in providing abasis for the solutions to some of these problems. The author alsoconsiders how the EM algorithm can be scaled to handle the fittingof mixture models to very large databases, as in data miningapplications. This comprehensive, practical guide: * Provides more than 800 references-40% published since 1995 * Includes an appendix listing available mixture software * Links statistical literature with machine learning and patternrecognition literature * Contains more than 100 helpful graphs, charts, and tables Finite Mixture Models is an important resource for both applied andtheoretical statisticians as well as for researchers in the manyareas in which finite mixture models can be used to analyze data.


Statistical Evaluation of Diagnostic Performance

Statistical Evaluation of Diagnostic Performance

Author: Kelly H. Zou

Publisher: CRC Press

Published: 2016-04-19

Total Pages: 243

ISBN-13: 1439812233

DOWNLOAD EBOOK

Statistical evaluation of diagnostic performance in general and Receiver Operating Characteristic (ROC) analysis in particular are important for assessing the performance of medical tests and statistical classifiers, as well as for evaluating predictive models or algorithms. This book presents innovative approaches in ROC analysis, which are releva


Improving Diagnosis in Health Care

Improving Diagnosis in Health Care

Author: National Academies of Sciences, Engineering, and Medicine

Publisher: National Academies Press

Published: 2015-12-29

Total Pages: 473

ISBN-13: 0309377722

DOWNLOAD EBOOK

Getting the right diagnosis is a key aspect of health care - it provides an explanation of a patient's health problem and informs subsequent health care decisions. The diagnostic process is a complex, collaborative activity that involves clinical reasoning and information gathering to determine a patient's health problem. According to Improving Diagnosis in Health Care, diagnostic errors-inaccurate or delayed diagnoses-persist throughout all settings of care and continue to harm an unacceptable number of patients. It is likely that most people will experience at least one diagnostic error in their lifetime, sometimes with devastating consequences. Diagnostic errors may cause harm to patients by preventing or delaying appropriate treatment, providing unnecessary or harmful treatment, or resulting in psychological or financial repercussions. The committee concluded that improving the diagnostic process is not only possible, but also represents a moral, professional, and public health imperative. Improving Diagnosis in Health Care, a continuation of the landmark Institute of Medicine reports To Err Is Human (2000) and Crossing the Quality Chasm (2001), finds that diagnosis-and, in particular, the occurrence of diagnostic errorsâ€"has been largely unappreciated in efforts to improve the quality and safety of health care. Without a dedicated focus on improving diagnosis, diagnostic errors will likely worsen as the delivery of health care and the diagnostic process continue to increase in complexity. Just as the diagnostic process is a collaborative activity, improving diagnosis will require collaboration and a widespread commitment to change among health care professionals, health care organizations, patients and their families, researchers, and policy makers. The recommendations of Improving Diagnosis in Health Care contribute to the growing momentum for change in this crucial area of health care quality and safety.


Principles and Practice of Clinical Trials

Principles and Practice of Clinical Trials

Author: Steven Piantadosi

Publisher: Springer Nature

Published: 2022-07-19

Total Pages: 2573

ISBN-13: 3319526367

DOWNLOAD EBOOK

This is a comprehensive major reference work for our SpringerReference program covering clinical trials. Although the core of the Work will focus on the design, analysis, and interpretation of scientific data from clinical trials, a broad spectrum of clinical trial application areas will be covered in detail. This is an important time to develop such a Work, as drug safety and efficacy emphasizes the Clinical Trials process. Because of an immense and growing international disease burden, pharmaceutical and biotechnology companies continue to develop new drugs. Clinical trials have also become extremely globalized in the past 15 years, with over 225,000 international trials ongoing at this point in time. Principles in Practice of Clinical Trials is truly an interdisciplinary that will be divided into the following areas: 1) Clinical Trials Basic Perspectives 2) Regulation and Oversight 3) Basic Trial Designs 4) Advanced Trial Designs 5) Analysis 6) Trial Publication 7) Topics Related Specific Populations and Legal Aspects of Clinical Trials The Work is designed to be comprised of 175 chapters and approximately 2500 pages. The Work will be oriented like many of our SpringerReference Handbooks, presenting detailed and comprehensive expository chapters on broad subjects. The Editors are major figures in the field of clinical trials, and both have written textbooks on the topic. There will also be a slate of 7-8 renowned associate editors that will edit individual sections of the Reference.


Introduction to Statistical Methods in Pathology

Introduction to Statistical Methods in Pathology

Author: Amir Momeni

Publisher: Springer

Published: 2017-09-07

Total Pages: 322

ISBN-13: 3319605437

DOWNLOAD EBOOK

This text provides a comprehensive and practical review of the main statistical methods in pathology and laboratory medicine. It introduces statistical concepts used in pathology and laboratory medicine. The information provided is relevant to pathologists both for their day to day clinical practice as well as in their research and scholarly activities. The text will begins by explaining the fundamentals concepts in statistics. In the later sections, these fundamental concepts are expanded and unique applications of statistical methods in pathology and laboratory medicine practice are introduced. Other sections of the text explain research methodology in pathology covering a broad range of topics from study design to analysis of data. Finally, data-heavy novel concepts that are emerging in pathology and pathology research are presented such as molecular pathology and pathology informatics. Introduction to Statistical Methods in Pathology will be of great value for pathologists, pathology residents, basic and translational researchers, laboratory managers and medical students.


Systematic Reviews in Health Care

Systematic Reviews in Health Care

Author: Matthias Egger

Publisher: John Wiley & Sons

Published: 2008-04-15

Total Pages: 512

ISBN-13: 0470693142

DOWNLOAD EBOOK

The second edition of this best-selling book has been thoroughly revised and expanded to reflect the significant changes and advances made in systematic reviewing. New features include discussion on the rationale, meta-analyses of prognostic and diagnostic studies and software, and the use of systematic reviews in practice.