TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES Good evaluation practice guidelines for health informatics - A shared networked initiative, EFMI, IMIA for guidelines development GEP-HI Coordinators Pirkko Nykänen, Tampere University Jytte Brender, Aalborg University

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES Motivation Need to have good practice guidance No single, global methodology exists –variety of approaches, methodologies, mindsets Good practice guidelines should serve as: – Framework to design evaluation studies, to select methodologies, to conduct studies – Support for health professionals and users to understand evaluation and contribute to evaluation studies

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES Objectives Generic practical guidelines that provide evaluators, users, health professionals with structured, scientifically justified and grounded, comprehensive and understandable set of rules for good practice –To design and carry out evaluation studies in health informatics domain

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES Objectives – practically Guidelines: – Criteria and aspects, both in quantitative and qualitative terms, to consider at each evaluation stage – Carry out evaluation of the specific aspect, criteria at specific stage – Design and manage the evaluation study Guidelines from all stakeholders viewpoints: third party evaluators, users, health professionals, managers, decision makers, health economists,...

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES How to develop GEP-HI guidelines Follow guidelines development principles Expert panel Existing literature and material on evaluation studies, methodologies, reported evaluation experiences, guidelines for good clinical practice, codes of ethics, good implementation practices critical study and assessment, review External review!

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES Exploration ReportBuyer Team: iterative discussion and consensus Selection of Methods Internal Report First Study Design ReportBuyer Study Design Including: Goal and Mandate ReportBuyer Participants Internal Reports Interim Reports Buyer (?) Participants Final Study Report Buyer Participants Publication 1... n Study Implementation The Study Project Controlling & Risk Management Receiver GEP-HI guidelines C Weßel GEP-HI Suggestion for a Flowchart

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES GEP-HI guidelines Study exploration: starting questions of the study First study design: preliminary design of an evaluation study Selection of methods: detailed methodological plan Detailed study design: project management and evaluation methods Study implementation: Designed study is performed Project controlling and risk management: good project management pracices Reports and publications: follow STARE-HI recommendations on how to report evaluation studies

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES GEP-HI guidelines.. status now Draft exists Guidelines are presented for evaluation and asssessment for the medical informatics community –IMIA, EFMI experts – Guidelines are made accessible soon through: iig.umit.at/efmi everybody is asked to contribute guidelines are published as a paper presented for IMIA

TAMPEREEN YLIOPISTOUNIVERSITY OF TAMPERE TIETOJENKÄSITTELYTIETEIDEN LAITOS DEPARTMENT OF COMPUTER SCIENCES Contact