Formative Evaluation of the Implementation of the Medical Education Metrics Standard for Continuing Education

Francis Kwakwa MA, Radiological Society of North America
Andrew Rabin, BS, CECity
Sean Hayes, Psyd, AXDEV Group
Valerie Smothers, MA, MedBiquitous
MedBiquitous Working Group Members

The MedBiquitous Metrics Working Group developed the Medical Education Metrics Standard (MEMS) to enable aggregate analyses of Continuing Education (CE) evaluation and outcomes data and to foster reliable research as to the extent CE impacts clinician knowledge and competence. Approved as an American National Standard in 2009, MEMS allows organizations to digitally compile continuing education evaluation data across systems. The Accreditation Council for Continuing Medical Education (ACCME) has launched a new online reporting system called the Program and Activity Reporting System (PARS). ACCME accredited providers are able to utilize this system in order to electronically report utilization information and other required data. PARS is designed to accept data in an XML format based in part on MEMS. As this constitutes the first large scale use of MEMS on a national basis, the authors will share their experience with using MEMS as part of PARS reporting. Additionally, there will be a focus on how several organizations are supporting PARS reporting, and as part of that, how the MEMS standard was implemented. The authors will also present formative evaluation findings collected regarding the experience of those who have implemented MEMS, and as such, offer case study findings as to how to assess implementation o f new systems. The approach may be a useful template for others seeking to evaluate the impact of new technologies.