VSTAR Portfolio: Design and Implementation of Competency-based Assessment Dashboard

May 17, 2016 9:45 AM – 11:15 AM

Toufeeq Ahmed, PhD, Jay Johnson, Amy Fleming, MD, Kim Lomis, MD, Anderson Spickard III, MD, MS
Vanderbilt University

The Vanderbilt School of Medicine has embarked on competency-based assessments in which students are promoted on the bases of competency demonstration and not only grades.  A key component to support this endeavor is creation of a state of the art competency based dashboard as part of student electronic portfolios.

Milestones Framework:

Using ACGME milestone framework, we created JSON taxonomy with each competency (example: PC- Patient Care), and sub competencies (History & Physical, PC1), sub-sub competency PC1.a, and performance levels (0 to 5, with 0.5 increments). We store assessments in a NoSQL data store, which supports different assessment data points and gives us the flexibility to aggregate the data for longitudinal tracking dashboards.

Assessment collection, MongoDB data store and Aggregation Pipeline:

For each course or program, we store JSON documents as separate activity collections in MongoDB. This allows us to query for a selected milestone across an entire collection for any value or track a single student over time for particular milestones. With real-time data aggregation, we are able to query this data and use Big Data approaches without the need of an expensive data warehouse.

In this presentation, we will demonstrate and discuss the technical features of VSTAR Portfolio, an interactive, student dashboard supporting competency–based self–assessment, assessment by advisors and teachers, and personalized learning plans.