[Competencies] Preliminary results of performance framework review

mjoclark at cox.net mjoclark at cox.net
Mon Jun 16 16:22:38 UTC 2014


I agree with some of the other comments that permitting negative numbers allows for flexibility and it should be up to the user to label performance levels. In some instances, negative scores might be appropriate.

MJC

---- "Larimer wrote: 
> All,
> 
> Am in agreement with what Sascha stated.  I don’t see any logic in using negative numbers to rate competency, and would find it very unusual if a medical student even displayed 0 competence on a given dimension.  A rating that low would call into question why the student was admitted to med school in the first place, or whether the student had lost all motivation to succeed and was in need of counseling. I suppose 0 might have some utility, but anything in a negative direction doesn’t seem helpful for identifying students who need remediation.
> 
> Deborah
> __________________
> Deborah Larimer, EdD
> Office of Medical Education
> Tulane University School of Medicine
> Phone: 504-988-3328; 504-988-6600
> Email: dlarimer at tulane.edu<mailto:dlarimer at tulane.edu>
> 
> From: Competencies [mailto:competencies-bounces at medbiq.org] On Behalf Of Albright, Susan
> Sent: Friday, June 13, 2014 7:20 PM
> To: Cohen, Sascha; Valerie Smothers; MedBiq Competencies Group
> Subject: Re: [Competencies] Preliminary results of performance framework review
> 
> Could we ever imagine a scoring range that ran from -5 through 0 to +5?
> 
> I've never seen it but would it help anyone to be able to use it?
> 
> Susan
> ________________________________
> From: Competencies [competencies-bounces at medbiq.org] on behalf of Cohen, Sascha [Sascha.Cohen at ucsf.edu]
> Sent: Friday, June 13, 2014 12:11 PM
> To: Valerie Smothers; MedBiq Competencies Group
> Subject: Re: [Competencies] Preliminary results of performance framework review
> If scores can be set against increasing or decreasing scales, and defined within the context of the scale, then there is little reason to disallow a scoring rule that begins or ends in negative integer territory. I wouldn’t personally use that construct, but hey, I’m just one voice in the peanut gallery.
> 
> I would lean toward the consideration that scores should reflect a realistic context, and that one aspect of our discussion in the past has been that the inclusion of “0” as a score value provides a meaningful data point (a learner has no competence within the evaluated scale); and I’m not sure that it would be meaningful to have a score that shows “less than 0 competence”, if you follow. So it really depends on context and the effective definition of both scales and scoring values.
> 
> So, I would vote for leaving the data type as xsd:integer, and providing guidelines of best practice regarding the definition of scoring and scale modeling.
> 
> Sascha
> 
> 
> Sascha Benjamin Cohen
> Director of Strategic Development for Ilios
> UCSF School of Medicine
> sascha.cohen at ucsf.edu<mailto:sascha.cohen at ucsf.edu>
> 
> 415.704.4521 vox
> http://curriculum.ucsf.edu
> http://www.iliiosproject.org
> 
> 
> 
> From: Competencies [mailto:competencies-bounces at medbiq.org] On Behalf Of Valerie Smothers
> Sent: Friday, June 13, 2014 6:46 AM
> To: MedBiq Competencies Group
> Subject: [Competencies] Preliminary results of performance framework review
> 
> Hi, everyone.
> 
> We’ve received some comments on the Performance Framework from the MedBiquitous Standards Committee. These are captured on the following wiki page: http://groups.medbiq.org/medbiq/display/CWG/Standards+Committee+review%2C+June+2014
> Please note there is a space for the working group’s recommendations on this table.
> 
> Several substantive comments from Dan Rehak relate to whether scores can be negative or 0. We had decided that scores could be 0. We never exclusively ruled out negative numbers. I emailed Dan to see if he had a compelling argument against allowing negative numbers:
> 
> I wanted to share my thoughts on your comments and see if you wanted to talk it over by phone. The biggest issue is whether scores can be negative. The working group did discuss allowing scores to be 0, but if there were a convincing argument to be made for disallowing that, I think they could be swayed. The group also decided to allow the direction of the scale to be determined by framework authors. So you may have a scale of 1 to 5 where 1 is most competent, or you may have a scale of 1 to 5 where 1 is least competent. The reason we decided to do that was that the ISO specs around proficiency levels point to scales of differing direction. They used Judo as an example, where the Kyu scale goes from 10 being least competent to 1 being most competent. After Kyu comes Dan, where 1 is least competent and 9 is most competent.
> 
> We haven’t seen any examples of negative numbers being used in scores, but I don’t know that we want to restrict that. Again, if there is a good argument to be made for restricting further, I am all ears.
> 
> Here is how Dan replied:
> 
> I don't have any strong feelings about the "right" answer here.  I just wanted to be sure about the valid range of values -- if zero or negative values are permitted, that's OK.  But if not, then I was thinking that the data type be more restrictive than just xsd:integer.  What triggered this was that the names are often associated with more restrictive ranges.
> I think the kind of things you mention, e.g., 0, negative, and scale and direction of scale should be mentioned in the text, and then just make the datatypes align.
> Please share your thoughts and recommendations by Thursday June 18. I would like to send the Standards Committee our recommendations by June 19.
> 
> Best regards,
> Valerie
> 
> Valerie Smothers
> Deputy Director
> MedBiquitous
> vsmothers at jhmi.edu<mailto:vsmothers at jhmi.edu>
> +1-410-735-6142
> www.medbiq.org<http://www.medbiq.org/>
> Sign up for our newsletter<http://visitor.constantcontact.com/d.jsp?m=1103590106851&p=oi>
> Follow us on Twitter!<https://twitter.com/medbiq>
> 





More information about the Competencies mailing list