In a recent blog post on the Canadian Medical Association Journal (CMAJ), president and CEO of the Association of Faculties of Medicine of Canada (AFMC), Dr. Geneviève Moineau, gave her thoughts on SortSmart’s new findings that show that the majority of medical students and residents recognize the need, and support the development of an improved, more transparent admissions screening tool. We would like to thank the CMAJ and Dr. Moineau for a thoughtful examination of our findings.
The leadership of the AFMC acknowledged that better tools to evaluate “the intrinsic motivation to provide care and to advance scientific discovery” are warranted. This is promising and demonstrates the continued dedication of the organization to advance scientifically sound and fair admissions practices in Canada. This reflects the opinion of 90% of medical school students and residents that overwhelming demanded improvements to admissions practices, 97% of whom indicated their willingness to support a new, improved, and transparent admissions screening tool.
The goal of our study, as discussed previously, was to examine admissions practices to find out whether the admitted students find the current admissions practices acceptable or do they prefer improvements; whether or not there is a continued evidence of implicit admissions bias as reported previously (for a few examples, click here, here, here, here, and here); and whether or not admissions practices can discern the level of intrinsic motivation of applicants and we’ll explain why we focused on intrinsic motivation shortly but first some of the comments made in the report merit examination.
Briefly, the findings suggest that:
- Admissions practices appear to select individuals that were mostly (68%) motivated to apply to medical school by status, financial gain, or familial tradition;
- 49% of overall medical students and residents identify themselves as Caucasian with 48% of those from families earning over $100,000/year, the highest proportion of all groups.
- On the other hand, 46% of those identifying as visible minorities were from families earning less than $60,000/year.
- No correlation was found between cultural background or socioeconomic status and intrinsic motivation to pursue medicine.
It is important to note that this study does not question the dedication, work ethic, or passion of current medical students, residents, practicing medical doctors, medical schools, any associations, admissions offices, and their members who have an impossible task and for all of whom we have vast admiration and respect as highlighted in our previous report. However, we must remain cognizant that the admissions practices and tools developed in the past, while all well-intentioned, are subjected to continuous learning and improvement over time as any body of knowledge and that is the essence of self-improvement as a society. And while the status quo is comforting, it is an impediment to growth and progress.
1. What is our motivation?
Our study motivations were discussed in a presentation by our CEO and founder, Dr. Behrouz Moemeni, PhD, who initially witnessed the shortcomings of current practices and the impact of the implicit bias at BeMo, an educational firm that helps aspiring future professionals that he co-founded several years before the founding of SortSmart. We have been working tirelessly towards our goal of creating a better candidate selection tool because we believe everyone deserves fair access to higher education regardless of their social, racial or economical background. We also believe, organizations deserve access to highly motivated, top performing candidates and this would create a win-win scenario for all parties involved, including the public at large.
One of the primary concerns raised by Dr. Moineau is the validity of the issues raised by the study and the motivation behind them. “This is a company that is creating a tool for candidate selection, so they have a vested interest in suggesting there’s a problem”. The problems with professionalism, bias, and burnout have long been well documented and acknowledged by the community. Secondly, we are a private independent company, but that fact does not invalidate our findings, rather strengthens it. In fact, our research is the foundation for our services, and our services are scrutinized daily by customers who expect results and we hold ourselves accountable by offering an unparalleled money back guarantee. This is why we aim to make our data as transparent and accessible as possible. In fact, we encourage and welcome genuine constructive feedback as it ensures our science is valid. But ultimately our success is directly tied to the success of those we serve.
It should also be noted that currently almost all Canadian medical schools use admissions tools sold by two for-profit companies. Both of these companies seem to also agree that there are concerns over professionalism and bias. While these tools are used, evidence suggests these tools appear to cause gender and socioeconomic bias, in agreement with our findings, and there has yet to be evidence that they have been able to reduce the problem with professionalism. However, in contrast, to avoid any conflict of interest we have intentionally avoided collaboration with any universities because our public universities must be able to remain hungry for continuous research and development and in our opinion, being part owner of for-profit companies impedes the ability of a university to adopt and pursue any new technologies that may result in dissolution of their existing companies. Furthermore, we fundamentally differ in our approach to selection, which brings us back to our question: Why is evaluating motivation important?
It is important because motivation directs behavior. Professionalism and academic tests can only predict future test performance and they may inadvertently select individuals who are great test takers, but not necessarily passionate about the profession. Importantly, such tests do not predict on-the-job performance. For a test to be valid it must be able to measure the constructs it is designed to measure. For example, a ruler is a good tool for measuring distance but it's not good for measuring the temperature of a room. Similarly, the current admissions tools have either zero validation or at best have been "validated" to measure future test performances such as performance on the objective structured clinical examinations (OSCE) or other examinations. This means that by design such tests are only valid measures of future tests and are not necessarily able to predict future on-the-job behavior. We believe the best approach to predict future behavior is to assess the level of motivation of each applicant. For a more in depth article discussing the shortcomings of current admissions practices, click here.
The reason motivation is so important can be summarized easily:
People whose motivation is genuine or self-authored (intrinsic) when compared to those externally controlled to act have more interest, excitement, and confidence, which results in enhanced performance, persistence, creativity, self-esteem, and general well-being because they experience a continuous positive feedback loop initiated by the activity itself. This is true even when the people have the same competence or efficacy for the activity. For more information on motivation, click here, here, here, here, here, here, here, here, here & here. Or read our previous article here.
According to our survey, it appears that majority of those selected using current admissions practices are extrinsically motivated. That means that the admissions practices are not able to discern the level of intrinsic motivation, which is not surprising given that neither of the current admissions tools have been developed nor claim to be capable of detecting level of intrinsic motivation.
2. What exactly is the difference between extrinsic and intrinsic motivation?
The CMAJ article also raised the concern that our line of questions regarding motivation was “leading” and thus inflating the lack of intrinsic motivation problem. We conducted a very similar independent survey in the US, using the same exact methodology, which closely matched the officially reported US demographics. Importantly, the 25% intrinsic motivation rate in our US study mirrors the rate reported by the American Association of Medical Colleges (AAMC). In the AAMC survey the students were asked, “How influential were the following in helping you choose your specialty?” The choices included extrinsic motives such as “income expectations”. The results show that year after year only ~22% of respondents chose income as bearing “no influence” on their decision to choosing a speciality, meaning only 22% were primarily intrinsically motivated regardless of monetary reward. This provides yet another layer of validation for our studies.
In that regard, Mr. Victor Do’s assertion that “I think residents who said they wanted to be a leader, or wanted to be paid well, would still be intrinsically motivated” is incompatible with Self-Determination Theory and its delineation of intrinsic and extrinsic motivation factors. Intrinsic motivation is defined as being motivated to action solely by the enjoyment of the task, free from influence of external factors such as status or financial remuneration. For more information on motivation, follow the links provided above or read our previous article.
3. Are the results representative of all Canadian medical students?
Another concern was whether the survey was representative of the population. Mr. Do noticed that “More than 15% of respondents came from the University of Toronto alone. Respondents from schools in the three prairie provinces made up less than 12% of the total.” According to the AFMC, medical students from Manitoba, Saskatchewan, Alberta, and Calgary combine for 17.1% of medical students in 2016/17. As mentioned in our report, our sample size based on the total number of medical school students and residents results in a 5% margin of error. These values are largely within that range. Nonetheless, this 5% underrepresentation would not skew our major finding, even if this 5% of students were ALL intrinsically motivated. From the same AFMC report, University of Toronto accounted for almost 10% of all Canadian Medical students, again within our reported margin of error.
The sample size was large enough to allow for statistical representation with accuracy of +/-5%, and more importantly our gender distribution, socioeconomic demographic, racial demographic and university distribution were similar to previous official reports in both the US and Canada. (You can compare our finding in Canada with AFMC reports and compare our US findings with AAMC reports)
We thank the CMAJ for choosing to cover our work and are grateful for the critical assessment it received from the AFMC and student association representative. We are excited to have this discussion and make a positive contribution.
It is reassuring that the AFMC has acknowledged the need voiced by most medical students and resident for better evaluation protocols. We believe that an improved selection protocol should include intrinsic motivation. We are willing to lend our expertise in that regard by joining the think-tank as humble advisors if measures are taken to avoid conflict of interest. Improving the established admissions process will be challenging, change always is. But not everyone will be ready for improvements. For those forward-thinking institutions interested in reducing inherent bias, widening their talent pool, streamlining the entire process, and cutting costs while (and most importantly!) selecting intrinsically motivated individuals, we invite you to join us on this mission. Join our free waitlist to get early access to our admissions screening tool.
To your success,
Your friends at SortSmart
SortSmart Candidate Selection
Photo credit: flickr/bertknot CC license: https://creativecommons.org/licenses/by-sa/2.0/
