In film, casting can make or break a picture. Can you imagine Indiana Jones played by anyone but Harrison Ford. Would the first movie spawned have ever happened if Steven Spielberg decided to give the titular role to Peter Coyote or Tom Selleck as he initially planned instead?
It is yet even more important in university admissions. The wrong doctor or lawyer can be rude to those they are meant to serve, or worse cause damage to their patients and clients, and bring down the reputation of a whole profession.
While candidate selection is of paramount importance, there is a great variability in how universities enact this critical process. Many continuously fall into antiquated and ineffective practices such as these, which cause implicit bias, cost time, money and in many cases the reputation of the entire university.
If you want a thriving program or an entire university that becomes a household name and attracts future noble prizewinners, the bottom line is this:
You must master the art of selecting the RIGHT students with the RIGHT skills and the RIGHT motivation.
Here are the proven 14 rules you MUST follow to ensure that your institution is comprised of the most passionate, skilled, and collaborative students.
1. Screen your entire pool of applicants.
Pareto’s principle posits that 80% of effect comes from 20% of causes. It is also known as the law of the vital few and has many applications in all aspects of life. It is a common principle in management that “80% of sales come from 20% of clients”, or conversely, that few employees drive the vast majority of productivity. The same principle should applies to applicant screening and selection. This is obviously a time consuming and costly process, because to get a few top notch applicants to you have to sift through hundreds, thousands and in some case tens of thousands of applications, but it absolutely vital. Traditionally, most admissions practices include some form of initial screening process that eliminates the majority of applicants using "rolling admission", grades, standardized testing, situational judgement tests, and personal statements, to make the pool of applicants more "manageable" for in-person interviews.
But the bad news is if you are not screening your entire pool of applicants and only inviting a few for an in-person interview, your admissions process is not fair nor scientific. Even worse, traditional pre-interview protocols mentioned above have been shown to cause gender and/or socioeconomic bias. That's not all! None of these traditional screening methods have been validated to predict future on-the-job behaviour, rather at best they predict future test performance, which means you could end up with a bunch of good test takers. Add a traditional in-person interview or mini interview to this mix, which is yet another source of bias, and you now have a guaranteed way to introduce inadvertent bias. The best way to overcome this is to use a screening method that allows for inclusion of your entire pool of applicants and selects applicants based on characteristics that do not correlate with socioeconomic status or gender.
SortSmart admissions screening includes a complete solution and allows every single one of your applicants to participate in the entire application process with less time and cost, rather than a few that make it to your in-person interview. And importantly it has been designed to select applicants based on three scientifically proven indicators of future on-the-job behavior (not future test scores), which do not correlate with income level, gender or race: intrinsic motivation, continuousness, and coach-abilty. More on these three characteristics below.
2. Select applicants based on their level of intrinsic motivation.
Most importantly, you must select applicants who are intrinsically motivated to pursue the profession. Intrinsic motivation is the desire to engage in an activity that is self-rewarding on its own regardless of external rewards. The intrinsically moved individual is moved to act by interest, curiosity, or the sheer enjoyment of doing the activity itself. Conversely, extrinsic motivation refers to the desire to engage in an activity due to external pressures. Extrinsic motivation factors can include status, money, social pressure or fear of being judged. In fact, in one of our studies, we discovered that only 25% of future medical doctors reported being intrinsically motivated to pursue medicine, a result duplicated by a survey done by the American Association of Medical Colleges (AAMC) with possible dire consequences for the profession as a whole and more importantly patient safety.
The reason motivation is so important can be summarized easily:
Motivation directs behavior. Moreover, people whose motivation is genuine or self-authored (intrinsic), when compared to those externally controlled to act, have more interest, excitement, and confidence, which results in enhanced performance, persistence, creativity, self-esteem, and general well-being because they experience a continuous positive feedback loop initiated by the activity itself. This is true even when the people have the same competence or efficacy for the activity.
This has been proven over and over in education, entertainment and media, health care, organizations and work, video games, physical activity and exercise and many other fields.
Selecting for intrinsic motivation can be accomplished with carefully designed questions that do not reveal themselves as designed to test motivation. Answers then need to be analyzed in detail by trained individuals. SortSmart admissions screening is the first and only screening tool that allows selection of applicants based on intrinsic motivation.
3. You must assess coach-ability.
Coach-ability or the ability of a person to accept and adapt to suggestion, or constructive criticism can often be more valuable that capability. For what good is a skill if it is not adaptable. Critical to this is the willingness not just to listen, but to act on the good advice of others. Does a trainee take an interest in the perspective of their supervisor or colleague and appreciate the experience and advice they offer? Do they become defensive when given feedback, or do they ask clarifying questions, and dig for more insight? The challenge is determining the coach-ability of an applicant during the selection process. Everyone has experience of working with the person that cannot accept negative feedback or constructive criticism, someone who is “never at fault” and refuses to change or blames others. The negative impact of such individuals on your program and the profession is significant and such individuals must be identified and eliminated during the admission process with careful applicant screening strategies.
4. Pay a lot of attention to conscientiousness.
Studies have shown that the most successful individuals always share the five characteristics of conscientiousness: attention to detail, industriousness, organization, diligence, and prudence. The trait is linked to income and job satisfaction. Additionally, it is one of the most important factors for finding and retaining employment. Selecting conscientious applicants ensures your future professionals are ultra organized, responsible, and plan ahead. They can control their impulses and persevere when faced with challenges. Without conscientiousness, no amount of training will help. You simply cannot steer a parked car.
5. Eliminate written applications, situational judgement tests and in-person interviews.
There are many reasons why being restricted by paper applications is bad for your admissions process as well as the environment. For example, most personal statements have little actual value because they are fluffed to make a mediocre applicant look much more attractive and they take up way too much time to screen, not to mention that most applicants outright lie on their written applications. Situational judgement tests and interviews (including mini interviews) have been shown to cause gender and socioeconomic bias. Even worse, these practices have not been validated to correlate with future on-the-job performance, rather they can only predict future test performance, so at best, you end up selecting great test makers who may be terrible future professionals.
6. Rule out the “Back-ups” to prevent voluntary turnovers.
Every one that has ever applied to any academic program is guilty of having positions that they apply simply as a "back up" to their ideal choice. Applicant turnover is expensive, the effect of losing a superstar, one with high potential, or simply one that is trained and doing an important job well can leave an impact both on team morale and the institution. The costs included in replacing an applicant include: cost of screening and selecting a new person (advertising, interviewing, screening); lost productivity (new person could take years to reach productivity of existing person); lost engagement (high turnover could lead to disengagement of other members of the profession); increase in errors (new applicants will lack specific problem solving skills); training cost; cultural impact (when one applicants leaves, other will undoubtedly wonder “why?”).
It is therefore essential to be able to recognize applicants that are at high risk for voluntary departure. If done correctly this can be done with pointed questions during the selection process. The admissions committee must look for the level of specificity, language use, and passion in the applicant’s response. How much does the applicant have invested in this interview? What do they know about our institution? How much research has this person done in preparation for this interview? Do they have any insight into our culture, operation, or core values that signifies some reflection on the part of the applicant? These are amongst some of the questions that the admissions committee must consider in analyzing each applicant's response.
7. Spot the lies.
Estimates of individuals who exaggerate their applications range from 40-70%. Results are similar when performing background checks on employees: 44% of applicants lied about work histories, 41% lied about education, and 23% lied about credentials or licenses. People think it is more acceptable to lie to an employer than to a romantic partner and that 95% of one sample were willing to lie to get a job. Students often lie even more because they don’t fully appreciate the consequences of their actions and are often willing to lie to get into a desirable profession. While the science of lie detection is still not perfect, fortunately, there are methods to detect lying during the selection process. Importantly, you do not need to rely on your "gut feeling" to do this, but you must have specific strategies to independently verify key facts. For example, if an applicant tells you that they left their recent job because they were laid off, you can simply speak to their former boss to see why this applicant was not offered any other role within their previous organization, rather than just verifying whether or not they were indeed laid off.
8. Prevent inherent biases from inadvertently narrowing your talent pool.
Bias can infiltrate and undermine many parts of the applicant screening process and it has become a major public issue in recent years. During the written application step, names and gender identifiers can implicitly bias the reader. A male name is significantly more likely to receive an interview or an offer than the identical application with a female name. An English-sounding name is more likely to receive an interview than an ethnic-sounding one. And the in-person interview that follows the initial written application process usually exacerbate this problem. Here in addition to gender, the interviewer may be biased by background, physical appearance, body posture, or manner of speaking. Science consistently shows that interviewers are subconsciously drawn to the applicants most like themselves. The overall negative effect is that bias can narrow your talent pool and not enough applicants are fairly and adequately screened because of time and cost to conduct in-person interviews for the entire applicant pool and the perfect applicants may slip right through your fingers. In addition to masking identifiers, the best way to avoid interviewer bias is to have multiple trained applicant evaluators, of various cultural and socioeconomic backgrounds.
9. Make sure your applicant screening process is timed.
Moving away from paper applications and embracing software allows for better data collection and analysis. One important component for this is timing applicant responses. Setting time limits in an online application ensures the applicant is thinking on the spot and not researching and tailoring their answers. Time limits raise the stakes for the applicant and can help ensure commitment from the applicant. However, the time limits must be within a range where they prevent abandonment which can be high in online applications. Essentially, they cannot be so long that the applicant gives up part way through, but must be long enough so that adequate data can be collected that will then be used to quantitatively rank applicants.
10. Test the functional ability of your applicant.
You wouldn’t allow a pilot to fly solo missions unless they have demonstrated their ability to fly a plane first in a realistic simulation. Your applicant screening must include a functional test to test the functional ability of the candidate. If you are selecting future law students, test their ability to debate complex problems and come up with rational solutions. If you are selecting a medical student, given them a task to test their level of altruism or self-directed learning. But a word of warning, functional ability or skills are not as important as intrinsic motivation, conscientious and coach-ability, because someone that has all these three characteristics (especially intrinsic motivation) will joyously learn any skill set quickly. This is why it is best to abandon setting strict grade or standardized testing cut offs for your program to give them too much weight especially at the early stages of your admissions process.
11. Increase the reliability of your applicant screening process by including multiple raters.
As we discussed before bias can undermine the application process and tarnish the reputation of your institution. Some examples include personal inherent bias, stereotyping, recency bias (where evaluator remembers best what has happened recently and allows these recent to influence ratings), leniency, severity, and similarity (person being rated shares some similarity with the rater). An effective approach to preventing bias as well as improving the reliability of the application process is to use multiple raters. This reduces evaluator bias because each of applicants’ responses is graded by a different grader. However, when using multiple raters inter-rater reliability is extremely important. This is the measure of agreement among raters. One measure to increase inter-rater reliability or agreement is to use a carefully structured and optimized scoring rubric. This way every rater is consistently following the same scoring guide when assessing applicants and thus heterogeneity amongst raters will decrease.
12. Use a robust numeric scoring system.
This is the only way to rank applicants. Period. Without it, you are simply out of control and have no way to tell which applicant is better than others and more importantly who is best. Numeric scales can also help eliminate bias by preventing resorting to intangible, subjective rankings, or “gut feeling”. High resolution ranking scales like Likert scales offer a lot more nuance that a simple “yes/no” or “agree/disagree”. Without going into the statistical details, the 7-point scale is the best scale to use, rather than a 5-point scale or a 9-point scale or worse an even-numbered scale. Working with the resulting quantitative data, it is easy to draw conclusions, reports, results, and graphs from the responses. Further categorization can be helpful in reducing the applicant to a single score but rather a multi-categorical breakdown of their assessment.
13. Choose the applicant’s references wisely.
References are unreliable and ineffective. There is no empirical evidence to support the reliability of referees in the selection process. The applicant can always simply select their (usually 3) best references. If the applicant has varied work experiences, asking for a very small sample will not give an adequate assessment of that applicant. This allows the applicant to shape the narrative of their professional career and can be no more trustworthy than the personal statements we discussed earlier. The identity of the referee is rarely confirmed and often difficult to do so. A more sophisticated admission process could involve asking for a chronological list of individuals who have acted as supervisors, bosses or colleagues and randomly sampling a number to contact. Another strategy would be for you or your admissions team to ask to speak with specific past supervisors or colleagues that came up during the course of interview. The bottom line is this: You must choose the referees yourself and not the applicant.
14. When you select a candidate selection software, select one where the creators practice what they preach.
The bad news is that if you were to implement all of the above must-have rules, you’ll be spending a lot of time and money and energy and you will likely give up and go back to old habits that are hurting your admission process and your efforts to promote diversity.
The good news is that our software and programs offer all of the above advantages and more at a fraction of cost and time.
Importantly, here at SortSmart, we screen and hire every single one of our own team members using our own software. This is very important to us because we wouldn’t want to offer a tool that we are not willing to use in our own selection process. In addition, this forces us to continuously make improvements to our software.
If you are an admissions professional and are serious about taking your institution to the next level by selecting top performing, highly motivated applicants while promoting diversity, click here to learn more and schedule your free initial discovery call today.
