Words Matter: 'Minimally Qualified Candidate,' 'Minimally Competent Candidate' — Or Another Phrase?

Thursday, October 10, 2024

Walking into a job analysis or standard setting study, one of the most common phrases we hear to refer to the examinee right at the cut score is the “minimally qualified candidate.” However, more and more often, subject matter experts (SMEs) are commenting that they would prefer a different phrase. From the experiences of the authors, we have found preferences for specific words or phrases are often related to industry. For example, those in professional credentialing seem to be less resistant to the phrase “minimally qualified candidate” versus those in the field of education. This investigation is a result of these observations: an attempt to understand whether there is truly a strong preference for one phrase over another and whether that preference varies by industry.

The primary purpose of credentialing programs is to protect the health, safety and welfare of the public. Passing a certification or licensure exam is one step (and sometimes the only step) candidates must achieve to become credentialed. SMEs help determine the passing score for a credentialing exam during a standard-setting study. For most standard-setting methods, SMEs discuss the knowledge, skills and abilities needed for mastery of the testing domain and then make performance judgments about individuals who have met but not exceeded the standard of mastery. One of the more common phrases used in the literature to conceptualize this individual is the “minimally competent candidate” (e.g., Plake, Impara, & Potenza, 1994; Plake, 2008; Ricker, 2006; Stone, Koskey, & Sondergeld, 2011). Terms such as “minimally acceptable person” (Angoff, 1971), “minimally qualified candidate,” “borderline” or “just qualified” are examples of other phrases used in the industry to describe the same individual.

While the cut score recommendations resulting from a standard-setting study are based on subjective judgments, the established standard is not arbitrary when SMEs can conceptualize and agree upon a definition of minimal competence. While there is some literature that investigates how SMEs’ conceptualization of the phrase used in a standard-setting study affects their ratings (e.g., Peabody and Wind, 2019), there is a lack of guidance in the literature as to which phrase(s) should or should not be used.

Investigating SME Opinions on Word Choice

Data Recognition Corporation (DRC) conducted a study to determine SMEs' preference for phrases. DRC administered a four-question survey with two demographic questions to individuals in the credentialing industry via an email blast to licensure and certification professionals as well as through postings on LinkedIn. The survey was live from July 14-31, 2023. The survey asked individuals which phrase(s) they believed should, as well as should not, be used when referring to individuals who pass a credentialing exam by achieving a score equal to the passing score. Respondents could select as many phrases as they felt were appropriate. On average, respondents selected three to four responses. Respondents were also asked to explain their choices. Figure 1 displays a copy of the survey.

Figure 1

Survey about the wording to use to refer to a candidate at the passing score

Ninety-two individuals from different industries responded to the survey. Approximately half of the respondents were in a healthcare-related field, and 41% had participated in a standard-setting study as an SME. Figure 2 displays the results of the survey and Table 1 lists preferred phrases by industry and previous standard-setting study participation. Since respondents could select multiple phrases, the percentages shown in Table 1 will not sum to 100%. For example, in the “Overall” row, 51% of respondents preferred the phrase “minimally competent” and 47% preferred the phrase “minimally qualified." However, there was a large overlap in the number of respondents who selected both of these phrases as a preferred phrase. Shown in both Figure 2 and Table 1, the most popular phrases that respondents believed should be used to describe a candidate at the passing score were “minimally qualified” and “minimally competent,” while “just barely proficient” and “borderline” were the least preferred. In the free response section of the survey, respondents suggested dropping the adverb and simply using words such as “qualified” and “competent” to describe candidates at the passing score. The responses were similar for both respondents who had and had not participated as an SME during a standard-setting study.

Respondents were asked to explain their preferences. Nearly two-thirds indicated that the context of the test’s intended construct of measurement is important to consider. For example, one respondent stated, “In some cases, ‘borderline’ is fine, but it's really a different concept than the others.” Another respondent wrote, “Depending on context, qualified or competent would be more or less appropriate.” In a third example, a respondent comments: “Minimally qualified is used for certifications unrelated to protecting the public while minimally competent is for certifications that do.” In addition to this observation, 35% of respondents believed that many of the survey phrases included negative connotations (e.g., “minimally”) and should not be used during a standard-setting study. Other respondents (12%) believed the selected phrase should depend on how well SMEs can understand and apply the phrase, e.g., one respondent commented, “Regardless of the term adopted for a standard-setting study, it is critical that the panelists for a specific study are properly oriented to understand and apply the concept when providing item ratings.” Some also expressed concerns about the lack of distinction between the terms “qualification” and “competence” because a qualified person is not necessarily the same as a competent person. Nonetheless, “minimally qualified” and “minimally competent” were the most favored phrases.

Figure 2

Which phrase(s) do you believe should and should not be used when referring to individuals who pass an exam by achieving a score equal to the passing score?

Table 1

Phrases respondents believe should be used to refer to individuals who achieve a score equal to the passing score on a credentialing exam.

Note: Respondents could select as many phrases as they felt were appropriate. Therefore, the sum of the percentages for each row exceeds 100%.

Evaluating the Study Results

This study investigated the preferred phrase(s) to use when referring to an individual earning the exact passing score on an exam. The results of the study suggest the words used during a standard-setting study should be given some thought. While “minimally competent candidate” and “minimally qualified candidate” were the most favored phrases in this study, it is possible that these were selected most often because they are the phrases most people hear in the industry.

While commonly used to describe an individual at the passing score, there is evidence that the word “minimally” did not resonate well with all respondents. The reason given for this unfavorable opinion was because “minimally” has a negative connotation. One respondent explains, “I try to avoid the word minimal as it automatically elicits a negative feeling.” However, no responses indicated firsthand experience with that word or other words directly affecting the outcome of a standard-setting study. Thus, without other changes to the standard-setting procedure, it is possible that using a phrase such as “competent” during a standard-setting study without the “minimally” descriptor word may lead to a passing score for an exam being set too high. On the other hand, including the descriptor may result in a more appropriate score. It is also possible that once the idea of the individual at the passing score is clearly defined, then the actual word, phrase or acronym used does not matter. If this is the case, then the word selection becomes more of a choice from an optics perspective than semantics.

Based on the research, we believe standard-setting study facilitators and organizations should consider the following when adopting a phrase to use during a standard setting study:

Which phrase(s) align best with the industry to which the cut score is being applied? For example, “minimally qualified” and “minimally competent” seem to align well with the medical/dental industry, while “borderline” is less aligned.
Does a phrase with an easy-to-say acronym (e.g., MQC, MCC) need to be used or could an individual at the passing score simply be referred to as the “candidate at the passing score?”
Is a qualifier word needed, such as “minimally” or “just,” or could an individual at the passing score simply be referred to as “qualified” or “competent,” without the standard setting exercise being unduly impacted?

While this study does not universally answer the above questions, it does highlight the need for consideration of the phrase used.

What Now?

Which word(s) or phrase should be used to refer to the individual at the passing score? For licensure and certification exams, the facilitator of a standard setting study should explain to the SMEs that a line must be drawn to differentiate those who will become credentialed and those who will not. The guidance used to validate that decision comes, in part, by having a collective understanding of the definition of a candidate at the passing score. The description of this individual as well as the training SMEs undergo to have a common understanding of this individual is vital to having consistent ratings during a standard-setting study, and it is important to ensure that the label used accurately reflects the individuals to which the standard is applied. With that said, further research could explore how the use of different phrases directly influences the cut score decisions or, at least, facilitates a discussion with SMEs about how they may change their ratings if a different phrase were used.

References

Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational Measurement (2^nd Ed., pp. 508-600). Washington, DC: American Council on Education.

Peabody, M. R. & Wind, S. A. (2019). Exploring the influence of judge proficiency on standard-setting judgments. Journal of Educational Measurement, 56(1), 101-120.

Plake, B. S. (2008). Standard setters: Stand up and take a stand! Educational Measurement: Issues and Practice, 27(1), 3–9. https://doi.org/10.1111/j.1745-3992.2008.00110.x

Plake, B. S., Impara, J. C., & Potenza, M. T. (1994). Content specificity of experts in a standard-setting study. Journal of Educational Measurement, 31(4), 339-347.

Ricker, K. L. (2006). Setting cut-scores: A critical review of the Angoff and modified Angoff methods. The Alberta Journal of Educational Research, 52(1), 53-64.

Stone, G. E., Koskey, K. L. K., Sondergeld, T. A. (2011). Comparing construct definition in the Angoff and objective standard setting models: Playing in a house of cards without a full deck. Educational and Psychological Measurement, 71(6), 942-962.