I.C.E. Exchange Session Preview — Making Validity More Holistic: Connecting Validity With Inclusivity and Test Security
While security measures are a critical part of test taking, some can negatively impact validity by not embracing inclusivity. Prioritizing inclusivity in the validity of a credentialing program creates more opportunities for everyone to earn credentials.
In their I.C.E. Exchange session “Making Validity More Holistic: Connecting Validity With Inclusivity and Test Security,” John Kleeman, MA, FBCS, C.Eng, CIPP/E, along with Liberty J. Munson, PhD, and Benjamin Hunter, advocate for finding the right balance between security and inclusivity.
Read Kleeman’s responses below for an insight into how the session will advocate for humanizing the exam experience.
I.C.E.: What is your background with credentialing, and what excites you most about being in this field?
John Kleeman (JK): There are three presenters for this session, and it’s very much a collaborative effort.
Benjamin Hunter is the vice president of sales at Caveon. Hunter broke into the testing field in 2012 and has specialized in test security since his arrival. His passion for test security comes from his experience helping his child through open-heart surgery and the associated importance of credentialing and valid testing.
Liberty Munson is the director of psychometrics at Microsoft. Munson has managed the psychometric elements of Microsoft’s vast credentialing program for nearly 18 years. She is proud of the work that Microsoft is doing in leading the industry to create more inclusive, accessible and delightful exam experiences.
John Kleeman (myself) is an executive vice president at Learnosity, an assessment technology company. I wrote the first version of the Questionmark assessment software in the late 1980s and have been in the field ever since. I am excited about how technology has the potential to democratize learning and assessment worldwide.
I.C.E.: Why is this topic important to the profession?
JK: A certification exam seeks to determine if someone is competent. In a hypothetically perfect certification exam, everyone who is competent will pass and everyone who is not competent will fail.
The reason why exam security is so important is that if someone cheats on an exam, there is a high risk there will be an error — the person may pass when they shouldn’t. Exam security efforts are directed to reduce the prevalence of such errors.
If an exam is not inclusive or if exam security measures penalize people with certain characteristics, then there will also be errors. Some people will fail the exam who shouldn’t. Both reduce validity and the usefulness of the associated credential.
The key message of our presentation is to encourage testing programs to consider the impact of their security measures on test takers who may be different from the typical exam candidate to humanize the exam experience. Security measures are useful because they improve validity, but when they impact some people negatively, they reduce validity.
I.C.E.: What are one to two key highlights you hope attendees will take away from your session?
JK: Considering the diverse needs of our candidates will improve the experience for everyone. Approximately 25% of the U.S. population has some type of disability, more than 20% speak a language other than English at home and more than 20% don’t have home internet. By creating inclusive experiences, you can not only make your credentialing or certification program more valid and more relevant, but you also create opportunities for traditionally underrepresented and marginalized groups.
We will share some examples of non-inclusive test security practices and how these practices have impacted people in the real world. We will also share examples of how Microsoft is bringing this idea to life in their certification program. We encourage attendees to walk a mile in the shoes of their candidates and think how to balance security and inclusivity.
I.C.E.: What is something many find surprising about the topic you will be covering?
JK: People sometimes think that making exams accessible and inclusive involves security compromises, but this isn’t the case.
For example, we will show how Microsoft has allowed candidates to take unscheduled breaks in proctored exams without advanced notice. A candidate who needs to visit the bathroom, get a snack or check on a child can do so, and this has not resulted in security issues. Microsoft has also allowed candidates to access their online technical documentation website, Microsoft Learn, during the exam without security incidents.
As past Microsoft CEO Steve Ballmer said, “Accessible design is good design — it benefits people who don’t have disabilities as well as people who do. Accessibility is all about removing barriers and providing the benefits of technology for everyone.”
I.C.E.: Is there anything else you want to share?
JK: Our I.C.E. Exchange session is at 4 p.m. on Monday, Nov. 18, 2024, in Miami, Florida. We promise to be interesting and engaging. We hope to see you there!
Registration is now open for the I.C.E. Exchange held Nov. 17-20 in Miami Beach, Florida. Early-bird discounts last until Oct. 4, so register early!