Inclusive Design in Testing: The Candidate Experience

Testing programs have a responsibility to hold high standards for quality and fairness, and the candidate experience is critical to consider at every stage. Inclusive design recognizes the variety of unique human experiences and needs and incorporates solutions to those needs in design. By accounting for the design needs of an otherwise excluded group of people, the benefits extend beyond that group and are often appreciated by the greater population.

Inclusive design is defined by the Inclusive Design Research Centre as “design that considers the full range of human diversity with respect to ability, language, culture, gender, age and other forms of human difference.” This definition includes three dimensions: diversity and uniqueness, inclusive process and tools and broader beneficial impact.

The practice of inclusive design covers all aspects of exam design and delivery — you could even consider the positive impact a well-designed exam may have on the population the test-taker will go on to work with after certification.

Exam Design and Practice

Drawing from research projects and case studies, we will discuss the importance of inclusive design and offer practical advice for its implementation in exam development and delivery. Consider the following when evaluating your organization:

• Do you have a policy in place to ensure that all members of your organization are aware of the regulations surrounding exams as a baseline for governance?
• If you deliver exams worldwide, have you considered the laws and regulations of other countries (and within the United States, legal differences between states)?
• How does your organization’s event planning impact the candidate experience?

Questions like these can get your organization thinking about small but impactful ways of incorporating inclusive design at various stages of the exam development lifecycle. For example, building exam dates to avoid religious holidays is respectful and prevents candidates from needing to contact the organization to arrange a religious exemption. A diverse population of subject matter experts is foundational to the creation of fair and valid exams, and inclusive design is essential to executing this work. A standard setting held in Florida may be less appealing for subject matter experts impacted by controversial new state laws. For an item writer who uses a wheelchair, a meeting held at a hotel directly attached to a major airport would reduce the number of transportation transfers compared to more remote locations.

Working with vendors and third parties that align with your organization’s values can be helpful when considering inclusive design. Forming a committee either internally, or between your partners, can help to establish guidelines and processes within your organization.

Case Study Examples: Designing an Exam for Inclusivity

Reed in Partnership (a partner and part-owner of Eintech) is a large United Kingdom-based organization that works with numerous government, public and private sector stakeholders. Their mission is “to positively transform people and their communities.” As part of their commitment to this mission, Reed in Partnership has created an inclusion and belonging (I&B) infrastructure internally which forms an integral part of their overall I&B company plan. This is made up of four subgroups with a protected characteristic: Disability, Ethnic Minorities, LGBTQ+ and Women in the Workplace. These working groups are comprised of internal Reed employees to guide the conversation around equity and inclusion, often working with third-party organizations to enact wider social change beyond Reed. By researching alongside these subcommittees, we were able to discover common barriers to inclusive design encountered in both the written exam and treatment by test center staff, remote proctors and other support roles, including unintentional appropriation.

After initiating the research and extending the conversation to others, we can now provide examples of everyday issues and proposals for how credentialing experts can assist with inclusive design. By working together with people affected by these issues, we were also able to form suggestions of how the candidate experience might be improved, and in doing so, noted that this would create a better and more consistent experience, positively impacting candidate anxiety and their ability to demonstrate knowledge in an exam.

The following case studies illustrate how organizations can proactively incorporate inclusive design to improve the candidate experience, and how these considerations extend beyond mere compliance measures for accommodations.

Case Study #1

An exam candidate has an unrecognized disability, such as undiagnosed ADHD or dyslexia, and takes a standard examination without requesting accommodations. This candidate is not receiving formal accommodations, but the inclusive design of the exam offers beneficial features that help not just this candidate, but all candidates, demonstrate their ability.

Inclusive design approaches:

• Use a platform that allows test takers to manipulate the font, color, size of text and layout.
• Formatting options can allow candidates to highlight or strikethrough text.
• Ensure consistency in exam instructions, such as clearly marking when only one answer is required and grouping items of one type together.
• Avoid overcomplicating the question by using plain language.
• Opt for positively phrased language when possible, and especially avoid double negatives.

All of the above can be achieved by ensuring a disability lead or “inclusion champion” is involved in both the initial platform or test design as well as in the early stages of item development. Eventually, their work will filter through an organization.

Case Study #2

Test center staff use colloquial speech and culturally appropriated language. Candidates who speak English as an additional language find the instructions more difficult to understand, and candidates who identify with the appropriated culture report feeling devalued.

Inclusive design approaches:

• Incorporate training for testing center staff that includes sensitivity training and education about mindful language.
• Standardize staff interactions with scripts developed by a panel or committee of people from diverse backgrounds. Ensure staff members explain instructions to candidates clearly and with empathy.
• Seek accreditation from an organization that promotes the use of plain English, such as the Plain English Campaign in the United Kingdom.

Case Study #3

A candidate who is lactating did not make advance arrangements with the testing program or the testing center to use a breast pump on exam day.

Inclusive design approaches:

• Make it easy for candidates to report changes in circumstance and clearly communicate that accommodations are available (e.g., reminder emails linking to candidate guides).
• Provide training across the organization to ensure staff awareness of accommodations.
• Investigate the accommodations your contracted testing centers can provide and ensure they align with the organization’s values. For lactating candidates, there is wide variability in the quality of the physical space the centers provide.

Case Study #4

An item writer for a medical board exam created a clinical vignette with a linked set of items. In an effort to be inclusive, the writer focused the vignette on a transgender patient. The writer mistakenly believed the patient’s gender was only relevant for the final prompt. During item review, a different subject matter expert revised earlier items in the linked set to reflect a more appropriate medical management plan for a transgender patient.

Inclusive design approaches:

• Recruit subject matter experts with diverse perspectives and areas of expertise and ensure representative voices are involved across all stages of exam development.
• Be careful to avoid depicting stereotypes or to overly constrain the representation of a group of people. For example, on exams covering medical content, ensure transgender patients are represented receiving medical care outside of “transgender health.”
• Leverage item metadata to create balanced exam forms that meet predetermined, consciously selected targets for representation.
• Use opportunities secondary to the testing point to provide representation (e.g., developing media representative of an intended population).

Inclusive Design Benefits Everyone

Did you notice that all of these solutions, despite being suggested as an answer to a specific concern, would actually benefit any person taking an exam by ensuring consistency and protocol? Inclusive design cannot be achieved without inclusive input, and it can be considered a proactive implementation for all rather than a reactive response to some. Inclusive design is not a fixed process, but a value system that is constantly searching for the next step to take in making the world more inclusive, and better, for everyone.