Across the Industry

Upcoming I.C.E. Research: Live Remote Proctoring and Candidate Experience

By Timothy J. Muckle, PhD, Linda Waters, PhD, and Kari Hodge, PhD

There has been a dramatic increase in the utilization of live remote proctoring (LRP) for administering high stakes credentialing examinations over the past two years, due to the COVID-19 global pandemic. As credentialing organizations adapted to the chaotic unfolding of 2020, remote proctoring provided a convenient, if not necessary, alternative for many certification bodies feeling the pandemic’s economic pressures, while simultaneously experiencing sustained demand for professional certification. With no clear end to the global crisis in sight, many organizations continue to include live remote proctoring as a viable option for the foreseeable future.

I.C.E. (and the NCCA) met its commitment to provide guidance to the industry for the implementation of remote proctoring. Specifically, the NCCA provided a pandemic-related waiver (Spring 2020) to allow programs to utilize LRP. Also, the NCCA undertook an assessment (planned in 2019 before the pandemic), beginning in January 2020, to more comprehensively examine whether a certification program could meet NCCA Standards while using LRP. Furthermore, several studies have investigated examination performance and candidate receptivity of online testing with remote proctors, as compared to in-person testing at brick-and-mortar centers. Future research promises to conduct more comparability and feasibility studies.

While the guidance and research thus far has focused on decision-making frameworks, implementation, comparability and receptivity, very little work has been done to provide impactful guidance to candidates to ensure that they select the modality that best suits their individual circumstances for testing. LRP typically involves the following segments:

  1. System-pre-check, verifying technical requirements at a specific point-in-time
  2. Down/load launch of secure browser
  3. Identity authentication
  4. Security readiness check/360-degree compliance scan of testing space
  5. Launch/administration of examination
  6. Ongoing monitoring throughout the exam
  7. Ability to terminate the exam, should misconduct be suspected or confirmed

Based on the experience of the I.C.E. Research & Development task group with various programs and vendor platforms, segments 1-4 are where candidates seem to struggle and require technical assistance or intervention. Once the candidate can start the examination, things progress rather seamlessly. The “rub” lies within the first four segments of the check-in procedures.

While LRP has been widely implemented to date, there is still some misalignment between the convenience candidates expect and what the LRP platforms and internet providers can realistically deliver. Candidates want the same immediacy and ease from LRP that Amazon shoppers get when they purchase and have delivered items with a single click; or the same instant connection experienced with virtual meetings providing visuals and audio of far-flung parties within seconds. Because LRP has been billed largely as a convenience accommodation (take your test at home!), candidates have come to expect the same streamlined ease and absence of barriers provided by other virtual platforms with which they are familiar. The security mechanisms necessary to ensure a secure administration introduces some impediments, which preclude this accommodation from working as seamlessly as Microsoft Teams or Zoom.

LRP vendors have made good-faith attempts to provide documentation on technical specifications, user-guides, FAQs and system pre-checks. However, these resources are often text-heavy, which make it difficult to locate the most relevant information; candidates are not reading the fine print.

While LRP vendors try to refine the technological efficiency of their platforms and delivery models, the question confronting the sponsor organizations now is how best to put the most pertinent information front and center so the end-user sees and processes it and has an optimal testing experience.

To that end, the I.C.E. Research and Development Committee has commissioned a study to gather information from organizations using LRP regarding communication strategies that best connect with test takers. The research has four aims:

  • Present an overview of LRP candidate experience considerations
  • Identify candidate technology and workspace requirements for LRP 
  • Identify candidate rights and responsibilities for LRP
  • Identify additional information and resources to prepare candidates for test day

The methodology will consist primarily of semi-structured interviews with member organizations. Keep an eye out in February for a call for volunteers for these interviews (estimated 1-hour duration). We welcome and look forward to your participation! Please feel free to contact Kari Hodge at khodge@theabr.org with questions and interest in participating in the interviews.