Features

A Look Inside the NCCA’s Assessment of Live Remote Proctoring

The National Commission for Certifying Agencies (NCCA) recently created a subcommittee to oversee an assessment of live remote proctoring. ICE spoke with subcommittee member Linda Waters, PhD, to learn more about this assessment and what the NCCA hopes to achieve with its outcome.

Linda Waters is a vice president at Prometric, and in her sixth year as a commissioner for the National Commission for Certifying Agencies (NCCA). Along with Commissioners William Ellis, executive director of the Board of Pharmacy Specialties, Joy Matthews-Lopez, president of JML Measurement & Testing Services, LLC, and Public Member Michele Warholic, she is part of a subcommittee that is overseeing an assessment of live remote proctoring on behalf of the NCCA. 

The NCCA defines live remote proctoring as “remote proctoring that occurs with a person actively watching a candidate during the time of the test administration and that provides safeguards for exam integrity and validity similar to in-person proctoring.” 

We spoke with Linda to learn more about this assessment (also referred to as the pilot assessment) and what it means for the credentialing community.

Why did the NCCA decide to undertake an assessment of live remote proctoring?

The credentialing industry is constantly evolving, in response to transformations in technology and assessments. In response to inquiries from accredited programs regarding the use of remote proctoring as a delivery mode, the NCCA believed it was incumbent to conduct a small-scale study to determine whether testing with live remote proctoring yielded results that would meet the NCCA Standards, particularly in response to comparability of exam performance and test security. There is limited professional literature about remote proctoring that can be used for benchmarking.

What is the outcome that the NCCA hopes to achieve?

The NCCA’s goal is to use both quantitative and qualitative data collected directly from the administration of exams via live remote proctoring to determine whether live remote proctoring produces outcomes that meet the NCCA Standards. 

Please describe how the assessment is being conducted and evaluated.

All programs enrolled in the assessment are already NCCA accredited programs. The NCCA believes that this requirement provides a level of control over extraneous factors, since each of these programs has successfully achieved accreditation by meeting the NCCA Standards. Several communications about the planned assessment of live remote proctoring were shared with accredited programs and the overall ICE community in the fall of 2019, encouraging accredited programs to participate in the assessment. An information session was held at the ICE Exchange in November of 2019 as well. Data collection by the participants began in January 2020 and will conclude this summer. The participants will complete a self-study using the NCCA Standards in which data obtained from live remote proctoring will be compared to data from other delivery modes. The goal is to have the data analyzed and findings reported by the fall of 2020.

In what ways does the Commission expect to share its findings?

Once the findings are compiled, the NCCA Commission plans to share the results with the ICE community via a written report and a session at the 2020 ICE Exchange.

Is there anything else that you would like to share?

The NCCA Commission recognizes that to meet the evolving needs of the credentialing industry, the NCCA Standards must be dynamic in evaluating how assessments are developed and administered by sponsoring organizations. The NCCA Standards are not prescriptive and the choice of how to meet the Standards is always left to the discretion of the sponsoring organization. At the same time, NCCA accredited programs want to protect the integrity of their exams and the value of their intellectual property. Live remote proctoring offers an innovative way to deliver exams that deserves careful study with actual comparative examination data.  

 Editor’s note: The pilot assessment referred to in this article is not to be confused with the NCCA Exception program for live remote proctoring, which was implemented in response to COVID-19.

Additional Resource

Considering remote proctoring and want to learn more? View our webinar, Remote Proctoring–Key questions for a timely and careful decision, where panelists discuss benefits and challenges to remote proctored exam administration, lessons learned and concerns moving into the future.