Artificial intelligence (AI) is transforming the way we work, and the credentialing industry is no exception. In a recent I.C.E. webinar, “The Power of AI in the Everyday Life of a Credentialing Professional,” industry leaders Liberty Munson, director of psychometrics for Microsoft Worldwide Learning organization, and Dianne Henderson, vice president of measurement innovation at ACT, shared their insights on how AI is being leveraged in their daily practice as credentialing professionals. In this article, we’ll summarize that conversation as well as provide advice for how credentialing professionals can incorporate generative AI into their daily work to make themselves more efficient and effective in their roles.

During the webinar, Munson shared examples of how she uses AI to summarize long emails and documents, as well as the day’s team chats. She also uses it to take meeting notes, relying on its ability to recap meetings into a few key bullet points, identify action items and note where her name was mentioned. By leveraging AI in this way, she saves time, with the ability to be more strategic in the meetings that she attends — she also doesn’t have to worry about missing a meeting due to a conflict.

In addition to these summary capabilities, Munson also uses AI to generate responses to emails, draft PowerPoint decks and documents and identify possible action items that she may have overlooked in chats, emails or meetings. Leveraging these tools to generate questions for panel discussions at conferences is another time-saving feature she has utilized, streamlining brainstorming and allowing panelists to focus on their responses.

Henderson shared how AI is being used at various organizations to generate items, score open ended essays and items and create ancillary products such as test prep materials. While automated item generation (AIG) has been used successfully in several different applications for the past several years, the introduction of large language models (LLMs) for item generation ranges from a number of organizations implementing pilot programs to others using AI in operational programs. Henderson also mentioned more recent advances in AI tools, which now allow for the easy creation and editing of video scripts — a process that previously would have taken several days and been high cost, but now can be completed by one team member in one to two hours.

An example of how AI has transformed test development was recently presented by Tyler Tu and Jim Mendes from Adobe in a recent webinar (February 2024). They described how the implementation of AI optimized time to market for exam releases, achieved 50% reduction in item development time and 50% or more in savings per item. Another company successfully using AI is Duolingo, who is leveraging technology to produce new lessons and materials faster. As pioneers in the use of AI, Duolingo has published a Theoretical Framework for Digital-First Assessment (Burstein, et al, 2022) and the Responsible AI Standards (Burstein, 2024).

In the webinar, Henderson and Munson also described how their programs are using generative AI to develop exam items. For both, this has been a long journey of trial and error as they work to craft prompts in a way that creates items that align to their program’s unique item writing standards. The key to any use of generative AI is writing an effective prompt and that applies to generating items.

This led to a discussion on the importance of the prompt in leveraging generative AI to be more efficient — which is very much an art and takes practice. If the generative AI solution has built-in prompts, use them as a starting point for learning how to create new, effective versions. Edit these prompts and experiment. Figure out what works to get the output that you want. It takes practice and patience, and you may experience some frustration as you learn. But the effort is worth it in the efficiencies that will follow as you develop the habit of using AI in your work.

It is important to ensure data privacy and security and to mitigate bias and ensure fairness, even as credentialing professionals use AI in their daily work. Creating clear, responsible AI principles (sometimes called responsible use policies) and operationalizing them are critical to the successful use of AI in your work. In some cases, organizations are opting to use closed systems that restrict access to specific training data (Retrieval-Augmented Generation (RAG)) and do not allow access to the internet. In all cases the recommendation is to have a human in the loop — always having subject matter experts (SMEs) as part of the review and finalization process.

In conclusion, AI has the potential to revolutionize the field of credentialing, not only in how we design, develop, deliver and maintain our assessment content, but in allowing us to be more efficient and effective in the work we do every day as credentialing professionals. Start experimenting with AI and see how it can help you be more efficient at work. Just remember to use the right tool for the right purpose and always include SMEs in the process.

A Peek Behind the Curtain

As an example of the power of AI in becoming more efficient, the authors used the generative AI system, Microsoft Copilot, to generate a draft of this article based on the webinar transcript, using this prompt:

Prompt: I am writing an article based on this document [the I.C.E. Credentialing Insights Content Contribution Guidelines]. Articles submitted to Credentialing Insights should be professional in tone, with the goal of informing the audience. Readers should gain new insight or practical knowledge to apply in their own work after reading the article. The article cannot exceed 1,000 words. Create a draft of this article for me.

Linked here is the first draft, copied directly from the AI solution the authors used.

And, for a bit of fun, we also asked Copilot to generate a draft using a slightly different prompt that would create a more humorous article. The text linked here has not been edited to illustrate the power of the AI to summarize text and the wording of the prompt.

Prompt: I need to write an article for Credentialing Insights summarizing this document and highlighting key tips for people using AI to be more efficient at work. The article should be professional in tone but with humor and fun. It cannot exceed 1000 words.