To SME or Not to SME — There Is No Question

Subject matter experts. SMEs. Those people who make the credentialing world go ‘round. They make up our committees, our boards and (sometimes) our nightmares. This article focuses on how to get (beg) SMEs, manage (herd) SMEs and retain (beg again) SMEs. I hope you find some of our tips and tricks useful as well as easily implemented.

There are essentially two layers of people involved in credentialing: The credentialing professionals — that's you and me — and then the industry experts. One group (us) knows how to build, implement, maintain and grow a credential program. The other group (SMEs) brings the specific know-how for the job role(s) on which we are focused. I don’t know about you, but I know nothing about installing commercial glass, training service animals, sweeping chimneys and diagnosing cancer — yes, these are just a few examples of the job roles with which we work! —  so, retaining and keeping the experts happy is essential to our programs’ success.

Why Do We Need Them? 

Developing a test that will yield valid and useful results necessitates the input of a variety of SMEs within the industry of interest. We cannot have just one expert’s opinion driving the test content, much less the actual definition of the job role itself. I have yet to come across a job role done the same way across geographic areas, organizations, subsectors and regulatory zones. While one credentialing program cannot be expected to cover — in depth — ALL the ways in which a job can be done, it is essential that the program at least acknowledges and accounts for the differences within the industry.  

Further, if you cannot gather a good variety of SMEs to define the job role and create the test content, you run the very real risk of developing a program that is biased toward the subsector / subsample of candidates that has the most in common with the SMEs who DID do the test development. “That is not how we do it,” and “that is an acceptable method, but this way is also good,” are common phrases we hear during the job task analysis (JTA) phase. Be very wary of the tendency to get only retired — a.k.a. “mature” — experts. They may have the most time on their hands, but do they have a comprehensive view of the current job role and the near-future trends of the industry? 

How Do We Get Them? 

There are two approaches here: 

1. Recruitment: This may involve begging. This may involve negotiation. It may even involve payment or compensation (if you are lucky enough to have such resources available to you as the credentialing body). Framing the SME experience in a positive light, as well as one that can benefit not only the credential program but the individual and the industry as well, is an important approach here.  
2. Accept applications or nominations: It’s all in the presentation! If you have the industry population to support this approach, this is a fantastic one. You can present the opportunity to sit on the JTA Committee, Exam Development Panel or Certification Board as one of great honor and essential to professional development. I have templates for SME Applications and Nominations, and love helping our clients design SME “badges” for their email signatures and LinkedIn profiles. Our clients have noticed a positive uptick in ease-of-recruitment as well as retention when using these tools. In addition to offering CEUs for regular participation, I have also had clients offer SMEs reference letters that detail (brag about) their time and dedication to the credentialing cause. 

Either way, please be aware of sample bias here: Whether you are paying them or asking them to volunteer their time, some experts will just never want to do it — and those people may be the best ones! If you have specific experts in mind (either due to their expertise, reputation or for political reasons), then a targeted focus on recruitment is great. In that same vein, as you move through your development or management process of your program, you will learn who is a great SME and who is not.  

What We Do Once We Get Them

Training, handholding, herding — all the above. The “training” piece may seem counter intuitive. They are already experts (and they know this), so why could they possibly need training? This is a large hurdle to jump with your new group of friends: Convincing them that, although they are experts in their field, they are not experts in credential development and test design. Unless you are working on a credential program for psychometricians, in which case, there is no hope for you. May the odds be ever in your favor.

Back to “normal” SMEs: Train them on the process of building a credential. What are the phases and how do they fit into the entire development cycle? Why are these steps important? Getting their buy-in at the program level is essential, without making it boring or too academic.  

Once they understand why they are doing these things (such as a JTA or an Angoff process), they are more likely to commit and to follow through. Then the management (a.k.a. “herding”) comes into play: little nudges, friendly chats, standing office hours — anything to keep them engaged and invested. We assign a “handler” to each SME, and this staff member is now their go-to for all things credentialing. If they don’t have an answer, they will find one for them or point them in the right direction. This relationship pays off in the long-run, as that SME is more likely to go the extra mile — to write that extra test question — for their credentialing colleague.  

This relationship also allows us to get to know them: Which SME is good at writing items? Which one should never be allowed to write items — but is a great reviewer? Some are super responsive for a quick turnaround, and some do great work but need to be slow and steady in their progress. Being able to advise a client on which SME to reach out to about a quick assignment is invaluable as a service and as a building block for a successful program.  

Final Piece of Advice

How you manage SMEs can greatly increase the success (or the headaches) of your program. Get to know them, so you can not only maximize their individual strengths but can also account for their weaknesses. SMEs can be your “credential ambassadors,” as we call them, and will provide free publicity for your program. They will tell others how great (or bad) the program is, and it’s important to remember that what you say and do will be advertised, whether you want it to be or not.