When The Needs Assessment for a New Credential Comes Back a Resounding ‘No!’

An Interview With Torryn P. Brazell, former managing director of the American Board of Audiology 

The American Board of Audiology (ABA) was formed in 1998. Since its inception, the Board Certified in Audiology credential has not been an exam-based credential but, over the years, the ABA Board of Governors has often wondered, “Should we develop a new credential, one with an exam component?” In 2010, a credentialing vendor was engaged to conduct a needs assessment to answer this question, but the results were inconclusive. A volunteer task force was convened in 2012 to explore the issue, but their work was likewise inconclusive. In 2014, the board decided to once again address this perennial topic, but with a wider stakeholder focus. They contracted with Professional Examination Service (PES) to determine whether a new credential was desired by the audiology profession. 

Torryn P. Brazell, MS, CAE, CMP, then managing director of the American Board of Audiology, and Patricia Muenzen, MA, of Professional Examination Service, presented the session “When the Needs Assessment for a New Credential Comes Back a Resounding ‘NO!’” at the 2016 ICE Exchange. Brazell is now the executive director and chief operating officer at the American Tinnitus Association. ICE Digest’s Joyanna Mills caught up with Brazell to discuss this topic. 

Please briefly describe the needs assessment. 

The effort began with the formation of a six-member advisory committee, which included one board member. One of the committee’s tasks was to develop a straw description of the proposed certification. The credential they designed provided specific and tangible policies, such as eligibility and maintenance requirements, for the stakeholders giving feedback. The assessment sought to learn who would want the credential and when, and what the advantages or disadvantages might be for the ABA, the profession of audiology and its practitioners, and the public. The methodology was three-pronged, including focus groups, interviews of key opinion leaders and a survey of the profession.

What stakeholder groups were represented? 

There wasn’t a rock that PES didn’t look under for stakeholders, and all were included. Members of the advisory committee represented a variety of groups including students, academicians, practitioners and industry. The focus groups were composed of those who had recently graduated with their AuD (doctor of audiology) degree, preceptors (supervisors) of audiology students during their year-long externships and future leaders of the profession. The opinion leaders who were interviewed represented a variety of work settings, including those in industry, private practice, hospitals and academia. Survey respondents included members and nonmembers of a professional association and sister organization, the American Academy of Audiology, and a broad array of work settings were represented by the survey takers: private practice, hospitals, clinics, physicians’ offices, the military, Veteran’s Affairs, corporate group practice, manufacturers, schools, and colleges or universities. 

What was the response rate for the survey? 

An astonishing 24 percent completed the survey. 

Were there incentives given for participation in the survey? 

Yes. Beta testing revealed that completion of the survey would require between 35 and 50 minutes, but we knew we needed many respondents for valid results, so we had a blind drawing to give away free audiology conference registrations and substantial Amazon gift cards. 

When the survey results were in, we published a paragraph-long communication to thank the participants and report the outcome. Though the results were not positive, we did not apologize or make a big splash, but we did close the loop with respondents in this way. 

What were the findings, and how were they presented to the board? 

The answer to the research question “Should we develop a new credential, one with an exam component?” was a resounding “no.” When those surveyed were asked if they would seek the proposed credential, 23 percent responded “definitely no,” 32 percent responded “likely no,” 14 percent responded “don’t know,” 24 percent responded “likely yes” and 7 percent responded “definitely yes.” 

To cushion the blow, the board was first given a short summary of the findings. At the very end of this summary was one sentence that stated that the analysis did not support development of the credential at this time. The full detailed results report from the credentialing vendor was then provided two weeks in advance of a board meeting at which the results would be discussed. The report was 100-150 pages long and the board members, who were very meticulous, read it all. 

Though I was the managing director, I knew I did not want to be the messenger to my board. They are passionate audiologists. Every single one of them had wanted this new credential for years. Patricia (Pat) Muenzen, Ph.D., was the facilitator and director of research for PES on this project. I asked her to walk through the results during the board meeting. What was supposed to be a 15-minute call was about an hour. And it was great because Pat was able to answer their questions clearly, in layman’s terms.  

In my career, I’ve found that board members can view an issue with their hearts, and that’s good. That’s not a bad thing. But you’ve got to temper it, back it up, with data. Focus was placed on the high participation rate and on comments that supported the conclusion. The board was disappointed, but at the same time they were able to say, “You know what? We did our best. We did it in the best way we could, and therefore now we’re moving on.” 

It is critical to constantly educate a board in best practices, credentialing requirements and the importance of aligning a program to accreditation standards. It was also explained to the board that environments can change and that the feasibility of a new credential can be revisited in five years if it is determined that the audiology landscape is significantly different. 

What was the time frame of the needs assessment? 

From beginning to end, from gathering vendor proposals to presenting the results to the board, the process took around 18 months. 

How did this project fit into the ABA’s overall strategic goals? 

This needs assessment helped the ABA to shift its focus away from the development of new (and costly) credentials to training in the form of assessment-based certificates. What we discovered is that audiologists were not asking for more credentials but for deep training in specific clinical areas. So after further research, we built a preceptor training certificate program for those supervising AuD students during their externships. We launched it, and it went just gangbusters. And another certificate, for tinnitus management, is now being developed. 

For anyone considering whether they should develop a new credential, what is the most salient lesson from this experience? 

Just because you think building a new credential is a great idea, and you sit around the bar where everybody agrees, “Yeah, we should!”, it is absolutely false that “if you build it, they will come.” And you have to educate your board and other stakeholders about this. You also have to look at sustainability. Let’s say you receive positive results from a needs assessment. Why spend a quarter of a million dollars doing a practice analysis and creating an exam and building the whole infrastructure if it’s going to fail in three to five years? A needs assessment is worth the investment regardless of its outcome