Diversity Really Does Matter: 6 Tips for Creating Work Groups

When it comes to building the right work groups for writing exam questions, diversity really does matter. Teams with a variety of experience and background are not just smarter, but more effective. Learn how you can build more diverse work groups in this article.

Wednesday, April 24, 2019

By Guillermo Ortiz de Zarate

Diversity and inclusion are hot topics in today’s workplace. Inclusion is especially important in the assessment industry where a deeper level of collaboration and creativity can be achieved that yields more strongly defensible exams. According to a paper titled, “Five of the Most Frequent Problem Areas with NCAA Applications,” by the Institute for Credentialing Excellence (ICE), certification programs should use qualified subject-matter experts (SMEs) to guide examination development activities. Further, “each panel must represent the relevant characteristics of the population…and the process of recruitment and involvement of SMEs must prevent the undue or disproportionate influence of any individual or group.”

Why is Diversity Important?

Diversity means different, or variety. The primary reason for creating more inclusive work groups is because they produce more innovative results. A study conducted by the Victorian Equal Opportunity and Human Rights Commission, in partnership with Deloitte, showed innovation levels can increase by as much as 80 percent when diversity is recognized. In addition, according to a research article in the American Sociological Review, racial and gender diversity is associated with increased sales revenue, more customers, and greater relative profits. An article from The Journal of Product Innovation Management also notes management teams that exhibit a wider range of educational and work backgrounds produce more innovative products.

There is no right or wrong way to create a more diverse work group. What’s important is that organizational leaders think beyond traditional traits of race and gender. Think deeper about what will make teams really gel. What traits, qualifications and expertise will stimulate discussion? What attributes are needed to provide a well-rounded perspective? What work styles will complement each other? These individuals are tasked with an important function. It’s important they work well together and can openly discuss biases and question assumptions so all voices are included.

This is great in theory because test publishers need to make sure their SMEs aren’t overused and they represent the different demographics of the profession they’re certifying. How can this be practically done?

Tips for Creating Work Groups

Here are six tips for building the most desirable teams:

	1. Define business need, goals and metrics. Identifying the business need for each work group and setting common goals is important because this will give SMEs clear focus and direction, inspire them to greater performance, foster creativity and innovation, and encourage them to communicate problems and results. Clearly state what the group should achieve and set SMART (specific, measurable, attainable, relevant, and time-related) goals to get there. There are already many SMART metrics requested from some workshop groups. Track progress and think about how to measure performance of the team as a whole and its individual members. Explore using SMART metrics, such as question survival rate, to measure quality of items produced.
	2. Design a topology for the desirable work group. Define a structure for each work group and identify the different positions needed for each examination activity, such as team leader, psychometrician or other SME. This sets a framework for whether the team will achieve quality results. Think through how the team should work and break down the group’s goals into actionable steps for each position so everyone has a clear understanding of what’s expected.
	3. Define the skills, traits and attributes needed for each position. What demographic, cultural or job-related traits are needed in the work group to represent the profession for which the examination will be designed? Define diverse set of traits such as expertise, experience, gender, race, ethnicity, region, language, behavior differences and generation. Document and store information about these traits in a way that allows one to easily filter on individual characteristics and fill positions readily.
	4. Search the SME pool and pick a strong combination of talent. Once a robust set of traits is defined for the SME pool, filter on each trait needed to find the right person for the right position to create a strong work group. There are many tools readily available to test publishers to support every step of the exam development and delivery process, including SME management. Explore the use of an online tool that eliminates the burden of managing the talent pool in a spreadsheet because filtering information about SMEs could be made easier. Ensure this tool integrates with the other vendors involved in the exam process.
	5. Measure and evaluate team performance. There are lots of ways to measure team performance; for example, 360-degree evaluations are a great option. Evaluations are very effective in tracking individual SME performance, documenting strengths and weaknesses and identifying emerging leaders. As evaluations are developed, consider using a starred rating system that allows team members to rate each other on qualities relevant to their particular work group. Feedback on general metrics such as attendance, helpfulness, efficiency, initiative and quality is also informative and helps tell a broader performance story. Also, consider comparing historic evaluations to gauge whether the mix of diverse characteristics is increasing performance.
	6. Learn and start over. Are team members happy? Understand how each team and individual SME has performed. Develop ways to capture and analyze data about the teams so trends can be identified. Make informed improvements that will ensure the right SMEs are placed in the right work groups every time.

Summary

What does this mean? Diversity really does matter when it comes to building the right work groups for writing exam questions. Teams comprised of SMEs who represent the nationalities, cultures, ethnicities, geographies and experience of their exam takers are more likely to write top-quality items that successfully measure achievement and enhance validity of the assessment. This results in innovative results and more strongly defensible exams.