By TJay Gerber, CSM, CSPO, Global Certification Manager, Scrum Alliance®

Many of us are familiar with the standard process used to develop a certification exam: job analysis, test specifications, item writing, item review, and standard setting. The process involves complex meetings and projects with a lot of moving parts.

The question is: Can we make these meetings more efficient and enjoyable? The answer is YES… with Scrum. Scrum is a simple yet incredibly powerful set of principles and practices to help teams deliver in short cycles, enabling fast feedback, continual improvement, and rapid adaptation to change. This enabled Scrum Alliance^® to create highly efficient teams that accomplished more than usual, including writing over 400 items in two days. It also created a positive atmosphere in which SMEs, psychometricians, and staff enjoyed the experience.

This article gives an overview of how Scrum Alliance applied principles, values and techniques associated with Scrum during their foundational certification test development meetings. The end goal was to elevate the value of the certification.

Scrum Values and the Sprint

Meetings involved volunteers from the Scrum Alliance community who were already well versed in Scrum techniques. This allowed us to hit the ground running incorporating the framework. The Scrum Values of Focus, Openness, Respect, Courage, and Commitment were discussed frequently during the meetings so that the subject matter experts (SMEs) adhered to them.

Before any meeting started, the main lead (a.k.a. Product Owner) outlined the vision for the group. This is an important step during any test development meeting to give SMEs a clear understanding of what is expected. For Scrum Alliance’s two main certifications, the Certified ScrumMaster and Certified Scrum Product Owner, the vision was to have four 50-item forms for each of the two certification programs.

The test development meetings were run as one-day Sprints for which the team set the daily Sprint goal. A Sprint, as defined by the Scrum Guide, is where a “useable and potentially releasable product increment is created.” For example, the Sprint goal was to have the basis for a job analysis survey that would support two certification programs. The SMEs understood that at the end of each day, there should be something completed that fell in line with the Sprint goal.

But how would the SMEs accomplish their goal?

Working Agreements and Timeboxes

One component that helped the team accomplish their Sprint goals for each step of the test development process was to have working agreements that the team developed themselves. Scrum asserts that it is easier to get the buy-in from a group when they are the ones who create the rules by self-organizing. The working agreements mostly involved the five Scrum values ( Focus, Openness, Respect, Courage, and Commitment), but they were also decided based on other simple notions such as the following:

• Don’t talk over each other.
• Document significant changes.
• Have a short walking break every two hours.
• Stick to the timebox (a.k.a. fixed maximum length of time).

Each agreed-upon item was written where everyone could see it during the meeting, so they were able to respectfully hold each other accountable.

By utilizing a timebox, groups were able to stay on track. For example, during an Item Review meeting, the team agreed to timebox each question to three minutes. Some items didn’t need much discussion, but others required the full three minutes to decide on what to do. What happened if they didn’t reach a decision? It would go into a “parking lot” to be discussed at the end of the day when there was time.

Sprint Retrospective

One of the tenets of Scrum is to inspect and adapt. There is always an opportunity to improve from one day to the next. Being honest about the work that was completed (or not completed) helps enact positive change. At the end of each meeting, we held a Sprint Retrospective, which is considered one of the most important Scrum events. A facilitator put up four Post-It^® notes on the wall: a smiley face, a frowny face, “Start,” and “Stop.” A variation of this is to write “Liked,” “Learned,” “Lacked,” and “Longed” on four separate areas on the wall. In either case, these four areas enabled the volunteers, staff, and psychometricians to write their thoughts and post them under each category.

Some themes that came out of a Retrospective included:

• Were there opportunities to improve the structure of the meeting?
• Did someone learn something that maybe someone else didn’t really understand?
• Why did we use a particular tool instead of a potentially more productive one?
• Was everyone happy with the lunch?

Everyone’s opinion mattered, which was transparent to the rest of the group. This enabled some less-vocal individuals to truly have their voice heard. The facilitator, or Scrum Master, then read each note aloud and suggested changes for the next day or meeting so that each day was a bit better than the last.

Summary

Scrum has been proven successful, but ultimately, you need to make it work in a way that’s best for your situation. It is called a “framework” for a reason. As Jeff Sutherland, the co-founder of Scrum, once said, “Scrum is easy to learn, but difficult to master.” At the end of the day, Scrum enables you to pick and choose from a variety of tools to help make your own test development efforts a bit more efficient and hopefully enjoyable.

2019 ICE Exchange Preview

Looking to better understand Scrum? Check out the 2019 ICE Exchange presentation on Nov. 20:

Scrum Basics – Learn How to be More Agile, by TJay Gerber, CSM, CSPO