Discipline and Rigidity in Item Writing
Good item writing lives at the intersection of structure and judgment. The guidelines are there for a reason but following them well means more than unquestioning compliance. It requires knowing when a rule serves the candidate versus when an overly literal application of that rule works against them. Let’s explore the tension between discipline and rigidity in item writing to better understand the balance needed to produce the best results.
Following the Rules
There’s a common misconception that mastering item writing is a matter of internalizing a checklist. Once you know the rules, you’ll just keep getting better and the process gets easier, right? In practice, it’s more nuanced than that:
- Item writers must continuously revisit their habits and question their assumptions with each new item
- Good technique is not built once; it’s renewed through deliberate reflection
Many item writers stay on track by leaning heavily on structure. They follow the rules precisely: every stem is positively phrased, every option is homogeneous, and every item has a citation. That discipline is essential. Best practices make clear that consistent application of these standards protects exam validity and fairness to candidates. But when writers follow the letter of a guideline while missing its spirit, rigidity has quietly replaced discipline.
Defining Discipline vs Rigidity in Item Writing
Discipline in item writing is purposeful structure.
It means each item measures a single, clearly defined aspect of the exam blueprint, the key is unambiguously correct and fully defensible, every distractor is plausible but clearly wrong, and every claim is supported by a citable reference. It also means controlling the impulse to write items that are interesting to the writer at the expense of items that are fair to the candidate. Discipline is how item writers honor the stakes of the exam.
Rigidity, in contrast, is structure without purpose.
When a writer applies a rule just because it’s a rule, even if the result is an item that confuses candidates, obscures the tested construct, or fails to reflect the appropriate level of practice, that is rigidity in action. The guidelines warn against trick items precisely because trickiness, whether intentional or accidental, threatens exam validity. A rigidly rule-following item can be just as problematic as a carelessly written one.
Step Back and See the Big Picture
When an item feels awkward or forced, the instinct is often to add more discipline: tighten the language, simplify the stem, trim the options. But sometimes the real solution is greater flexibility: a willingness to step back, reconsider the construct being tested, and approach the item from a different angle.
Stubborn adherence to a rule, even a good one, can sometimes produce items that work against examinees rather than for them.
The goals of item writing best practices are:
- Validity
- Fairness
- Accurate measurement of competence
Writers who keep that purpose in mind are better equipped to make sound judgments when the guidelines don’t quite cover the situation at hand.
Trust But Verify
A useful operating principle for item writers is “trust but verify”:
- Trust that the guidelines reflect accumulated expertise about what makes exam items valid and fair
- Verify, item by item, that what you’ve written achieves what the guidelines are meant to accomplish
Learning From Other Item Writers
An effective way to expand your item writing repertoire is to study how other writers navigate challenging scenarios.
Experienced item writers tend to develop a broader set of strategies precisely because they’ve encountered more situations where the standard approach needed to be adapted. Engaging with their problem-solving approaches gives newer writers more to draw from when facing their own difficult cases.
Consider a scenario many item writers eventually encounter: a clinical vignette that contains important contextual information but is so detailed that it introduces excessive verbiage.
The guidelines make it clear that verbosity threatens clarity and that wordy items may reduce the number of questions that can be included within a fixed testing period, undermining reliability.
But the guidelines also call for real-life scenarios that reflect the practice environment the candidate will encounter. How do you honor both?
Writers who have navigated this tension successfully are often direct about their reasoning:
- They can draw a clear distinction between information that is necessary to answer the question and background details that distract
- If a piece of context doesn’t change what a minimally competent candidate would need to know to select the correct answer, it probably doesn’t belong in the stem
Just as valuable as direct advice is watching how skilled writers handle revision. Observing how an experienced reviewer responds to a weak distractor can reshape a writer’s approach to distractor construction. That kind of observational learning builds the judgment that guidelines alone cannot fully convey.
Don’t Let Who You Think You Are Hold You Back
Identity can also get in the way of growth as a writer. An item writer who sees themselves as rigorous may resist feedback that a technically correct item is still confusing to candidates. A writer who values creativity may push back on guidance that a cleverly constructed stem is tricky rather than challenging. Both instincts can interfere with producing items that accurately measure what they’re supposed to measure. The guidelines exist not to constrain creativity but to channel it productively.
The Problem with Rigidity in Item Writing
When item writers treat the guidelines as a ceiling rather than a foundation, it limits their ability to handle novel cases. An unwillingness to question a standard approach, even when the resulting item is awkward or misleading, can lead to stagnation in both individual development and overall item pool quality.
Excessive rigidity in applying standards can make it harder to:
- Write items: You’re less likely to generate items that deviate even slightly from the familiar template, even when that deviation would produce a better question
- Grow as an item writer: If you feel you cannot experiment within the guidelines, you’ll plateau in your skill set
- Collaborate across fields: Item writing involves working together with reviewers, subject matter experts, and psychometricians who each bring different perspectives to the table
Rigidity in Item Review
Rigidity can also constrain the review process. When reviewers insist on a single acceptable interpretation of every guideline, it limits productive dialogue about how best to handle novel scenarios. Good item review depends on the ability to reason together about edge cases, not just apply a checklist. Writers and reviewers who treat the guidelines as conversation starters rather than conversation enders will produce better outcomes for the exam as a whole.
Writers who treat editorial suggestions as an attack on their item’s integrity, or who resist revisions on the grounds that the original was technically compliant, are harder to work with and less likely to improve. Openness to revision is not a concession of weakness, but rather the hallmark of writers who understand that the exam’s quality matters more than any individual item.
Let Quadterion Help Balance Discipline and Rigidity in Your Item Writing
The guidelines are the foundation. Discipline is how you apply them consistently and flexibility is how you apply them wisely. Quadterion is ready to train your item writers to know the difference and when each is called for.
reach out now
References
National Association of Boards of Pharmacy (NABP). Item Writing Best Practices. NABP; June 2024.
Certification Management Services. Item Writing Guidelines. Certification Management Services; 2006.
https://www.psychologytoday.com/us/blog/in-practice/202508/how-to-become-less-stubborn-and-rigid
https://claresdiary.medium.com/the-fine-line-between-discipline-and-rigidity-d65da01b2310