When Do You Need a Psychometrician?
Whether you’ve been in the credentialing space for decades or are just getting a new program off the ground, you may find yourself asking, “When should we bring in a psychometrician?”
Read on to see several scenarios where a Quadterion psychometrician can help, no matter where you are in the process.
Scenario #1: Your organization is considering launching a credentialing program, but you don’t know where to start. You need help defining the scope of competency needed to achieve the credential.
Working with a psychometrician right off the bat as you’re developing a certification program can set you up for success. Based on your description of the credential’s objectives and value, a psychometrician can help you define whether the program would be strongest as an assessment-based certificate or a complete certification, and whether the certification is role-based or skills-based.
Psychometricians can also facilitate strategic objectives that are indirectly related to your exam program, such as volunteer workflows and committees, firewall processes, and accreditation requirements.
Now is also a great time to get feedback on whether your competency objectives are appropriately measurable and transparent for prospective certificants. Measurable competencies must be explicitly defined so that successful certificants can demonstrate mastery of the real-world knowledge, skills, and abilities (KSAs) required to safely and capably perform the specified work.
Not sure if you’re ready to start? Let’s talk about where you are and where you’re headed.
Scenario #2: You have the foundation for a credentialing program in place, but you need to develop an assessment to verify future credential holders.
Now that you’ve laid all the necessary groundwork, you’re ready to start the exam development process. This is when a psychometrician usually comes in to guide the essential first step: your job task analysis (JTA), also called a job analysis or practice analysis.
The JTA requires input from a panel of Subject Matter Experts (SMEs) that your organization identifies to lend their real-world expertise on which KSAs are required to perform the job effectively. You will also need to survey a larger pool of certified professionals to generalize the test content outlined in the SME panel.
The outcome of the JTA process is an exam blueprint that lays out the structure and content of your test. Additional elements of the blueprint may include item classifications such as cognitive level, item types, and total number of items, as appropriate for your program.
Our psychometricians will help you:
- Gather essential background information
- Prepare for and lead your SME panel meetings
- Build and disseminate your validation survey
- Analyze the survey data and provide recommended test weights for each content area
- Prepare your exam blueprint and a test specifications document that outlines major processes, functions, and aspects of the exam for ongoing and future use
- Document the process to effectively communicate with various stakeholders, including accreditation bodies
The JTA is the first step toward developing a certification exam, and we’d love to get your assessment off to a good start. Contact us to get started.
Scenario #3: Your credentialing program is already well-established. However, it’s been years since your exam was updated.
Quadterion has worked with numerous credentialing organizations to update their existing examinations. Some issues that a well-established program needs psychometric support for include:
- A previous JTA wasn’t up to the proper standards
- It’s been longer than 5 years since the exam was updated
- The job or work performed has changed substantially
These concerns can be addressed by conducting a new JTA to produce an updated exam blueprint. Following this process, you’re ready to begin developing items (exam questions).
Our psychometricians and exam development experts provide guided item writer workshops and meetings for training and item writing facilitation. We train SMEs to develop and write valid, reliable items that provide a strong, foundational exam.
We’re ready to support you by:
- Developing workflows and processes to meet the demands of your assessment
- Creating training materials to guide SMEs in the item writing process
- Facilitating meetings to guide your staff and SMEs in the best practices of content development
- Analyzing items following exam administration to meet industry standards
Scenario #4: Your program is mature and running smoothly, but you need ongoing psychometric support to keep it that way.
Credentialing programs benefit from long-term psychometric support even when nothing is broken. For example, if questions arise about validity or fairness during the program lifecycle, our psychometricians can provide analysis and reporting to support your validity argument. In addition, we can help identify the best path forward if you are considering a specialty certification or other growth opportunities.
Our ongoing services include:
- Navigating accreditation standards
- Annual exam analysis
- Ongoing form assembly and item banking
- Periodic standard setting
- General psychometric consultation
It’s never a bad time to get a psychometrician involved in your credentialing program.
Ready to learn how Quadterion’s psychometricians can get your program in shape? Drop us a line!
Related Quadterion content: