Job Description

Psychometrician

Job Type: Full-Time 
Location: Toronto, ON 
Work Model: Hybrid (2–3 days in office) 
Salary: $100,000 – $115,000 per year 

Job Summary

Reporting to the Manager, Psychometrics, the Psychometrician supports the quality, fairness, and defensibility of national certification examinations by conducting psychometric and statistical analyses across multiple examination programs. The role ensures that examination design, scoring, and monitoring activities are supported by sound quantitative evidence and aligned with established psychometric and regulatory standards.

The Psychometrician contributes to the integrity and continuous improvement of assessment programs by supporting research, data analysis, reporting, and examination quality assurance activities. The position operates within established psychometric methodologies under the guidance of the Manager, Psychometrics, exercising professional judgment and escalating high-stakes or policy-level matters for review.

Key Responsibilities

Psychometric Analysis and Examination Quality

• Perform psychometric analyses including item and test analyses, scaling, equating, and quality assurance to maintain the validity and reliability of certification examinations.

• Monitor examination performance across administrations, identifying trends, anomalies, or risks affecting examination integrity.

• Apply established psychometric methodologies consistently and document analytical assumptions, methods, and outcomes.

Test Development and Assessment Support

• Collaborate with psychometric, data, and examination development teams to support test design, blueprinting, item review, standard setting, and post-examination evaluation processes.

• Provide psychometric input to inform assessment decisions within approved frameworks and governance structures.

• Support the defensibility of examination outcomes through appropriate analysis and documentation.

Data Interpretation, Reporting and Communication

• Interpret and summarize psychometric findings into clear, accurate reports for internal leadership, committees, and governance purposes.

• Support preparation of briefing materials, presentations, and summaries used for internal reporting or formal review processes.

• Explain statistical findings to non-technical stakeholders to support informed decision-making.

Training, Guidance and Collaboration

• Train and assist examination development teams on psychometric and assessment processes including test design principles, item analysis, standard setting, and quality assurance practices.

• Provide guidance on the appropriate application of psychometric concepts within the assessment environment.

• Work collaboratively with internal teams and external consultants as required.

Research and Continuous Improvement

• Contribute to research, innovation, and continuous improvement initiatives related to assessment design and psychometric methodology.

• Stay current with developments in psychometrics, educational measurement, and professional licensure assessment standards.

• Exercise professional judgment in identifying issues requiring escalation to the Manager, Psychometrics.

Education & Experience

• PhD in educational measurement, psychometrics, or a closely related quantitative discipline.

• 1–3 years of experience conducting psychometric or statistical analysis for educational or professional licensure assessments.

• Recent graduates with directly relevant applied research experience may be considered.

• Demonstrated knowledge of psychometric methodologies including Classical Test Theory (CTT), Item Response Theory (IRT), and Generalizability Theory.

• Experience designing, executing, and documenting analyses related to item performance, test functioning, reliability, validity, and examination monitoring.

• Proficiency with statistical and psychometric software (e.g., R, Python, SPSS, WINSTEPS or equivalent).

• Proficiency with Microsoft Office tools for reporting and documentation.

• Ability to clearly communicate technical statistical findings to non-technical audiences.
  
  **Staffworks may use AI-enabled tools to support recruitment activities while respecting privacy and employment regulations.**