Assessment Design Orchestrator

Routes between five assessment pathways — formative, rubric/criteria, authentic/performance, peer/self, and diagnostic — with validity and equity checks. Use when a teacher needs help choosing how to assess.

What it does

Helps teachers navigate the question "how should I assess this?" — a question whose answer depends heavily on what is being assessed, why, and in what context. The orchestrator presents five assessment design pathways, recommends a fit based on the teacher's situation, and coordinates the relevant component skills for the selected pathway.

Use this when the assessment decision is not yet made. If you already know you need a rubric, use criterion-referenced-rubric-generator directly. Use this orchestrator when the type of assessment itself is uncertain.

Do not use this orchestrator before learning goals are clear. Assessment design begins with the learning, not the instrument.

The evidence behind it

The five pathways draw from well-established assessment research traditions. Formative assessment: Black & Wiliam (1998), effect size ~0.66, and Wiliam (2011). Authentic assessment: Wiggins (1998) and Sadler (1989) on criteria and feedback. Validity: Messick (1989) unified validity framework — the principle that assessment must measure what it claims to measure runs through every pathway. Peer and self-assessment: Topping (2009) meta-analysis and Zimmerman (2002) on self-regulated learning. Feedback: Hattie & Timperley (2007).

Sources

• Black & Wiliam (1998) — Inside the Black Box (formative assessment, effect size ~0.66)
• Wiliam (2011) — Embedded Formative Assessment
• Wiggins (1998) — Educative Assessment (authentic assessment design)
• Sadler (1989) — Formative assessment and the design of instructional systems
• Hattie & Timperley (2007) — The power of feedback
• Messick (1989) — Validity (unified validity framework)
• Topping (2009) — Peer assessment (meta-analysis)
• Zimmerman (2002) — Self-regulated learning and self-assessment
• Manning (2023-2026) — REAL School rubric logic and competency assessment (practitioner framework)

How to use it in your lesson

For the best results with EvidenceLesson, give it:

• learning_goal — What students are expected to know, understand, or be able to do
• context — Subject, year group, unit, or programme context
• assessment_purpose — Why you are assessing: to inform teaching, grade, certify, give feedback, build student capacity, or understand prior knowledge
• learner_variability (optional) — Known diversity: EAL students, students with learning differences, mixed prior knowledge, etc.
• time_in_unit (optional) — Beginning, middle, or end of unit — or ongoing
• constraints (optional) — Time, moderation requirements, reporting formats, school policy, or technology access
• pathway_choice (optional) — Optional selected pathway. If absent, present options before designing the full assessment.

Known limitations

1. Cannot evaluate task difficulty for a specific group. This orchestrator designs assessment structures but cannot determine whether tasks are appropriately challenging for specific students without knowing those students.
2. Validity is context-dependent. A valid assessment in one context may be invalid in another. Validity is a property of assessment use, not a fixed property of the instrument (Messick, 1989).
3. Cannot ensure consistent teacher judgement. The skill designs rubrics and criteria but cannot replace moderation, calibration, and professional judgement. Consistent application of rubrics requires human collaboration.
4. Authentic assessment is not always better. Some learning goals are best assessed through well-designed tests. This orchestrator should help teachers choose the right fit, not default to performance assessment because it feels more progressive.
5. Equity check flags barriers; it does not resolve them. Identifying format barriers is not the same as removing them. Barrier removal may require specialist knowledge or skills not in this library.

Before you deliver: a quick check

1. Designing assessment before learning goals are clear. Assessment must follow from learning, not precede it.
2. Confusing authentic with valid. An authentic task is not automatically a valid measure of learning.
3. Building rubrics without exemplars. Rubrics without calibrated exemplars produce inconsistent teacher judgement and confuse students.
4. Using peer assessment without training. Untrained peer feedback is often vague, positively biased, or harmful.
5. Ignoring format barriers. A well-designed assessment that is inaccessible to some students due to unrelated format demands is not a valid assessment for those students.
6. Using every tool. Select the pathway that fits the purpose. Not every assessment needs formative, summative, and self-assessment components.

Pairs well with