Chapter 6: Assessment
Traditional and authentic assessment, the characteristics of effective assessments and tests, learner-centered grading, critiques, and oral questioning.
Chapter summary
Assessment is an essential part of teaching and learning, giving the instructor immediate feedback on the quality of instruction and giving the learner practical, specific direction for improvement. The handbook describes two broad categories. Traditional assessment, such as multiple choice and matching tests, has a single correct answer and usually measures the rote and understanding levels of learning. Authentic assessment asks the learner to perform real-world tasks and generate a solution, demonstrating the application and correlation levels of learning against criteria known in advance. Assessments may be formal or informal, and diagnostic, formative, or summative.
An effective assessment is objective, flexible, acceptable, comprehensive, constructive, organized, thoughtful, and specific. It is based on the facts of the performance as they were, not as they could have been, and it avoids halo error, the sympathy or over-identification with a learner that distorts objectivity. It points toward improvement, respects the learner's dignity, and leaves the learner with no doubt about what was done well, what was done poorly, and how to improve. Effective written tests share six characteristics: reliability, validity, usability, objectivity, comprehensiveness, and discrimination. Validity is the most important.
Learner-centered grading, sometimes called a collaborative critique, guides the learner through self-assessment with four steps: replay, reconstruct, reflect, and redirect. The instructor and learner then compare assessments and record progress on rubrics. One rubric covers maneuvers or procedures (describe, explain, practice, perform, not observed) and another covers single-pilot resource management (explain, practice, manage-decide). These grades describe a level of performance rather than prestige, so the learner cannot flunk a lesson, and evaluation should be progressive from lesson to lesson.
To choose an assessment, instructors determine level-of-learning objectives, list indicators of desired behaviors, establish criterion objectives, and develop criterion-referenced test items. Practical tests defined in the ACS and PTS are criterion-referenced, so the instructor should evaluate every task before recommending an applicant. Critiques may be led by the instructor, a learner, small groups, another learner, or the learner alone, or may be written. Oral questioning is the most common form of assessment; good questions are brief, clear, centered on one idea, suited to the learner, and challenging, while puzzle, oversize, toss-up, bewilderment, trick, and irrelevant questions should be avoided.
Main topics
- Traditional versus authentic assessment
- Formal, informal, diagnostic, formative, and summative assessments
- Characteristics of effective assessment
- Characteristics of a good written test
- Learner-centered grading: replay, reconstruct, reflect, redirect
- Rubrics for maneuvers, procedures, and single-pilot resource management
- The four-step process for choosing an assessment method
- Criterion-referenced testing and the ACS/PTS
- Types of critiques
- Oral assessment, effective questions, and questions to avoid
Key terms
- Traditional assessment
- Written testing such as multiple choice, matching, or true/false, with a single correct answer, that usually assesses the rote and understanding levels of learning.
- Authentic assessment
- Assessment that requires the learner to perform real-world tasks and generate a solution, demonstrating the application and correlation levels of learning.
- Rubric
- A guide for scoring performance assessments in a reliable, fair, and valid manner, made up of dimensions for judging performance, a rating scale, and standards of excellence.
- Diagnostic assessment
- An assessment of learner knowledge or skills before a course of instruction begins.
- Formative assessment
- An ungraded assessment, limited to what happened in the lesson, that wraps up the lesson, sets the stage for the next, and tells the instructor what to reinforce.
- Summative assessment
- An assessment used periodically during training, such as a stage check or end-of-course test, to measure how well learning has progressed.
- Halo error
- Sympathy or over-identification with a learner to such a degree that it influences the objectivity of an assessment.
- Reliability
- The degree to which test results are consistent with repeated measurements.
- Validity
- The extent to which a test measures what it is supposed to measure; the most important consideration in test evaluation.
- Objectivity (of a test)
- Singleness of scoring, so that the grader's own knowledge, style, or opinions do not affect the grade awarded.
- Comprehensiveness
- The degree to which a test measures the overall objectives by sampling a representative cross-section of them.
- Discrimination
- The degree to which a test distinguishes differences between learners; useful for academic achievement, but minimum standards matter more for pilot certification.
- Collaborative critique
- A form of learner-centered grading in which open-ended questions (replay, reconstruct, reflect, redirect) guide the learner through a complete self-assessment.
- Criterion-referenced test
- A test that measures performance against set standards; ACS/PTS practical tests are criterion-referenced because all successful applicants must meet the required standards.
- HOTS question
- A question involving why or how that requires the learner to combine factual knowledge with the ability to analyze situations, solve problems, and reach conclusions.
Revision quiz
10 questions. Pick an answer to see whether it is right and why.
This page is a study aid written from Aviation Instructor's Handbook (FAA-H-8083-9B), Chapter 6. The handbook is the authoritative source; read the full chapter in the official FAA PDF.
VectoredOps is not affiliated with the Federal Aviation Administration.