Standard Setting

SECTION I: FUNDAMENTALS OF STANDARD SETTING 1. Contemporary Standard Setting: An Enduring Need The Need to Make Decisions The Benefits of Standard Setting General Approaches to Standard Setting Current Contexts for Standard Setting 2. What is Standard Setting? Kinds of Standards Definitions of Standard Setting Policy Issues and Standard Setting Item Scoring Criteria and Total-Test Performance Standards Conclusions 3. Common Elements in Setting Preformance Standards Purpose Choosing a Standard Setting Method Performance Level Labels Performance Level Descriptions Key Conceptualizations Selecting and Training Participants Evaluating Standard Setting Providing Feedback to Participants Professional Guidelines for Standard Setting Evaluating Standard Setting Conclusions SECTION II: STANDARD SETTING METHODS 4. The Nedelsky Method Procedures for the Nedelsky Method Alternative Procedures and Limitations 5. The Ebel Method Procedures for the Ebel Method Alternative Procedures and Limitations 6. The Angoff Method and Angoff Variations Procedures for the Angoff Method Procedures for Angoff Variations Alternative Procedures and Limitations 7. The Direct Consensus Method Procedures for the Direct Consensus Method Alternative Procedures and Limitations 8. The Contrasting Groups and Borderline Group Methods Procedures for the Contrasting Groups Method An Example Using the Contrasting Groups Method The Borderline Group Method Alternative Procedures and Limitations 9. The Body of Work and Other Holistic Methods The Judgmental Policy Capturing Method The Dominant Profile Method The Analytic Judgment Method Summary Analysis of Three Holistic Methods Overview of the Body of Work Method Procedures for the Body of Work Method An Application the Body of Work Method Alternative Procedures and Limitations 10. The Bookmark Method Overview of the Bookmark Method An Implementation of the Bookmark Method Alternative Procedures and Limitations 11. The Item-Descriptor Matching Method Procedures for the IDM Method Alternative Procedures and Limitations 12. The Hofstee and Beuk Methods The Hofstee Method Procedures for Implementing the Hofstee Method The Beuk Method Procedures for Implementing the Beuk Method Alternative Procedures and Limitations SECTION III: CHALLENGES AND FUTURE DIRECTIONS IN STANDARD SETTING 13. Scheduling Standard Setting Activities Scheduling Standard Setting for Educational Assessments Scheduling Standard Setting for Credentialing Programs Conclusions and Recommendations 14. Vertically-Moderated Standard Setting The Interrelated Challenges A Brief History of Vertically-Moderated Standard Setting What Is Vertically-Moderated Standard Setting? Approaches to VMSS Applications of VMSS An Illustration of VMSS Procedures Alternative Procedures and Limitations 15. Standard Setting on Alternate Assessments The Unique Challenges of Alternate Assessment Necessary Conditions for Alternate Assessment Systems A Generalized Holistic (GH) Method Limitations and Alternative Procedures 16. Special Topics and Next Steps Rounding Methods of Adjusting Cut Scores Deciding How to Incorporate Uncertainty Generalizability of Standards Using Multiple Methods of Standard Setting Improving Participant Training