Evaluation and Measurement in Health Promotion

Preface xiii Instructor Resources xv Acknowledgments xvii About the Authors xix CHAPTER 1 INTRODUCTION TO EVALUATION IN HEALTH EDUCATION AND HEALTH PROMOTION 1 Key Concepts 1 Chapter Objectives 1 The Field of Health Education and Health Promotion 1 Defining Evaluation 3 Evaluation Competencies for Health Educators 4 Historical Milestones in the Evaluation of Health Education and Health Promotion 10 FOCUS FEATURE 1.1 Hypothetical Account of a Health Educator's Evaluation Responsibilities 10 Basic Terms in Evaluation 16 Skill-Building Activity 19 CHAPTER 2 PLANNING EVALUATIONS FOR HEALTH EDUCATION AND HEALTH PROMOTION PROGRAMS 25 Key Concepts 25 Chapter Objectives 25 Evaluation and Continuous Health Program Improvement 25 Logic Model for Health Education and Health Promotion Programs 26 Logic Model and Comprehensive Evaluation 29 Health Program Planning and Evaluation: Integration 30 Political Climate and Evaluation 32 Political Climate and Program Stakeholders 32 Balance Between Evaluation Rigor and Utilization 35 Conducting a Utilization-Focused Evaluation 36 Conducting a Rigorous Evaluation 36 FOCUS FEATURE 2.1 Planning a Health Education/Promotion Program Evaluation 38 Critical Issues for an Evaluation Project 39 Skill-Building Activity 39 CHAPTER 3 BASICS OF MEASUREMENT 45 Key Concepts 45 Chapter Objectives 45 Definition of Measurement 45 Scales or Levels of Measurement 46 Conceptual Paradigm of Measurement 48 Types of Variables in Health Education and Health Promotion 51 FOCUS FEATURE 3.1 Example of a Scale Measuring Knowledge 52 FOCUS FEATURE 3.2 Example of a Scale Measuring Attitudes 54 Skill-Building Activity 59 CHAPTER 4 STEPS IN INSTRUMENT DEVELOPMENT 63 Key Concepts 63 Chapter Objectives 63 Basic Principles of Instrument Development 63 FOCUS FEATURE 4.1 Example of a Theory-Based Scale 71 General Principles of Writing Items 78 Types of Instruments Measuring Knowledge 78 Types of Instruments Measuring Attitudes 81 FOCUS FEATURE 4.2 Rosenberg Self-Esteem Scale 82 Types of Instruments Measuring Health Behavior 85 Physiological and Biomedical Instruments: Outcomes of Behavior 88 Skill-Building Activity 89 CHAPTER 5 RELIABILITY ASSESSMENT 93 Key Concepts 93 Chapter Objectives 93 Definition of Reliability 93 Types of Reliability 96 FOCUS FEATURE 5.1 Using IBM-SPSS to Calculate Reliability Statistics 101 Tips to Increase the Reliability of an Instrument 102 Skill-Building Activity 102 CHAPTER 6 VALIDITY ASSESSMENT 107 Key Concepts 107 Chapter Objectives 107 Definition of Validity 107 Types of Validity 108 Assessing Face and Content Validity 108 FOCUS FEATURE 6.1 Form Letter to Panel of Experts for Face and Content Validation 110 Construct Validity and Factor Analysis 111 Criterion Validity 115 FOCUS FEATURE 6.2 Using IBM-SPSS for Confirmatory Factor Analysis 116 FOCUS FEATURE 6.3 Case Study of Confirmatory Factor Analysis 116 Skill-Building Activity 121 CHAPTER 7 MEASUREMENT ERRORS 127 Key Concepts 127 Chapter Objectives 127 Definition of Measurement Error 127 Classification of Measurement Errors 129 FOCUS FEATURE 7.1 Strategies to Minimize Errors in Self-Report Instruments 136 FOCUS FEATURE 7.2 Strategies to Minimize Errors in Interviews 137 FOCUS FEATURE 7.3 Strategies to Minimize Errors in Observations 138 Skill-Building Activity 138 CHAPTER 8 PROCESS EVALUATION 143 Key Concepts 143 Chapter Objectives 143 Defining Process Evaluation 144 Targets of Process Evaluation 144 Implementation Fidelity 148 FOCUS FEATURE 8.1 Example of Implementation Fidelity Assessment 149 Process Evaluation: Foundational to Impact and Outcome Evaluation 151 Pragmatic Planning for a Process Evaluation 152 Formative Evaluation 152 Skill-Building Activity 155 CHAPTER 9 DESIGNS FOR QUANTITATIVE EVALUATION 159 Key Concepts 159 Chapter Objectives 159 Questions in Evaluation 159 Evaluation Design 160 Components of a Testable Hypothesis 162 Internal Validity 162 External Validity 165 Evaluation Design Notation 169 Types of Designs in Health Promotion and Education 169 Barriers to Randomization 171 FOCUS FEATURE 9.1 Considerations in Choosing a Design 172 Skill-Building Activity 173 CHAPTER 10 APPROACHES TO QUALITATIVE EVALUATION 177 Key Concepts 177 Chapter Objectives 177 Introduction to Qualitative Evaluation and Research 177 Quantitative versus Qualitative Evaluation 179 Empowerment Evaluation 180 Illuminative Evaluation 183 Participatory Evaluation 184 FOCUS FEATURE 10.1 Case Study of Participatory Evaluation in Vietnam 186 CIPP Model of Evaluation 193 Utilization-Focused Evaluation 194 Goal-Free Evaluation 194 Qualitative Data Analysis and Interpretation 195 Skill-Building Activity 197 CHAPTER 11 SAMPLING 203 Key Concepts 203 Chapter Objectives 203 What Is a Sample? 203 Probability Sampling 204 Barriers to Probability Samples in Health Program Evaluations 206 Nonprobability Samples 207 Sample Size: How Many Participants Are Needed for the Evaluation? 208 Response Rates 211 FOCUS FEATURE 11.1 Calculating Sample Size Using G*Power 212 Generalizing Evaluation Results 213 Describing the Evaluation Sample and Population 214 Skill-Building Activity 214 CHAPTER 12 QUANTITATIVE DATA ANALYSIS 219 Key Concepts 219 Chapter Objectives 219 Introduction 219 Data Management 220 Data Cleaning 221 Describing the Sample 223 Data Analysis: Descriptive Methods 224 Frequency Distributions 227 Measures of Central Tendency 227 Measures of Variability 228 Graphing Data 229 Statistical Inference 232 Hypothesis Testing 232 Statistical Tests used in Health Evaluations 234 FOCUS FEATURE 12.1 Using IBM-SPSS for a Paired t-Test 238 FOCUS FEATURE 12.2 Using IBM-SPSS for Repeated Measures ANOVA 238 Using Effect Sizes to Communicate Evaluation Results 239 Skill-Building Activity 241 CHAPTER 13 DATA INTERPRETATION AND REPORT WRITING 247 Key Concepts 247 Chapter Objectives 247 Introduction 247 Interpretation of Evaluation Results: Conceptual Tools 248 Statistical Significance and Practical Importance 248 Potential Failures in Interpretation 251 Evaluation Standards: How Much Is Enough? 256 Effectively Communicating Statistical Results 258 Evaluation Report Writing 259 Sections of the Evaluation Report 260 FOCUS FEATURE 13.1 Example of an Evaluation Report in Health Education 269 Summary 280 Review Question 281 Websites to Explore 281 References 282 Glossary 285 Index 303