Flashcards
Program Evaluation Exam 1
Cards in this deck (71)
- 1Utilization focused evaluation (utility focused)Not only prove, but improve programs
- 24 Evaluation Standards- Utility
- Feasibility
-Propriety
-Accuracy - 3Step 1 in CDC frameworkEngage Stakeholders
- 4Step 2 in CDC frameworkDescribe the program
- 5Step 3 in CDC frameworkFocus the evaluation
- 6Step 4Gather credible evidence
- 7Step 5Justify conclusions
- 8Step 6Use lessons learned
- 9Engage StakeholdersWho needs to be part of design and implementation
- 10Describe the programPaints a picture of the programs activities and intended outcomes. Demonstrate the mission, objectives, and strategies.
- 11Focus the evaluationDecide which evaluation questions are key and the primary goal of the evaluation.
- 12Gather credible evidenceWrite indicators and choose and implement data collection, sources, and methods. The evidence stakeholders regard as credible.
- 13Justify conclusionsReview and interpret data and evidence to determine success or failure and how to improve it. How the stakeholders interpret data.
- 14Use lessons learnedUse evaluation results in a meaningful way. It is utility focused so we know the intended user and aren't searching for one to use the results.
- 15UtilityWho needs the information from this evaluation and what information do they need? Influences key stakeholders and data collection methods.
- 16FeasibilityHow much money, time, and effort can we put into this evaluation?
- 17ProprietyIs there something I have to do to be ethical?
- 18AccuracyWill the design lead to accurate information? Must be balanced against feasibility and utility.
- 193 primary stakeholder groups1. Those affected by the program
2. Those involved in program operations
3. Primary users of the evaluation - 20Describe the program: NeedProblem or opportunity that the program addresses
- 21Describe the program: Expected effectsWhat the program must accomplish to be successful
- 22Describe the program: ActivitiesWhat the program does to effect change
- 23Describe the program: ResourcesTime, talent, technology, equipment, information, money, and other assets available to conduct program activities.
- 24Describe the program: Stage of developmentMaturity of the program
- 25Describe the program: ContextSetting and environmental influences
- 26Describe the program: Logic modelSequence of events for bringing about change.
- 27Focus the evaluation design: PurposePrimary goal of the evaluation
- 28Focus the evaluation design: UsersDirectly experience consequences so they should participate in choosing evaluation focus so it stays relevant.
- 29Focus the evaluation design: 4 general purposes1. Gain insight
2. Change practice and improve
3. Assess effects
4. Affect those who participate - 30Focus the evaluation design: UsesSpecific ways information will be applied. Should be planned and prioritized with input from stakeholders.
- 31Focus the evaluation design: QuestionsCreates boundaries for the evaluation by stating what aspects of the program will be addressed.
- 32Focus the evaluation design: MethodsSpecific design, data collection, and data analysis procedures.
- 33Focus the evaluation design: AgreementsSummarize the procedures and clarify the roles and responsibilities among those who will execute the evaluation plan.
- 34Gather credible evidenceEvidence must be credible and relevant.
- 35Gather credible evidence: Sources of evidence-People
-Documents
-Observations - 36Gather credible evidence: Quality of evidenceCriteria for amount of evidence should be set ahead of time and continuously monitored.
- 37Gather credible evidence: Logistics for gathering evidenceMethods, timing, and physical infrastructure for gathering and handling evidence.
- 38Justify conclusionsConclusions are justified when they are linked to evidence gathered and consistent with standards set by stakeholders.
- 39Ensure use and shared lessons learnedMost important step.
- design the evaluation for intended uses
- prepare stakeholders for evaluation findings
- provide continuous feedback to stakeholders
- schedule follow-up meetings with intended users
- disseminate lessons learned to those who have a need, right to know, or interest in the project. - 40Utility Standards1. Stakeholder identification
2. Evaluator credibility
3. Information scope and selection
4. Values identified and are made clear
5. Report clarity (easily understood)
6. Report timeliness and dissemination
7. Evaluation impact - 41Feasibility Standards1. Practical procedures. Keeps disruption minimal.
2. Political viability. So no group can curtail evaluation operations or to bias or misapply the results.
3. Cost-effectiveness. Justifies expended resources. - 42Propriety Standards1. Service orientation. Assist organizations in addressing and serving needs.
2. Formal agreements. All parties should agree to obligations in writing.
3. Rights of human subjects.
4. Human interactions. Interact respectfully so participants don't feel threatened.
5. Complete and fair assessment. Strengths can be enhanced and problems addressed.
6. Disclosure of findings.
7. Conflict of interest handled openly and honestly.
8. Fiscal responsibility. - 43Accuracy Standards1. Clear program documentation
2. Context analysis. Probably influences on program.
3. Described purposes and procedures.
4. Defensible information sources.
5. Valid information.
6. Reliable information.
7. Systematic information. Info reviewed and errors collected.
8. Analysis of quantitative and qualitative information.
9. Justified conclusions for stakeholders assessment.
10. Impartial reporting. Guard against bias and personal feelings. - 44Why engage stakeholders?- Add credibility
- Resources
- Implement or advocate for change
- Build trust and understanding - 45Logic modelGraphical depiction of the relationship between a programs resources and activities and intended outcomes.
Inputs - Activities - Outputs - Impacts - Outcomes - 46Logic model inputshuman, financial, and community resource available to support the program.
- 47Logic model activitiesAll of the actions of the program to produce desired outcomes.
- 48Logic model outputsDirect products of activities or program deliverables. Tangible/countable terms.
- 49Logic model impactsChange in participants knowledge, skills, behaviors.
- 50Logic model outcomesChange in health status.
- 511. Formative EvaluationImprove programs design or performance. Measurements obtained before a program is fully implemented to ensure the program is feasible, appropriate, and acceptable.
- 522. Process EvaluationTo find out whether the program adhered to the plan. Measurements obtained during implementation of program activities to control, assure, or improve quality of performance or delivery.
- 533. Summative EvaluationEnd products of the program. Measurements and judgements that permit conclusions to be drawn about impact, outcome, or benefits of the program.
- 544. Impact EvaluationAssesses changes in health related cognitive and behavioral domains. Immediate observable effects. 6 months to 2-3 years.
- 555. Outcome EvaluationAssessing changes in health indicators. Long term effects. 5-10 years.
- 565 Steps to decide which questions are most important.1. Define the purpose of the evaluation.
2. Determine who and how the evaluation will be used.
3. Determine what type of evaluation to conduct.
4. Write primary evaluation questions.
5. Determine appropriate indicators for each evaluation question. - 57CBOCommunity based organization
- 58IndicatorsMeasurements that answer your evaluation questions. Observable evidence of accomplishments, changes made, or progress achieved.
- 597 Core Components of Process Evaluation1. Fidelity
2. Dose delivered (completeness)
3. Dose received (exposure)
4. Dose received (satisfaction)
5. Reach
6. Recruitment
7. Context - 60Process Evaluation Component 1: FidelityTo assess the extent to which a program is implemented as intended. High quality, high expectations, intended to be carried out successfully.
- 61Process Evaluation Component 2: Dose delivered (completeness)Number of intended units provided by the program (number of sessions). You want the entire pizza delivered.
- 62Process Evaluation Component 3: Dose received (exposure)The extent to which participants engage with and use materials and activities (attendance and responsiveness)
- 63Process Evaluation Component 4: Dose received (satisfaction)Participant satisfaction with the program
- 64Process Evaluation Component 5: ReachProportion of the intended audience that participates in the program. Me and you are part of the program (denominator)
# of those in priority population who participate
__________________
Total # of people in priority population - 65Process Evaluation Component 6: RecruitmentProcedures to approach and attract participants. (If you want to attract, recruitment does that)
- 66Process Evaluation Component 7: ContextAspects of social, physical, and political environment that may influence program implementation or outcomes.
- 67Purpose of evaluation-social betterment
-program planning and decision making
- empowerment of individuals - 68Summative Evaluations"Should the program be cancelled, continued, or expanded?"
- 69Formative Evaluations"Is the program being conducted as planned? Is the program doing well?" Improve programs or policies.
- 70Research vs. evaluationResearch asks if it will work across multiple settings and evaluation asks if this program is working. Evaluation results are not usually generalize and focus on a particular situation and evaluation of a single program in a particular context. Research seeks to prove, evaluation seeks to improve.
- 71Why conduct process evaluation?To gain an in depth understanding of project implementation including number of activities carried out, quality of activities, reaction of target audience, and problems encountered.
Information
- Description
- Flashcards on Program Evaluation Exam 1, created by Ashley Hundley on 03/10/2017.
- Created by
- Ashley Hundley
- Views
- 7
- Rating