Effective evaluation is essential for understanding whether Communities of Practice (CoPs) within ClimateSmartAdvisors project delivered meaningful capacity development for Climate-Smart Advisors. However, focusing evaluation solely on outcomes (measuring accomplishments and achievements) risks overlooking important insights into how Communities of Practice function (drivers, barriers, participation etc.), as well as missing opportunities to adjust while they are still running. This Practice Abstract demonstrates the value of evaluating both process (how the CoP functions) and outcomes (what it achieves) to strengthen advisory capacity development in CoPs.
Process evaluation enables CoPs to periodically examine their functioning and identify drivers and barriers influencing progress towards goals. It occurs mid-way (after one year) and at the end of the CoP cycle, using evaluation surveys and in-person reflexive sessions focused on: Content relevance; effectiveness of the methods; Participation and engagement; Group dynamics; Organizational aspects.
Outcome evaluation, then, focuses on understanding results in three areas: Changes in capacity of CoP members; Changes in advisory practice; Changes in farming practice. These changes are assessed in final evaluation session.
Evaluating process alongside outcomes transforms evaluation from measurement into a learning tool. Process evaluation answers “how and why” while outcome evaluation addresses “what changed.” Together, they create feedback loops that enable understanding which processes produce desired outcomes allows facilitators to replicate success and address challenges proactively. This integrated approach builds causal understanding supporting knowledge transfer-other CoPs can adopt effective processes, not just aspire to similar outcomes.
Difference in focus between process and outcome evaluation
Process evaluation is focused on:
• Content relevance and alignment with advisors’ needs
• Methods effectiveness (farm visits, peer learning, expert sessions)
• Participation patterns and engagement drivers
• Group dynamics: trust, collaboration, motivation
• Organization: meeting frequency, communication quality
Outcome evaluation focuses on understanding results in three areas:
• Changes in capacity of CoP members: knowledge, skills, and confidence. Assessed at beginning and end using the CSA Capacity Assessment Tool (CAT).
• Advisory practice: integration of climate-smart perspectives into daily work (assessed in final evaluation session and in CSA CAT)
• Farming practice: on-farm changes resulting from improved advice. Assessed in final evaluation session.