4. Monitoring

How to measure organizational and business success on a shoestring or when you have little to no budget

Background¶

Monitoring, Evaluation, Accountability, and Learning (MEAL) is crucial for ensuring the effectiveness and success of projects, as well as long-term organizational and business viability. MEAL enables organizations to make informed decisions and adapt strategies in real time by

🎯 Systematically tracking progress
🎯 Assessing outcomes and analyzing data
🎯 Maintaining transparency
🎯 Creating a culture of learning, sharing results, and creating opportunities to compare projects across landscapes, hopefully finding solutions to broader conservation challenges

This comprehensive approach helps identify and address issues early and fosters a culture of continuous improvement. Effective MEAL practices enhance accountability to stakeholders, ensure resources are used efficiently, and contribute to the overall sustainability and impact of projects. Despite the importance of MEAL, donors rarely provide adequate funding for it. Given this reality, you must find ways to implement MEAL with whatever resources—personnel, time, and budget—are available. This quick-step guide will get you started.

Takeaways¶

Key points to monitoring, evaluation, accountability, and learning on a shoestring:

• Big picture first. There is a tendency to jump straight to metrics and indicators before developing a theory of change and programmatic goals and objectives. Monitoring plans should consider the big picture, so go through the process of developing the program strategy first. However, once created, feel free to change items, such as false assumptions or critical pathways that don’t reflect reality.
• Assessment & learning. Link your assessment or theory of change to your monitoring and learning, e.g., Theory -> Monitoring -> Learning.
• Time to learn. Build learning into our organizational culture. Plan a retreat to analyze results and write about project learning that you publish in blogs, posts, and journals. Build learning into project closing and developing new projects.
• Big investment/big rewards. Building a successful measurement system takes time, staff capacity, and investment. However, the rewards are huge, and the cost dramatically decreases once the system is in place. Don’t fall into the trap of processing, analyzing, and sharing data near the end of a project or phase.
• Participatory research and measurement. Partner with science-based organizations to increase your learning and expand capacity, or use citizen science to engage and connect the general public to your mission and collect data/demystify conservation science at the same time. Employing participatory approaches at all levels of evaluation also takes more time but makes for more meaningful, longer-lasting, and effective results in the long term.

Challenges¶

Monitoring could be the subject written about the most but implemented the least in the nonprofit world. Why is this? The challenges involved give an insight:

• Monitoring is hard to do and is hard to do right.
• Monitoring is boring.
• Your organization doesn’t have staff capacity or applied scientists who can conduct monitoring.
• Project funding doesn’t include monitoring, and it can be costly.
• It could demonstrate that your theory of change, project assumptions/interventions, or organizational mission is incorrect. That’s a big elephant in the room.
• Your organization lacks a learning culture.

Solutions¶

How can these challenges be overcome without requiring massive institutional investment and resources? Let’s start with the big picture of starting with the whole and work through other recommendations.

Start with the Whole¶

It is critical to work from the big picture to your metrics. Follow a process and be deliberate about creating your success measurement system.

• Situation assessment. Utilizing metrics without a measurement system or situation assessment is like building a house without a blueprint (see Program Design). Start with a situation assessment and understand how you, your staff, team, and partners understand the current contextual situation of the system you’re working in and how your project interventions or business application will positively change that system. Do not just assume everyone gets what you’re going to do. For example, if you ask the ten people you work with to define restoration, you might get ten different responses. Memorializing the situation, debating the relationships, and sharing them reach that common understanding.
• Results chains. Create results chains from your interventions to project outcomes. For more details on results chains, see the Program Design chapter and the Open Standards.
• Metrics. Once results chains are established, add how you will measure intermediate and long-term outcomes.

Workshopping these steps, preferably with staff and partners, can get you to a cost-effective system sooner. Don’t worry if it seems to take a while at the outset. Socializing across staff and partners can seem to delay the process, but it will ultimately create longer-term solutions.

Utilize Existing Data Sources¶

Review all your existing data sources, as well as free data sources, before you invest in any new data collection efforts. Organizations often collect relevant data that could be used for M&E, but staff responsible for monitoring and evaluation must be made aware of it. For example, if you are seeking to measure return on investment (ROI), your finance team might already have information you can use to crunch ROI figures. Also, investigate open-source data, such as government and citizen scientist databases, before you decide to go out and collect more information. For example, a community forestry initiative seeking to gauge the fire resistance of a stand it manages could search available USGS datasets to find previous fires that may have impacted its site.

Employ Participatory Approaches¶

Engaging stakeholders in the M&E process enhances data validity and relevance and reduces data collection and analysis costs. Participatory approaches empower community members, beneficiaries, and partners to contribute their perspectives, insights, and local knowledge, enriching the M&E process. For example, a nonprofit organization conducting a program evaluation might organize focus group discussions, community meetings, or participatory workshops to gather feedback from program participants and community members, thereby reducing the reliance on expensive external consultants.

Participatory approaches may also employ apps to crowd-source data and engage key audiences in your mission.

Embrace Technology¶

Technology offers cost-effective and time-saving data collection, analysis, and reporting solutions. Many open-source and low-cost software, mobile applications, and digital platforms can streamline M&E data collection, facilitate data analysis, and enhance data visualization Russell (2024). For instance, a conservation organization might use a mobile data app such as Survey123, QField, or Kobo Toolbox to collect real-time forest health indicators, utilize GIS software such as ArcGIS Pro and QGIS to analyze, and use cloud-based platforms to store data, eliminating manual data entry and paper-based reporting. App-based data collection also allows multiple users to collect and input data.

ArcGIS Survey 123, from ESRI, is a good example of a mobile app for field data collection and mapping. It can be customized to collect various forest metrics such as tree species, diameter at breast height (DBH), and tree condition, and this information can be fed directly into ArcGIS software. It is easy to use and synchs effortlessly to your desktop or online GIS software. However, it does require a subscription, which can be expensive for some organizations.

Build Internal Capacity¶

Consider training existing staff to build internal M&E capacity, reducing your reliance on costly external consultants and service providers. By equipping staff with the necessary skills and knowledge, organizations can conduct M&E activities in-house, saving costs and ensuring sustainability. A small non-profit organization might provide M&E training workshops for its staff, empowering them to design surveys, analyze data, and generate reports independently, enhancing organizational efficiency and effectiveness.

Foster Collaborative Partnerships¶

Collaborating with other organizations, academic institutions, and government agencies can facilitate resource-sharing, knowledge exchange, and joint M&E initiatives. By pooling resources and expertise, organizations can achieve more comprehensive M&E outcomes at a lower cost. For example, a consortium of NGOs working on a regional conservation project might establish a joint M&E framework, share data, tools, and methodologies, and coordinate evaluation activities to avoid duplication and maximize efficiency.

Seek Creative Ways to Fund Monitoring¶

Many funders don’t provide funding for monitoring, but they almost always require project reporting. Incorporating your measurement into reporting may help cover the costs. Educate the funders, too. For example, don’t let them require reporting on ‘acres and bucks’ and explain why those metrics are okay, but don’t tell the full picture about complex states such as forest health or resilience.

Going back to partnering with other organizations is critical here. For example, you could partner with a university on a research project. First, create a research agenda for your work to point researchers to what data and information you need (rather than them coming to you with pre-formed research). Academic researchers will want specific outcomes that might not connect with your data and outcomes needs, but they could help develop technology or collect some data you need.

Car Analogies¶

Originally, we framed organizational monitoring approaches as the extremes of owning a Yugo vs. a Cadillac. This is a poor analogy since Yugos were cheap, broke down frequently, and some consider them the worst cars ever. On the opposite end of the spectrum, Cadillacs are expensive and beyond the purchasing power of many individuals.

When I was at university, starting my senior or fourth year, I owned a simple red Honda Civic hatchback. It was plain but incredibly dependable and gas efficient. I never had a single-engine problem with that car, and it always started, even during the dark, cold months of winter when some autos from the 1970s and 80s rebelled and would not turn over in frigid northern US temperatures.

On the other hand, you may need something equally dependable, affordable, and expansive, such as the workhorse minivan. A Civic will get you reliably where you need to go, whereas a min-van will do that and allow you to bring along your kids, their friends, and all their stuff. I rented a Kia minivan once when the rental agency at the San Diego airport ran out of cars. That thing was a rocket. I had it going 90 mph several times (don’t tell CHP) while driving through the Cleveland National Forest. Vroom vroom!

What is the absolute bare-bones, dependable monitoring system that could be implemented (Civic) vs. a more expansive version (Minivan)? We suggest a couple of examples in the pulldowns below. Both examples are hypothetical stand-thinning projects with prescribed fire applied periodically after thinning for maintenance. The project assumption is that thinning will lead to increased forest health and reduced high-intensity wildfires. Sometimes, you need a dependable Honda Civic:

And sometimes you need a Minivan:

Resources¶

• Recipes for Conservation. A summary of how to develop the conservation standards within or among organizations. The recipes have some valuable sections for businesses but are nonprofit and agency-focused.
• Pause & Reflect Toolkit. Pause and reflect is an organizational stop and step back to reflect on what you’re doing and why as well as an evaluation of how it’s going. This toolkit provides project leaders with a centralized collection of resources, enabling them to find everything they need to lead and support a pause and reflect process.
• Measures of Success. Practical guide (in English and Spanish) for designing, planning, and implementing conservation projects. The Spanish version can be found as a free download, and the English version is available for purchase from Island Press Margoluis & Salafsky (1998).
• The Open Standards for the Practice of Conservation, developed by the Conservation Measures Partnership CMP (2020), is a definitive resource for program and project design.