The Planning, Monitoring and Evaluation Challenge

Projects invest time, effort and money. At the end, one simple question needs answering: was it worth it?

The answer can be given in different ways, by looking at return on investment, achievement of objectives, or the level of satisfaction among beneficiaries. But they all have this in common: data are required.

Managers also require data, to make decisions and improve implementation. Data are needed to prepare good quality reports. Data are important for the future, to learn lessons about what works and what does not.

This is the important role monitoring and evaluation has to play in projectsThe focus is on agricultural development projects and programmes supported by the International Fund for Agricultural Development (IFAD) - click to visit.: to collect and analyse data.

The Challenge

There can be many reasons why Planning, Monitoring and Evaluation (PM&E) in a project is weak, but they usually fall under these three headings:

• we don't know what to measure
• we don't have appropriate tools
• we don't have the necessary people and skills.

A fourth issue is lack of management supportManagement includes the project coordinator, but also managers of implementing agencies, service providers and IFAD. Managers should show interest, demand information and provide feedback: this can motivate and improve M&E performance. They should intervene in case of non-compliance and when staff or institutions do not deliver.: without some sort of 'enforcement' PM&E systems rarely work.

Three pillars of PM&E:

• the results framework: what to measure
• a system for collecting and managing data
• the people who have to make it work

Quote

What gets measured, gets managed.

Peter Drucker, management consultant, educator and author

If you don't measure progress and results, you don't know what's going on. How can you then manage or improve your project?

The Approach

The three 'excuses' mentioned above can be addressed by putting in place three pillars for effective PM&E:

1. a clear results framework that defines what needs to be planned for, what needs to be measured and what data needs to be collected;
2. a system with clear procedures and user-friendly tools that facilitate how data is collected, recorded, transmitted, stored, processed and used;
3. adequate human resources: enough staff, a clear division of responsibilities, adequate knowledge and skills related to PM&E, and capacity building when these are lacking.

Results Framework

The results framework defines expected results at different levels
Results at different levels, and thus defines what needs to be planned for, what needs to be measured and what data needs to be collected.

Logical Framework

The logical framework, or logframe, has a central place in the design documents of IFAD-projects. It provides a 'snapshot' of the project, a roadmap for implementation. It is a management tool that can guide PM&E.

Preparing a logframe is given much attention during the design stage of a project, which includes defining the results hierarchy
M&E can test the results hierarchy or intervention logic or intervention logic, selecting indicators and setting targets. Nevertheless, the quality of the logical framework can vary, while project managers and implementers need a good logframe that they fully understand and use effectively.

Reviewing the logframe

At the start of a project, it is therefore important to review the logframe and make modifications if needed. The intervention logic of a project is not always easy to understand, and it can help to transform the first column of the logframe into a diagram.
A diagram that maps expected project results Such a diagram can be used as a planning tool and to explain the project to others. Special attention should be given to having a good logic from output to outcome level, and a manageable set of project-specific indicators that are linked to IFAD Core Indicators
IFAD core indicators (source: Taking IFAD's Results and Impact Management System
to the Next Level, IFAD 2017). IFAD's Core Indicators are used by all projects
to report results in a standardised manner..

As a result, project managers and staff should be fully conversant with the revised logical framework, should be in agreement that it adequately represents the project design, and should be motivated to pursue result-oriented annual planning and management. While reviewing the logframe can be repeated during the life of a project, care must be taken that there is enough continuity in the use of indicators.

The PM&E System: Tools and Procedures

When a good results framework is in place and it is clear what should be measured, tools and procedures are needed to:

• collect data from the field
• record data in a consistent manner
• transmit data to a central place, often via the M&E Officer
• store data safely, well organised, easily accessible
• process data so that it can be used
• present information to management and other stakeholders, in reports and through other means.

PM&E systems, in the broader sense, consist of a variety of tools and procedures that can be used to accomplish these tasks.

Tools

Tools can include paper or electronic templates to record data at field level; survey questionnaires; guidelines for semi-structured interviews; handheld GPS units to capture geographic coordinates of infrastructure or farmer groups; and digital cameras to get a visual impression of project activities and results. These are just some examples of data collection tools, there are many other toolsSome more examples: quadrats, sampling bags and weighing scales for yield measurement; smartphones for interviews, to take pictures, or simply communicate with the M&E Officer; computers, printers and projectors to store, process and present data. Each project will have to decide what exactly is needed.. In the right hands, even a simple notepad can be a powerful M&E tool.

Procedures

Procedures can be broad, for example stating that Annual Outcome Surveys will be conducted from the third year onwards. Procedures can also be detailed, for example specifying how often implementers and service providers submit monitoring data, in what form, when and to whom. It is useful to list the key recurrent tasks on an annual planning and monitoring calendarClick to open a one-page example of an annual planning and monitoring calendar with dates and the person responsible. Likewise, listing the major PM&E activities that will take place over the life of a project on an overall PM&E calendarClick to open a one-page generalised planning, monitoring and evaluation calendar is useful to ensure there is a systematic approach. Evaluation activitiesEvaluation activities can include a baseline survey, mid-term review, annual outcome surveys, impact assessment, stories from the field, and thematic studies on, for example, gender aspects or innovative technologies. are often contracted out because of the need for impartiality and the special skills involved. This takes time and should be prepared for well in advance.

The Central PME/MIS System

Much of the data that is collected would normally be kept in a central M&E System (a system in the narrow sense): when people talk about "developing an M&E system" this is usually what they mean. The following options are available:

• a spreadsheet-based system
• a tailor-made database, designed specifically for the project
• a ready-made, off-the-shelf database system, which can normally be modified to some degree.

In reality, a database is only one of various tools that a project will use. It has a specific function: to keep data in one place, safely, easily accessible; and to transform large quantities of data into useful information.

Which type of system is most suitable will depend on the project: aspects such as the desired features, flexibility, cost and user-friendliness should be looked at. Each project should discuss in detail how data collection will be done, what tools will be used, who will be involved, and how data will be entered and consolidated. Equally important is to discuss what the M&E system should produce: what types of reports, tables, graphs and maps. Regardless of the type of system, the M&E Officer must closely follow the design process to ensure the project gets a system that meets the needs.

Human Resources

People have to make it work. A simple system consisting of some spreadsheets can contribute greatly to effective PM&E when used by capable, motivated staff. Sophisticated database systems can become worthless when people don't use them well. Human resources and capacity building clearly need attention: what does this entail?

Define roles and responsibilities

A motto for project M&E could be: "to implement means to monitor". In larger projects with multiple components and implementing institutions, monitoring cannot be carried out by one person. The PM&E function should therefore be decentralized, and the M&E OfficerClick to open a one-page description of some of the key roles and qualifications of the M&E Officer becomes a process manager who focuses on facilitation and capacity building.

Each implementing institution should have a designated person responsible for PM&E. These focal persons participate in annual planning and coordinate collection and submission of monitoring data. What data needs to be submitted, when and by who should be clearly described and agreed on. The M&E responsibilities of each institution should be included in contracts, TORs and MOUsTerms of Reference, Memoranda of Understanding and other formal documents that spell out the tasks of implementing agencies and service providers.. This is a task for the M&E Officer.

Capacity building

The persons involved in designing the M&E system should receive hands-on training as the system is developed or installed. Initial training for other project staff is equally important: the purpose is to impart M&E knowledge and skills but also to ensure they share a common understanding of the projectThe logical framework, targeting approach, M&E roles and responsibilities, and reporting requirements are among the aspects that must be well understood and agreed upon.. Such training should involve the entire PMUProject Management Unit, also referred to as Project Coordination Office (PCO), Project Coordination Unit (PCU), Project Management Team (PMT) and Project Facilitation Unit (PFU). and selected staff from implementing agencies. The focus should be on practical PM&E activities and tools, so that participants can start using the project's M&E system.

However, initial training typically takes only a few days, and it often happens before all implementers have joined. Having a well-designed system with guidelines and initial training is thus no guarantee that it will be used effectively. Couple this with staff turnover, and it becomes obvious that more is needed.

Follow-upFollow-up is mostly done during supervision missions. Occasional follow-up by M&E specialists is another possibility. Ideally, each country programme would have access to at least one local consultant who specializes in M&E. is needed to check whether PM&E systems are used as intended. For financial progress this is done: disbursement, and submission of SOEs, WAsStatements of Expenditure, Withdrawal Applications and other financial documents. and audit reports, is carefully tracked, the documents are scrutinised and action is taken if there are shortcomings. The same is needed for physical progress: action should be taken if M&E data or progress reports are below standard, and backstoppingBackstopping: technical advice and coaching, provided as needed to those involved in PM&E, based on the findings of data quality assessment and in response to performance gaps. should be provided if the problem is lack of skills.

Additional training and refresher training will help to improve the skills of those involved in PM&E over time, to bring new staff joining the project on board and to ensure that new tools, introduced at a later stage of the project, are understood and used properly.