This report presents a revised logical framework (logframe) and objectively verifiable indicators (indicators) for the Aceh Forest and Envrionment Project (AFEP) to form the basis of an improved results framework for the project.
The need for a revision of these has arisen from the recognition that the initial Project Performance Indicators (Schedule 4 to Annex 1 in the Grant Agreements) include indicators that are either inappropriate or unmeasurable within the timeframe of the project (i.e. not "SMART"). There also project activities and outputs that are not adequately represented in the results framework summary in the Project Appraisal Document (PAD) (Part X; page 63).
The revised logframe, indicators and results framework have been developed through a series of meetings between LIF and FFI management staff, World Bank (WB) and Multi Donor Fund (MDF), and a consultant hired to review the Logframe and Indicators.
2. Current Project Objective, Activities, Logframe and Indicators
The existing sources of information about the AFEP project objective, activities, logical framework and indicators, are: the Grant Agreements between the International Development Association (acting as the trustee and Partner Agency under the Multi-Donor Fund for Aceh and North Sumatra) and the Implementing Entities (LIF and FFI); and the Project Appraisal Document from which the Grant Agreements were derived.
2.1 Project Objective and Activities
A comparison of the project objective and activities as described in the Grant Agreements with the Implementing Entities (IE's), and the PAD are shown in a Annex 1. While intuitively these are similar, there are some subtle differences. Most notable are differences in the Project Objective, which in the Grant Agreements is to
" Ensure that environmental services provided by the Leuser and Ulu Masen forest ecosystems are effectively protected during the post-tsunami reconstruction process in Aceh"
while in the PAD there is an implication of primary and secondary objectives:
Protect Critical Environmental Services from the Leuser and Ulu Masen ecosystems during the post-tsunami reconstruction process, and mitigate negative impacts of reconstruction on the forests of Aceh, and mainstream environmental concerns into planning processes
The titles of activities given in the Grant Agreements are more informative than those in the PAD, but the latter has more detailed descriptions in its narrative.
2.2 Results Framework and Indicators
The Grant Agreement has a list of Project Performance Indicators (SCHEDULE 4 TO ANNEX I). These are identical to the Verifiable Indicators of Achievement of the Project Results Summary in the PAD (shown in Annex 2).
In the PAD, there is also a "Timeline of Activities and Milestones", but this only covers the first 2 calendar years of the project
3 Issues, Concerns and Constraints
The Monitoring and Evaluation Consultant (full report: see Annex 3) employed to review the Logframe and Indicators listed a number of issues and concerns. To address these we have done the following:
1. Developed a revised and improved Project Intervention Logic (see section 4) based on a clear standard structure (see box below). This allows for much clearer differentiation of the outputs/results/outcomes and objectives within the project, particularly which are within project control and which are subject to externalities. In the revised Project Intervention Logic, it will also be noted that we have addressed the issue of confusion between interpretation of some activities proposed in the PAD. In particular, activity group 1.1: Establish a multi-stakeholder governance framework, as been subsumed into other activities. Furthermore the development of this intervention logic jointly by the 2 IE's consolidates a common understanding of the project activities and outputs by the IE's. Component 3, dealing with project management, has NOT been included in the Project Intervention Logic as it covers activities that are NOT interventions directly contributing to the Project Objective. Indicators for this component are however included in the results summary, and successful implementation of Component 3 ensures that the project can make the interventions shown in the Intervention Logic. The common Intervention Logic also solves the issue of FFI having a different workplan structure to the AFEP activity structure, because the suite of individual activities in the FFI workplan do coincide with the suite of activities in AFEP, and that as long as it the common AFEP reporting framework is followed this does not represent a major obstacle. For this year (2007), this will be addressed by additional codes in the FFI workplan spreadsheet so that it can be re-organized to match the appropriate AFEP structure. It is anticipated that in future years, a fully functional Project Coordination Unit (PCU) will ensure that fully integrated joint workplans will be produced for the Project. The use of different terminology in the FFI workplan will also be brought into line with standard usage.
2. Developed a more realistic and clear set of Indicators, which also follow the revised Project Intervention Logic (section 5).
3. Clarified which baseline data exists and which baseline data is being collected (to be further detailed in the Monitoring and Evaluation Plan).
4. Rationalized the presentation of FFI's workplan with respect to the AFEP intervention logic and results summary.
The following concern will be addressed in the Monitoring and Evaluation Plan:
- The need for Indicators and Milestones for the duration of the project, not just the first and second years
Both IE's are aware of and actively pursueing the establishment of the Project Coordination Unit (PCU). In future, many of the issues previously highlighted will avoided, as monitoring and evaluation, and planning will be coordinated by the PCU.
4. Revised Intervention Logic
Based on the comments and recommendations of the M&E Consultant, and the intensive joint planning sessions held at the end of April to review the Indicator framework, a revised Intervention Logic for AFEP has been developed. This clearly differentiates between outputs from the project and the higher level results, outcomes and objectives (enclosed in the shaded box). However due to space constraints the project outputs shown outside the shade box are "cummulative" outputs which could are at the interface with Intermediate Outcomes, where some externalities may still occur. The detailed activities and their outputs from which these cummulative outputs are derived, are not shown. In general, the format follows that recommended by the consultant in his report (see Figures 1 and 2 of his report).
One particular issue was how address the "addional" level represented by the Components 1 and 2. These have been included in the revised Intervention Logic as showing logical "clusters" of results (two under Component 1 and three under Component 2). However in the development of the indicators (see below), a simplified structure for the project objective indicators is recommended as per "Option 2" of the consultants report (page 12).
Every attempt has been made to ensure that the Intervention Logic is applicable to both sites and the only site specific outputs are in those related to developing environmental and conservation awareness.