"We will more than triple investments in baseline information collection—just the collection of basic data from the inception of programs—so we can improve outcomes by checking progress and making course corrections as we go." —Rajiv Shah, USAID Administrator, June 18, 2012
Baseline values for performance indicators for which there are existing data are called for in a PMP. USAID expects that baseline will be collected during the 90 day PMP preparation period, or if they are not a schedule for their collection will be provided. Consistent with USAID “best practice” instruments used to collect baseline data are to be the same ones that will be used subsequently to gather performance data. This implies that those instruments will be developed, pre-tested and used to collect baseline data before a PMP is submitted for approval.
The timing of baseline data collection is dictated not only by USAID's requirement to report performance indicator baseline values in a Mission PMP, but also because failing to collect baseline data before the interventions that could affect them start could compromise USAID's ability to estimate the changes its interventions appear to have brought about. A World Food Programme guide to baseline studies illustrates this point using information on the amount of change a nutrition program appeared to have caused, based on when baseline data were collected. As the table below shows, program impact was seriously underestimated when baseline data collection did not occur until after the intervention started.
|Week in which Baseline Data were Collected||Baseline Estimate||Subsequent Estimate at Week 10||Estimate of Change Over Time (baseline estimate minus 10 week estimate)|
Plans for conducting an indicator baseline study will vary as a function of the indicators for which baseline values are needed. For some of the DOs on which a Mission focuses, including economic growth and trade performance improvements, existing in-country data and international data series may provide the baseline values needed for many of the indicators included in a CDCS. For other indicators, and in other sectors, investments in baseline data collection will be required. Sometimes a survey of a representative sample of intended beneficiaries will provide many of the baselines values that are needed. For other programs, more varied data and methods may be required, including instrument measures (time, distance, weight); written scales for assessing policy or organizational status; or techniques for determining citizen awareness of information or regulations.
USAID's requirement that the techniques and data collection instruments used to collect baseline data be the same ones the Mission will later to collect performance data means that they must be developed with repeated use in mind, and include sufficiently detailed instructions to allow a future set of data collectors to copy the methods, including any sampling procedures. Annual replication of the baseline data collection procedures to obtain performance information has important implications for how much USAID spends collecting baseline data. Questions included in a draft version of a baseline study that are not focused on indictor data warrant careful consideration, as they can drive up costs. The indicator values focus of this type of study, and the need to be able to repeat it, almost exactly, multiple times distinguishes a USAID baseline study from other types of early situational analyses and assessments called for in USAID's CDCS guidance to help a Mission improve its understanding of trends and existing programs in various sectors as well as broader contextual issues that will interact with a CDCS strategy.
When preparing a PMP, Missions also need to consider who should be asked to gather baseline data and how large their studies need to be. USAID's evaluation policy, for example, requires that where baselines involve probability sampling, the sample size is expected to be large enough to permit valid statistical comparisons over time. Further, if the Mission is planning to undertake an impact evaluation that will involve collecting data on some of the indicators identified in its CDCS Indicator Annex, it may be advisable to ask the external evaluator USAID plans to engage to conduct such an evaluation to gather both the baseline and endline data required for USAID's target group or location as well as for the impact evaluation's comparison groups.
Once collected, baseline values should be entered into the Mission’s Performance Indicator Summary Table.