You are here

Conducting Data Quality Assessments

Data Quality Assessments (DQAs) are one of two important ways in which USAID checks on the quality of the performance monitoring information it collects and publically reports. The second way in which USAID staff learn about performance data quality issues are through project audits conducted by the Office of USAID’s Inspector General which sometimes include a review of project performance monitoring. Occasionally a USAID evaluation will also touch on performance data quality, but this occurs less frequently.

Planning for Data Quality Assessments (DQAs) begins with the development of a PMP and continues, at the project level, in a project M&E Plan, as outlined in earlier kit pages on Data Quality and Limitations and Data Quality Assessment Plan. During implementation the focus shifts to the practical steps involved in conducting a DQA.

Who Conducts DQAs?

USAID's November 2012 update of ADS 203 indicates that Missions should not hire an outside expert to assess the quality of their data. Mission staff, usually the technical offices, Monitoring and Evaluation staff, or project/activity implementers, as part of their award, can conduct the assessment, provided that mission staff review and verify DQAs conducted by implementing partners. This may entail site visits to physically inspect records maintained by implementing partners or other partners. This revision of ADS 203 is somewhat more explicit than earlier versions in defining DQAs as an in-house USAID and Implementing Partner responsibility, rather than one that would normally be contracted out to a third party.

Tools for Conducting a DQA

USAID's TIPS on Conducting Data Quality Assessments outlines an eight step process for carrying out a DQA as summarized below. In its November 2012 revision of ADS 203, USAID incorporated a new template for writing up the results of a DQA which can be filled in on line. This USAID Data Quality Assessment Checklist can be accessed on the next page of this kit and completed on your computer, saved or printed out. This checklist helps user assess performance data in light of USAID's five quality standards—validity, integrity, precision, reliability and timeliness. At the end of USAID's checklist are several recommendations for conducting an effective DQA. These recommendations are reproduced below.

Recommendations for Conducting Data Quality Assessments

  1. Data Quality (DQ) assessor should make sure that they understand the precise definition of the indicator by checking the Performance Indicator Reference Sheet. Please address any issues of ambiguity before the DQA is conducted.
  2. DQ assessor should have a copy of the methodology for data collection in hand before assessing the indicator. For USAID Missions, this information should be in the PMP’s Performance Indicator Reference Sheets for each indicator. Each indicator should have a written description of how the data being assessed are supposed to be collected.
  3. Each implementing partner should have a copy of the method of data collection in their files and documented evidence that they are collecting the data according to the methodology.
  4. DQ assessor should record the names and titles of all individuals involved in the assessment.
  5. Does the implementing partner have documented evidence that they have verified the data that has been reported? Partners should be able to provided USAID with documents (process/person conducting the verification/field visit dates/persons met/activities visited) which demonstrates that they have verified the data that was reported. Note: Verification by the partners should be an ongoing process.
  6. The DQ assessor should be able to review the implementing partner files/records against the methodology for data collection laid out in the PMP (for USAID Missions only). Any data quality concerns should be documented.
  7. The DQ should include a summary of significant limitations found. A plan of action, including timelines and responsibilities, for addressing the limitations should be made.

Missions benefit from a written report on a DQA that includes specific findings about data quality issues and recommend improvements that might be made. A USAID/Macedonia DQA report, while not new, nevertheless includes a useful structure for presenting DQA issues and recommendations as the section below on the Mission’s indicator for exports illustrates.

Click on the image to view it larger in a new window