Posted by: Jessica Elisberg, Marketing Communications Associate, Microfinance Information eXchange (MIX)
At the SEEP Network Annual Conference earlier this month in Washington, DC, a group of practitioners gathered for a full day to discuss their experiences with using various measurement tools to inform and improve their microfinance programs. This session focused on client assessment, as opposed to impact analysis, and the importance of moving beyond measurement to using data for decision-making with the goal of ultimately improving MFI’s service to clients.
Practitioners identified a number of measurement tools that are being used by development organizations, including the FINCA Client Assessment Tool (FCAT), Progress Out of Poverty Index™ (PPI™), Poverty Assessment Tool (PAT), participatory wealth rankings, food security surveys, and market surveys. These tools are being used to varying degrees to better target programs and to inform product and service development by microfinance organizations, but a number of key questions were posed in the day’s discussions that speak to the broader challenges of effectively integrating data collection and program design to achieve social and poverty outreach objectives.
Poverty Data Analysis: What data are being collected, what is the quality of the data, and who does the analysis?
These are big questions for a lot of organizations that struggle to find a balance between the need for good information and the lack of resources available to collect that information consistently and efficiently. A common experience among practitioners is that data analysis tends to take place at headquarters or by a third party, thus separating the information from the context in which it was collected. This can either make the information more objective, make it lose its meaning, or a bit of both. Just as importantly, this separation reduces the value of the analysis for those who are working in the field. It’s difficult to get those on the ground to truly value data and utilize analysis well in program design if they are far-removed from the analysis process, but it’s also difficult to create the time and develop the capacity to have data analysis done in the field. Even if this balance is achieved, important questions remain: what data are being collected, and what measures are being taken to ensure that they are of consistently high quality?
Data Interpretation: What do the data really tell us? And how do you take context into account?
Many practitioners raise these questions because they are fundamental challenges in development. Certainly poverty scorecards and other measurement tools endeavor to tailor questions and scoring to country-level context, but it’s difficult to adapt them to more specific settings – at the village level, for instance. Additionally, the questions that are included in such assessments may not actually be the best indicators for measurement, and without control groups it is always difficult to show causal relationships. One practitioner remarked during the discussion that contextual analysis tends to be reactive rather than proactive, thus limiting the effectiveness of the data interpretation.
Application of Knowledge: How do you use poverty data and analysis for decision-making?
Because experience thus far is relatively limited, this remains the least-tested question. Many organizations have found that their measurement tools have primarily served to reinforce existing decisions and program designs rather than to inform changes. This could be due to the accuracy of the critical assumptions practitioners are making when designing programs, or it could be because they aren’t asking the right questions when analyzing the data. The general belief is that poverty measurement tools are helpful in identifying whether program targeting is effective, although in some cases these tools simply reveal that better tools are needed. As a result of this discussion, it is clear that much remains to be learned and best practices using data to improve social performance have yet to be established. SEEP is considering addressing this gap in the coming year’s learning agenda.
It is worth noting that the common experience amongst organizations is to combine multiple tools for measurement. Furthermore, the need for regularly-updated existing tools and fostering the creation of new tools to improve poverty measurement and client assessment in microfinance is widely recognized.
For notes and powerpoint presentations from other sessions at the SEEP Conference on Social Performance and Poverty Outreach, you can visit the SEEP Network’s Annual Conference page.
Comments