Increasing representativeness in data scarce environments

Léo Gorman
Léo works in the RHoMIS team in Bristol. His work focuses on data analysis, reporting and developing back end systems. Leo grew up in Ireland and his background is in Physics and international development.

Editor: Léo has been an invaluable member of the RHoMIS team, adding a thorough systems-thinking approach to our data analysis, skilled programming, and on top of that, an amazing fun attitude full of adventure! As he is begins his RHoMIS-related PhD, we have asked him to mark the transition with a new blog post. We are grateful for all he has given RHoMIS so far, and wish him the best as he begins these further studies.

---

​I have been part of the RHoMIS team for the past year and a half, both as an intern and as a consultant. My work during this time was focused on backend processes.

I helped design systems for consistent and more automated data-processing, indicator calculations and project reporting. For the past three months I have been trying to ensure that the systems that were developed are transparent and usable for the wider community of RHoMIS users.

​I have recently embarked on a doctoral program with the University of Bristol and the Alan Turing Institute, the UK’s national institute for data science and artificial intelligence. ​

In addition to work on the backend of RHoMIS, Léo has conducted enumerator training in The Comoros, DRC and Kenya.

I hope this blogpost will provide the opportunity for people to understand what I hope to achieve over the next three and a half years. I am still learning, so if you think anything I have said needs to be corrected or needs to be further developed, don’t hesitate to contact me. I hope this blogpost will encourage people to share their thoughts and ideas on what I am doing and how to do it better!

At the early stages of my PhD, my precise question is still evolving. However, at the core of my project lies the aim of increasing the representativeness of RHoMIS data and maximising its usability. To give some context, RHoMIS interviews are conducted on a project-by-project basis. This model has been very successful and has led to approximately 28,000 interviews in 31 countries. However, many of the data are only representative of smaller areas, such as the counties and sub-counties which are of interest to the project involved. This can make it more difficult to conduct analyses which are nationally or regionally representative.

How can the issue of representativeness be addressed? In short, by drawing parallels between RHoMIS data and datasets with greater coverage (such as satellite data or census data), trends identified using the RHoMIS data could be scaled up over larger areas. This could allow us to generate synthetic data in areas where RHoMIS surveys have not been conducted before. This ‘borrowing of strength’ (often called small-area estimation or multiple imputation) has been used to estimate things such as poverty rates and land area in data scarce environments (bottom of article for reference).

Leo helped support RHoMIS implementation with a local development organisation in The Comoros in early 2019.

I find this technique very exciting. However, when applying it to smallholder farmers there are some key points to consider:

• Overgeneralising: It has been shown that multiple imputation methods can blur out some of the interesting differences between particular areas. Smallholder farmers are very diverse so it is essential that any generalisations can capture this diversity. Considering the influence of less quantifiable contextual factors (e.g. local culture) will be an essential part of the process.
• Undergeneralising: I want these synthetic data to provide a good representation of households. However, the more variables we try to model, the greater the uncertainty. Hence, a needs assessment which identifies the key variables of use will need to be conducted. 
• Version control: RHoMIS data is being collected at an increasing rate. This could greatly increase the strength of the synthetic data that is generated. It will be interesting to track how the synthetic data evolves as these new interviews are conducted.
• Uncertainties: The errors in this dataset and any associated indicator calculations must be made explicit.

I am sure that I have only touched on a small fraction of the barriers to come over the next few years. Nonetheless, I am excited by the opportunity to try and build on the fantastic work of the RHoMIS team and I hope that the RHoMIS community of practice can help me develop these ideas to produce something that will be useful for all of us.

Select references:

• Kilic T, Djima IY, Carletto C, 2017, Mission Impossible? Exploring the Promise of Multiple Imputation for Predicting Missing GPS-Based Land Area Measures in Household Surveys, World Bank Working Paper, 8138.
• Dang H, Jolliffe D, Carletto C, 2018, Data gaps, data incomparability, and data imputation: A review of poverty measurement methods for data-scarce environments, World Bank Working Paper,  ECINEQ – 456.​