Data collection is now an integral part of humanitarian action. And in this area of their work, as in the aid activities themselves, the emergence of the pandemic has forced organisations to adapt. This feedback from the camps in Cox’s Bazar, Bangladesh, shows how precious a resource refugees are.
In 25 March 2020, the Refugee Relief and Repatriation Commissioner (RRRC) suspended all non-essential humanitarian activitiesUnder operational guidelines, only critical services and assistance are permitted, including all health and nutrition services, WASH (Water, Sanitation and Hygiene) activities and services, Liquefied … Continue reading in the Rohingya refugee camps in Cox’s Bazar and reduced the number of humanitarians allowed in by 80%Ibid.. All regular primary data collection stopped. Six months later, programming and access restrictions remain in place. The number of access permits has increased slightly, but large-scale face-to-face data collection by external teams remains suspended. However, for the response to make evidence-based decisions and meet the needs of the affected population, primary data collection needs to continue.
The approach to data collection in Cox’s Bazar has been predicated on the belief that face-to-face data collection using external teams would always be an option. An accessible population combined with lack of coordination and limited information sharing between organisations led to an environment where actors quickly collected primary data as information needs arose. The sudden loss of access forced humanitarians to rethink data collection.
The sudden loss of access forced humanitarians to rethink data collection.
Prior to the pandemic, representative needs assessments were regularly conductedLarge, regular, representative data collection exercises in the response are: Joint Multi-Sector Needs Assessment (J-MSNA), Nutrition Sector SMART (Standardized Monitoring and Assessment of … Continue reading. A review of over 200 reports published between August 2017 and December 2019 found that most were “assessment reports” (46%) based on face-to-face primary data collection through household interviews, key informant interviews (KIIs), focus group discussions (FGDs), and data collected from service providersAn assessment and analysis landscape review was conducted by the ACAPS analysis hub prior to the Covid-19 pandemic. However, it is yet to be published due to a drastic shift in focus due to the … Continue reading. This paper explores how data collection units continued operating to provide critical information to responders during the pandemic. Through KIIs with seven data collection actors currently operating in Cox’s BazarKIIs were conducted with REACH, BBC Media Action, Translators Without Borders (TWB), CARE, the IOM’s NPM unit, IOM’s Communicating with Communities (CwC), Harvard academics conducting academic … Continue reading and the findings of previous research, it explores how data collection was impacted by physical access constraints and the potential lessons learned for primary data collection in such contexts.
Adapted data collection methodologies used during Covid-19
Remote mobile data collection
To implement programmes, field operators need data on who is working where (4W)The 4W databases are used by the UN Cluster System to provide key information regarding which organisations (Who) are carrying out which activities (What) in which locations (Where) in which period … Continue reading, access and security information, needs assessments, and beneficiary data. Coordinators require needs assessment registers, project databases, or multi-sector reports“Bangladesh country report. A case study of humanitarian data transparency in the Rohingya crisis”, Publish What You Fund, August 2020, … Continue reading. To continue to meet this need, large data collection units had to adapt to remote data collection through phone surveys. However, teams were not set up to do this on such a large scale because a strict telecommunication ban had been placed across the camps until April 2020On 2 September 2019, the government of Bangladesh ordered companies to stop selling SIM cards to Rohingya refugees and shut down mobile phone services in the camps; authorities also confiscated … Continue reading. The lifting of the ban was not followed by the restoration of 3 and 4G internet until August 2020Md. Kamruzzaman, “Bangladesh to restore phone, internet at Rohingya camps. Restoration of communications is important for better services to Rohingya amid the pandemic, says refugee … Continue reading, and network coverage remains poor, unreliable, and patchy, resulting in calls dropping mid-interview and unreachable phone numbers. Rohingya refugees also rely mainly on solar power to charge their phones, which are often turned off to conserve battery. Coordination and real-time lessons learned allowed humanitarians to develop a general understanding of the best times to call, but the combined issues resulted in an answer rate of roughly 30-40%. This meant sample sizes needed to be significantly larger than usual.
Many data collection teams rely on Bangladeshi enumerators who speak Chittagonian, a language similar to RohingyaIn November 2018, TWB found that 36% of Rohingya refugees could not understand a basic sentence in Chittagonian. Rohingya is the only spoken language that all refugees understand and thus, … Continue reading. While many enumerators speak some Rohingya, translation and language issues make data collection over the phone difficult. To mitigate the increased potential for misunderstanding, some data collectors increased quality checksOxfam GB, Going digital: improving data quality with digital data collection, July 2018, … Continue reading, such as senior enumerators calling back respondents to verify answers and daily monitoring of incoming data to flag and correct errors. This increase in quality checks along with network issues, larger sample sizes, and reluctant participation almost doubled the time needed to collect data. Attention spans waned and the quality of interviews declined at the 20-30-minute mark. To address this, quantitative surveys were shortened, and questionnaires simplified. Some data collectors split their interviews into two shorter sessions to maintain focus.
The biggest challenge faced by data collection teams was the lack of a large representative phone database to produce the sample needed for a representative surveyJ-MSNA, an annual crisis-wide and inter-agency multi-sectoral assessment conducted on behalf of the humanitarian response, combined the UNHCR refugee registration database, UNHCR census data, and the … Continue reading. Many teams utilised snowball samplingBusiness Research Methodology, “Snowball sampling”, https://research-methodology.net/sampling-in-primary-data-collection/snowball-sampling to try to address this, asking participants to connect them with other people. To collect a large sample this way takes time and, in this case, did not yield a representative sample because the original participants were more connected with humanitarian actors and more likely to be well-educated males with connections to camp authorities.
Some data collection units offered small financial incentives to encourage participants to recruit others and lend their phones to under-represented groups. However, Rohingya refugees were hesitant to connect researchers to other people. Decades of persecution combined with high levels of corruption and insecurity in the camps and fear of repatriation, relocation to Bhasan Char, or family separation, makes it extremely difficult to gain trustCarlos Sardiña Galache, Burmese labyrinth: a history of the Rohingya tragedy, Verso, 2020; Allard K. Lowenstein International Human Rights Clinic, Yale Law School, “Persecution of the Rohingya … Continue reading. This mistrust and reluctance to share phone numbers is also partly due to the phone ban and random confiscation of SIM cards and phones. In the Rohingya refugee camps and the Bangladeshi host community, mobile phone ownership and literacy is low. Only 9% of Rohingya and 25% of Bangladeshi households own “extended assets”, a classification that includes mobile phonesWorld Food Programme, “Refugee influx emergency vulnerability assessment (REVA) – Cox’s Bazar, Bangladesh”, April 2020, … Continue reading. In households that do have phones, usage is normally controlled by the head of household, who is most likely male.
To overcome these challenges, the Needs and Population Monitoring (NPM) unit collaborated with the International Organization for Migration’s (IOM) Communicating with Communities (CwC) team to use its Interactive Voice Response (IVR) programmeCovid-19 INFO LINE is an Interactive Voice Response (IVR) programme run by IOM CwC to disseminate information and collect community feedback during the Covid-19 epidemic. IVR is a mass communications … Continue reading to identify participants and design data collection tools based on community feedback. Rohingya refugees using the IVR service to report issues were called back and the relevant needs assessment survey administered. The results gave humanitarians an understanding of needs and coping mechanisms of camp residents, and information was submitted to the agencies responsible for resolving specific issuesRelief Web, “IOM’s Needs and Population Monitoring Covid-19 NPM-IVR needs assessment”, July 2020, … Continue reading. However, the findings remained indicative and 90% of the respondents were maleIbid..
Privacy during remote interviews could not be assured. This was an impassable difficulty in the overcrowded and congested camps seeing increased rates of insecurity, corruption, sexual and gender-based violence, and intimate partner violence. Some questions could not be asked remotely in a safe manner, and sensitive topics had to be removed or worded generally. Humanitarians were thus unable to gauge the scale of specific issues, only to identify their general existence. Distrust and positive bias also meant perception questions were difficult to ask over the phone and produced very different results than when asked in person.
Small scale data collection through essential programming staff
Implementing agencies need granular information regarding specific and immediate needs and the impact and progress of active programmes. This is normally collected by monitoring and evaluation (M&E) units. Due to Covid-19, M&E activities were put on hold. Instead, critical programming staff and Rohingya volunteers conducted small-scale qualitative KIIs. However, they could only manage to conduct a small number of interviews per day as this was not their primary role. Many programming staff also do not receive extensive training on enumeration, qualitative data collection, or accurately recording and transcribing KIIs. Some households declined to participate in data collection activities out of fear of catching Covid-19, regardless of the prevention measures employed. External staff who continued working in the camps reported initial resentment and fear towards them. Data collection by staff implementing essential services also impacted responses, especially when collecting sensitive information. Despite these challenges, BBC Media Action and TWB were able to set up a rumour tracking system leveraging critical programming staff with access to the camps to understand the uptake of messaging and mitigate potentially harmful rumours.
Some households declined to participate in data collection activities out of fear of catching Covid-19.
Although face-to-face data collection can address some language, cultural and communication barriers, challenges that existed prior to the Covid-19 pandemic remained. Most interviews continued to be conducted by Bangladeshi staff in ChittagonianAll Rohingya refugees speak Rohingya and some also speak Burmese. In the learning centres, children are taught English and Burmese. Chittagonian is the language spoken in Cox’s Bazar and is similar … Continue reading. Rohingya refugees respond differently to sensitive and perception-based questions depending on whether the enumerator is Bangladeshi or RohingyaREACH, Participation of Rohingya Enumerators in Data Collection Activities, April 2019, … Continue reading. Women are less likely to be educated and to speak languages other than Rohingya, impacting their ability to engage in data collection exercisesTranslator without Borders, “The Language Lesson: What we have learned about communicating with Rohingya refugees”, December 2018, … Continue reading.
Rohingya refugees as researchers
IOM’s CwC unit was able to continue face-to-face data collection by using Rohingya field researchers who were trained in qualitative data collection methods and transcription before the pandemic. Training included participatory methods, participant observation, thick description, and open and semi-structured interviews. When external access to the camps was blocked, this predominately Rohingya team continued consultations on community perceptions, raising Rohingya voices during critical planning periods for the Covid-19 response. Small FGDs were held and interviews recorded and transcribed before being verified and analysed by the IOM CwC unit. The use of Rohingya refugees as researchers helped overcome some of the pervasive trust and communication challenges, allowing the Rohingya to communicate in their native language and through open-ended discussions without the presence of external humanitarian responders. The researchers were also able to reach specific vulnerable groups, such as those without access to phones and with less access to public spaces (often women and people with disabilities).
However, theoretical saturationThe use of the term saturation is referring here to “the point at which the researcher does not find new information or themes in new data collected”. and coverage of different demographic groups across the camps remained challenging. To comply with Covid-19 prevention measures and reduce the risk of transmission, a combination of purposive and convenience sampling was used and the number and frequency of data collection rounds was reduced. This meant large sections of the population were inaccessible if no Rohingya researchers lived in those areas.
Lessons for data collection
Technology and innovation are “go-to” solutions when physical access is restricted. However, in the Rohingya refugee context the solution lies in developing an in-depth understanding of the population, increasing community engagement and participation in data collection processes, and investing in assessment preparedness to ensure that qualitative and quantitative data collection exercises include the most vulnerable and hard-to-reach populations.
All key informants (KIs) recognised that remote data collection, though necessary during the Covid-19 response, is not the preferred or most appropriate data collection method. Not only did operational challenges impact the effectiveness of this method, but high levels of distrust resulted in extreme reluctance to speak over the phone or share contact information. Remote data collection teams also struggled to include vulnerable groups, such as women and girls, older people, people with disabilities, and people who are generally less vocal. Poorer refugees were also left out of most phone-based data collection because of the financial constraints around phone ownership. Some KIs reported collaborating with non-governmental organisations who work with vulnerable groups to reach a more diverse sample during remote data collection while others relied on trusted Rohingya with phones to locate different demographic groups within their social networks. However, these solutions could not be implemented at the necessary scale. Remote quantitative data collection alone failed to produce a cohesive narrative that accurately represented the situation, focusing on the “here and now” instead of the “how and why”. Challenges around trust also meant that even information on the here and now was limited to basic immediate needs and perceptions. Behaviour could not be accurately captured.
Lack of contextual knowledge about Rohingya social dynamics and sociocultural beliefs meant the response struggled to understand Rohingya behaviours, impacting the response’s ability to implement Covid-19 programming in line with Rohingya needs. The pandemic shined a light on major challenges and shortfalls in the humanitarian response in regard to data collection while also catapulting emerging good practices such as an emphasis on trust-building, the use of trained Rohingya researchers, and the introduction of more rigorous quality control methods. This provided humanitarians with an understanding of the cultural nuances and drivers of needs to improve engagement and develop an understanding of Rohingya perceptions about sensitive topics. The reduced capacity of humanitarians to conduct large-scale representative data collection showed that a combination of smaller primary data collection exercises (both qualitative and quantitative) can provide an overall picture of a complex reality.
The use of Rohingya refugees as researchers helped overcome some of the pervasive trust and communication challenges.
The more complicated the data collection environment, the greater the need for comprehensive, high-quality, real-time information and context-appropriate data collection methods. This requires well-trained teams and a better understanding of research methods. Lack of investment in long-term training combined with low literacy rates, no agreed script for the Rohingya language, restrictions on employment, and restrictions on refugee movement both within and outside camps meant that when the pandemic hit, there was a lack of trained Rohingya enumeratorsREACH, Participation of Rohingya Enumerators…, op. cit.. Developing a team of trained Rohingya researchers takes more time and investment than building an external data collection team, and restrictive gender norms and security risks makes recruiting female researchers and ensuring their safety while working difficult. However, the value of a Rohingya research team during the Covid-19 pandemic showed that building such teams is not only possible with investment, but essential.
The localisation of research and data collection is important. Without a strong push for the incorporation of strong ethical standards, increased coordination, and investment into data collection preparedness, the shortcomings revealed by Covid-19 are unlikely to result in improved practices. Data collection must accommodate the preferences of Rohingya refugees on how they would like to tell their stories and report their needs. To continue with face-to-face qualitative data collection, response actors should invest in and recognise the capacities of Rohingya refugees to work as researchers and enumerators. Ongoing investment in training will not only guarantee access when entry is restricted but should result in Rohingya researchers taking the lead in research design.
The value of a Rohingya research team during the Covid-19 pandemic showed that building such teams is not only possible with investment, but essential.
For remote quantitative data collection to be useful and effective when required, a representative phone database needs to be established in advance. This would address issues around sampling. Another element of assessment preparedness is testing and training, and the affected population needs to be sensitised to remote data collection methods. Teams should be trained in remote data collection before the need arises, and remote research design (including sampling and questionnaires) needs to be trialled in advance. The increased use of call backs and cross-checking of KIIs and verification through secondary data reviews will continue as the KIs found it increased their data quality. We must build upon these gains, strengthening coordination between data collection actors and improving data sharing practices to reduce duplication, ensure data is used to its fullest, and improve overall data collection quality and standards.
ISBN of the article (HTML) : 978-2-37704-745-1