Humanitarian artificial intelligence begins before the model: lessons from KoboToolbox for responsible digital commons

Who governs artificial intelligence used in the name of crisis-hit pop­ulations? And who can say “no” to it? The co-founders of KoboToolbox offer lessons from their own platform about a form of humanitarian artifi­cial intelligence that is governed – not endured – by those it is supposed to help.

Artificial intelligence (AI) is no longer on the doorstep of the humanitari­an sector. AI is already in the sector’s project proposals, narrative reports, translations, interview summaries, monitoring dashboards and coordina­tion memos. It is used to put together an organisation’s forward-looking devel­opment plan, summarise an assessment, translate a personal account or analyse open-ended responses.

AI is here to stay because of its usefulness

AI is spreading through the sector because it addresses a real-life chal­lenge: how to do more with less. Needs are increasing, funding is contracting, re­porting requirements are mounting up and teams are wearing themselves out. AI seems to be a practical, instant and almost self-evident solution in this con­text. Indeed, it is becoming the technolo­gy of humanitarian austerity: it promises efficiency for a sector having to alleviate more suffering with fewer resources.^[1]United Nations Office for the coordination of humanitarian affairs (OCHA), Global Humanitarian Overview 2026, 8 December 2025, … Continue reading

AI’s use is therefore no longer up for debate. It is already in the build­ing. In a global survey conducted in 2025 by Data Friendly Space and the Humanitarian Leadership Academy of 2,539 respondents from 144 countries and territories, 93% of the respondents had already used AI tools, while only 22% stated that their organisation had a formal AI policy. A follow-up survey con­ducted in January 2026 confirmed the trend: 95% of the respondents used AI tools, but less than one-quarter worked in an organisation with a formal policy.^[2]Data Friendly Space and Humanitarian Leadership Academy, 2026 AI Pulse Survey Findings [briefing report], 19 March 2026, https://www.datafriendlyspace.org/resources/2026-ai-pulse-survey-findings

The choice is therefore not between opt­ing for or rejecting AI, but between allow­ing the most opaque systems to become the default infrastructure in humanitar­ian action and developing safer, more open and better-governed alternatives.

This article does not deal with all of AI’s effects on humanitarian work. It does not focus on fake news, cyberattacks, AI as civilian infrastructure exposed to conflict or all the geopolitical changes brought about by this technology. Its aim is narrower: how AI is incorporated into humanitarian practices, particu­larly through data collection, analysis, needs representation, consultation and decision-making. The core issue is not so much model performance but, rather, rights, power and responsibility when this technology is actually used in practice.

The lesson from KoboToolbox – infrastructure is just as important as the tool

The experience of KoboToolbox (see box) sheds light on this debate, though the analogy does have its limits. KoboToolbox is not AI. Its story shows how a humanitarian technology can es­tablish itself when it tackles a clear op­erational issue, while reminding us that a responsible tool does not only define itself by its value in use. Digital data col­lection has reduced some costs, limited data entry errors, improved data quality and made data collection accessible to teams who did not have the resources to develop their own systems. However, a free open-source tool does not auto­matically become trustworthy. It needs to work in challenging conditions, be maintained, enable data to be checked, and be included in training, support, doc­umentation and governance practices. This lesson applies to AI: a humanitarian technology only becomes responsible if it is produced, accessed, overseen, main­tained and used responsibly.

KoboToolbox and humanitarian AI

As we are gradually integrating AI into the platform, KoboToolbox is a good illustra­tion, as well as a trial, of this transition. This is taking very tangible forms: fa­cilitating and speeding up interview transcription, translating responses, pro­cessing qualitative data, helping create forms, and developing smart surveyors able to chase up, ask for clarification, de­tect an incomplete response or suggest a follow-up question – this is particularly useful in certain scenarios, for example an Ebola epidemic. Some of these fea­tures – specifically transcription, trans­lation and qualitative analysis of audio responses – are already available.

These uses address the very core of hu­manitarian work, at least with regard to knowledge production. In a survey, response quality hinges on the lan­guage, context, confidence and ability of the surveyor to recognise nuance. AI that supports interview transcription, translation or monitoring can signifi­cantly reduce workloads and improve some aspects of data quality. Yet AI can inconspicuously shift the place where meaning is constructed.

While AI helps create forms, translate, transcribe, code responses or conduct an interview, it also influences what will be asked, retained, grouped together or highlighted. So, the aim is to incorpo­rate AI without losing the strengths of tools such as KoboToolbox, namely user control, data protection, visible limits, human oversight, documentation and adjustment attuned to the situation on the ground.

The risk is that the production of human­itarian knowledge is shifted to systems that the field teams, local partners and affected populations cannot inspect, influence or challenge. AI can support the tools, collection and analysis. It should not become a silent authority on what is worth asking, understanding or retaining.

The risk of AI by default

So, the danger is not that humanitari­an workers use AI. Rather, danger aris­es if they use it through infrastructure they do not control. In practice, the “default” AI will often be the quickest, least-expensive and best-integrated into existing tools, or simply the only AI avail­able. Very often, it will be a commercial tool hosted by private infrastructure and governed by usage conditions that few organisations really read, let alone negotiate.

“Digital tools are never neutral instruments: they reshape power relationships, institutional practices and possible types of action.”

Yet it would be unfair to conclude that teams are acting irresponsibly. Many use AI with care and good intentions. Still, asking people working under pressure to reject useful tools without a safe al­ternative is not a governance policy. It would be shifting responsibility for the matter. If the humanitarian sector does not develop its own usage conditions, others will develop them for it: provid­ers, funders, budgetary constraints and informal habits.

The risk is not specific to AI. Critical work on humanitarian technology has already shown that digital tools are never neu­tral instruments: they reshape power relationships, institutional practices and possible types of action.^[3]Kristin Bergtora Sandvik, Maria Gabrielsen Jumbert, John Karlsrud et al., “Humanitarian technology: a critical research agenda”, International Review of the Red Cross, vol. 96, no. 893, 2014, p. … Continue reading But AI is mak­ing this issue more acute as it does not merely collect or organise information.

When consultation becomes simulation

AI does not just speed up data col­lection: it can also get involved in interpretation, representation and decision-making. Worse still, AI can generate an ostensible response with­out anyone having been asked. The “new generation” KoboToolbox does not seek to replace consultation with generated responses, simulated communities or automated decision-making. Instead, the intention is to use AI for the tasks it is well suited to: helping phrase ques­tions better, facilitating data collection, speeding up analysis and organising re­sponses. AI is therefore used to enhance consultation processes, rather than re­place them.

Indeed, humanitarian data is never a purely technical matter. A survey, in­terview, discussion group, complaints mechanism or community consultation are encounters shaped successively, or simultaneously, by trust, fear, language, power and institutional expectations.^[4]Patrick Vinck and Phuong N. Pham, “Consulting Survivors: Evidence from Cambodia, Northern Uganda, and Other Countries Affected by Mass Violence” in Steve J. Stern and Scott Straus (eds.), The … Continue reading Before an AI model can predict needs, classify situations or recommend ac­tion, people’s experience has already been turned into data: a response to a survey, GPS coordinates, a photo, a per­sonal account, a vulnerability category, an indicator of need, a complaint or an interview note.

AI then increases this conversion. It al­lows large volumes of data to be pro­cessed a lot more quickly, allows the data to be linked to other sources and helps set categories, profiles and priori­ties that were not necessarily envisaged when collecting the data. Data collected to gain an understanding of a crisis or improve a programme can be reused, summarised, combined, modelled or simulated in unforeseen contexts. So, data extraction is not the only risk. The risk is of intelligence extraction: turning populations’ experience into models that are useful to the humanitarian sys­tem without giving them any say in what these models highlight, erase or deem to be priorities. Research on human­itarian data protection and metadata has already shown that the digital trails generated by humanitarian work can expose those concerned to secondary usage, surveillance and risks they can­not anticipate or control.^[5]Alessandro Mantelero, “Artificial intelligence”in International Committee of the Red Cross, Handbook on data protection in humanitarian action, chapter 17, Cambridge University Press, 2024, … Continue reading For instance, an AI system could analyse safeguarding reports, survey responses and location data to allocate a risk level to certain households. A classification error could affect access to help, a monitoring visit or safeguarding measures, without those affected knowing how this classification has been generated or being able to cor­rect or challenge it.

Community consultation becomes even more important in this scenario. The use of large language models (LLMs) to sim­ulate a consultation with the affected communities may seem efficient: why arrange a long, costly consultation if a model can generate responses that purport to reflect a group’s probable reactions? This temptation must be called out for what it really is: a sub­stitution. A model trained to imitate affected populations does not allow them to have their say on the matter, while providing institutions with a sub­stitute that costs less than listening to their views.

Even in the case of only trialling a service or digital tool, specialists rec­ommend limiting use of AI-generated profiles – supposed to simulate real us­ers – to simply formulating hypotheses, without using them to replace discus­sion with real people or as the basis for final decisions.

The value of consultation is not solely limited to the information it gener­ates. Its value also lies in the process itself: meeting with the people con­cerned, listening to their views, poten­tial disagreement, trust and recognition of the other person as a partner in the process rather than a source of data. Replacing this relationship with a sim­ulation would amount to conflating in­formation trails and political presence.

The affected populations not only have an interest in being better represented. They are also entitled to be heard, to re­ject some forms of representation and to challenge how data is used or accounts are produced using their experience as a basis. This requirement is in accordance with both the principles of a human rights-based approach to data, human­itarian accountability commitments made to the affected populations and research on data justice, which focuses on how people are made visible, repre­sented and treated via the data generat­ed about them.

Decision-making can not be automated

Finally, use of AI raises a decision-making issue. AI is often portrayed as a tool to aid analysis, prioritisation and planning. And this is often true. Yet help can quick­ly turn into delegation in organisations under pressure. A recommendation gen­erated by a system, particularly when in­cluded on a dashboard or aligned with a funder’s expectations, can take on dis­proportionate authority.

The difficult decisions in the human­itarian sector are not just optimisa­tion issues. Who should be helped first? Where and with whom should ac­cess be negotiated? How can a balance be struck between speed, impartiality, safety, dignity and protection? These questions harness data, but cannot be answered by data alone. They involve values, responsibilities, accepted or re­fused risks, and sometimes the courage to not implement the most seemingly efficient solution.

These machines cannot provide a certain portion of humanitarian judgement: hes­itation, doubt, empathy, and sometimes even a form of prudent irrationality. These human forms of slowness are not always shortcomings. They can be what prevents a quick decision from becom­ing irreversible.

AI can detect regularities, generate sce­narios, summarise information, flag up inconsistencies, and generate hypoth­eses. It can help humans think. But it cannot take on the moral responsibil­ity for a decision, bear the burden of decision-making or be accountable to a community. This is why the concept of “keeping the human in the loop” is not enough if it does not go hand-in-hand with a true capacity to intervene and challenge and with real institution­al responsibility. Research on human oversight of automated systems shows that this oversight can become mere­ly symbolic when it does not actually take into account the automation of practices, human deference to systems or real human inability to challenge an algorithm’s recommendations.^[6]European Data Protection Supervisor, TechDispatch #2/2025: Human Oversight of Automated Decision-Making, 23 September 2025, … Continue reading

The case for responsible humanitarian digital commons

Avoidance cannot be the answer: the humanitarian sector will not be able to prevent humanitarian workers from us­ing tools that meet real needs. The chal­lenge is to facilitate less dangerous uses and make riskier ones more difficult.

This involves shifting from a principles-based approach to an ap­proach based on a framework for tech­nology usage. Responsible humanitarian AI cannot just have a charter, training course or prohibitions list. It must op­erate within a practical framework that comprehensively links together usage types, risk levels, data harnessed, peo­ple affected, institutional responsibili­ties, appeals procedures and technical conditions for rollout. Recent research on humanitarian AI specifically insists on this switch from general ethics to oper­ational mechanisms: risk classification, validation phases, refusal conditions, post-implementation monitoring and accountability to the people affected.^[7]CDAC, “SAFE AI – A Governance Framework for Humanitarians using AI: How to turn SAFE AI principles into practical action. Version 1.1”, Reliefweb, 19 May 2026, … Continue reading

Such a framework should start by differ­entiating forms of usage. Not all forms of AI use pose the same level of risk. There is a major difference between summa­rising a public document and using AI to process sensitive personal accounts, classify safeguarding reports, direct as­sistance, prioritise households or simu­late a community’s preferences. Some usages can be encouraged with simple precautions. Others require enhanced humanitarian validation, risk assess­ment, public documentation, community consultation or a right to object. Some should simply be banned.

The framework should also focus on the technical conditions. The humanitarian sector does not need the most powerful AI model. The sector needs models that are sufficiently useful, lean, multilingual, well-documented, auditable and adapt­able in order to be governable. It needs environments designed for its practices: data classification, contextual warnings, limits on sensitive information, secure hosting, approved corpora, traceability and the option of working locally or of­fline. Openness may help but it is not enough in itself.

“An AI usage framework is only meaningful if it turns principles into effective rights and turns responsibilities into verifiable obligations.”

Finally, an AI usage framework is only meaningful if it turns principles into ef­fective rights and turns responsibilities into verifiable obligations. The framework should set the red lines, rules of procure­ment, incident procedures, audit mecha­nisms, risk thresholds and forms of appeal. It must guarantee the possibility of actual human review, the right to refuse certain forms of usage, the right to challenge a decision and, in some cases, the need for consent or collective governance. These guarantees are not just ethical preferenc­es: they are part of a broader development of the legal and normative frameworks relating to automated decisions, human oversight and risks to fundamental rights.

Humanitarian AI cannot solely increase the capacity of large international or­ganisations. It must also build the ca­pacity of local partners and affected populations: training, resources, acces­sible tools, less dominant languages, easy-to-understand documentation and the actual power to challenge.^[8]Read Jean-Baptiste Lacombe Lavigne’s article in this issue, “Can artificial intelligence provide leverage for local organisations?”, pp. 84–95.

This is where the full value of humanitar­ian digital commons comes into play. A common is not just free or open-source tool. This is an institutional agreement about a shared resource: who contrib­utes to the resource, who accesses it, who decides, who maintains, who ben­efits and who can challenge. Applied to AI, this idea forces the sector to ask a political question before moving on to a technical one: who governs the intel­ligence generated in crisis scenarios and in whose name? This concept is in line with the tradition of research on com­mons governance, which shows that shared resources require rules, institu­tions and accountability mechanisms tailored to actual forms of usage.

Humanitarian AI will not be made re­sponsible by its usefulness or by a handful of principles added as an after­thought. It will only become responsible if the sector can shape the conditions in which humanitarian AI is designed, rolled out, overseen and challenged. The challenge is not to go faster and at any price, nor is it to measure, model or simulate affected populations more and more. Instead, the challenge is to devel­op technologies that build the capacity of affected populations to be heard, to understand what is done in their name, to reject certain uses and to hold peo­ple to account. So, the real question is about determining who is served by AI, who governs it, who can challenge it and who can say “no” to it.

Translated from the French by Gillian Eaton