Picture-based crop insurance: is it feasible?

Using farmers’ smartphone pictures to minimize the costs of loss verification

Berber Kramer, Francisco Ceballos, Koen Hufkens, Eli Melaas, Azad Mishra, Micheal Mann, Mann S. Toor & Miguel Robles


The Picture-Based Crop Insurance (PBI) project aims to develop a new way of delivering affordable and easy-to-understand crop insurance using farmers’ smartphone pictures to minimize the costs of loss verification. The project, funded by the CGIAR Research Program on Policies, Institutions, and Markets(PIM), is a partnership between the International Food Policy Research Institute (IFPRI), the Borlaug Institute for South Asia (BISA), HDFC Ergo General Insurance, Limited, and researchers from The George Washington University, Boston University, and Ghent University.

Millions of smallholder farmers lack access to affordable insurance – their farms are simply too small and too remote for insurers to affordably verify damage on insured farmers’ crops. By taking regular pictures using their own smartphones, farmers can reliably document damage after a natural calamity and provide evidence that the crop was managed appropriately until that point. This brings down the costs of loss verification substantially. Instead of sending an insurance agent to verify a farmer’s claim, insurance companies can appraise losses by simply processing the smartphone pictures, and can even rely on advances in image processing techniques to help automate the loss assessment procedure. In other words, PBI could directly provide insurers with eyes on the ground, at limited cost. While the potential for such a system is vast, it is critical to first assess its feasibility.

The PBI project has been underway since 2015 to address the following questions: (1) Can farmers take regular and consistent pictures of their fields using their own smartphones for loss assessment purposes? (2) To what extent is damage visible and quantifiable in smartphone pictures, and what types of damage are visible? (3) Does picture-based loss assessment capture damage that weather index-based insurance products do not capture? (4) Do farmers strategically reduce efforts or tamper with pictures to receive payouts when they have PBI coverage (in other words, does PBI induce moral hazard)? (5) Does PBI increase the demand for crop insurance?

This project note describes the results from a formative evaluation of PBI in six districts of Haryana and Punjab, India. Here, we focus on the first three questions, which all relate to measuring damage using smartphone pictures. These are more general questions that not only have implications for the design of insurance products but are also of interest to other institutions interested in measuring crop damage (for instance, statistical agencies and agro-advisory service providers). The last two questions are relevant mainly in an insurance context and will be discussed in a separate project note (forthcoming).

Methods and data

The study focused on villages near 25 weather stations in selected districts of Haryana and Punjab, India (Fatehgarh, Ludhiana, and Patiala in Punjab; Fatehabad, Sirsa, and Yamunanagar in Haryana). Specifically, within a radius of five kilometers from each weather station, two villages were randomly selected and 15 farmers per village were invited to participate. These farmers were randomly selected from a list of all farmers within the village who satisfied the following criteria: (1) having less than 15 acres of operational farmland and (2) planning to grow at least two acres of wheat during the upcoming Rabi season.

Of these invited farmers, 592 (approximately 12 farmers per village) agreed to participate in the PBI study. In October 2016, participating farmers received insurance to cover one acre of their wheat crop during the upcoming Rabi (winter) 2016-2017 season. Insurance was conditional on taking regular pictures of their plots from sowing to harvest using WheatCam, a smartphone app developed for the project. In addition, data plans were provided for farmers to upload these pictures. Farmers were told that their insurance included coverage against damage visible in their WheatCam pictures (PBI) and coverage against excess rainfall and above-normal temperatures between February and April.

Overview of procedures

  1. Before cultivation: The farmer downloads the app, enrolls in insurance, and takes an initial overview picture of the insured site, including identifiable objects in the background. The app uploads the picture with its location.
  2. During cultivation: Every few days, the farmer takes geotagged pictures with the same view as the initial picture, in addition to close-up pictures when the crop has suffered any damage.
  3.  After cultivation: All pictures (including both the time series of overview pictures and the close-up pictures) are analyzed along with auxiliary data to verify losses.
  4. After loss verification: Farmers for whom the pictures or auxiliary data show damage are sent an insurance payout
To enroll, farmers took an initial overview picture of their plot, facing north, with an identifiable object in the background (for this season, a reference pole placed in the cultivated field was used, although alternative objects, such as trees or buildings on the horizon, could be considered to fix the reference frame). Farmers also received a low-cost auxiliary pole that acted as a tripod upon which to place the phone; this helped fix the position for taking repeat pictures

Repeat pictures

After taking the initial picture and throughout the entire season, farmers were asked to take three repeat pictures per week between 10am and 2pm (in order to preserve lighting conditions), from the exact same location as the initial picture and with the same view angle every time.

To facilitate this, the smartphone app included geotags to check whether the repeat picture was taken at the same location as the initial picture; it also provided visual aids in the form of a “ghost” image (a partially transparent image of the initial picture) that allowed the farmer to align static features in the landscape (such as distant trees or structures) and the reference pole with those in the initial picture, thus ensuring an almost identical view frame throughout the season. Valid pictures were then uploaded to an online server and processed by the research team.

Figure 1: Map indicating the study areas in Haryana and Punjab

Figure 2: Ghost image of initial picture is shown when taking repeat picture

Loss assessment

At the end of the season, an independent panel of wheat experts evaluated the time series of pictures for each farmer. They assessed whether there had been any damage to the crop and, if damage had occurred, they determined the percentage by which the crop had been damaged. Assessments were first done individually. When large disagreement existed between the different experts’ assessments, experts would jointly review a case and agree upon a final damage assessment; otherwise the median assessment across experts was considered for insurance payouts. For farmers with more than 20 percent of assessed damage, a damage report including the pictures and the expert loss assessments was sent to HDFC, the project partner underwriting the insurance product, to issue payments directly into farmers’ bank accounts.

For the 2016-2017 season, the project used these expert assessments as a transparent, pragmatic solution to providing PBI coverage in the absence of automated tools to estimate damage from the time series of pictures. One of the future objectives of the project is to develop an image processing algorithm using the data from the 2016-2017 evaluation and subsequent seasons to automate loss assessment. Collecting more imagery and training data to develop these algorithms not only for wheat but also for different crops is an important next step. The end result from these data collection efforts will constitute a valuable public good which private insurance companies will be able to use to improve their products.

Crop cutting experiments

At the end of the season, researchers visited the photographed plots to conduct crop cutting experiments for yield measurement. For each plot, the researchers sampled two different square meters that were visible in the pictures: one to the left of the reference pole and one to the right of the reference pole. The heads of the wheat plants falling inside these sampled square meters were threshed, the resulting grains were weighted, and the average weight from these two square meters was used to calculate yields per acre. These yield data were not used as input in the loss assessments; the primary reason for collecting these data was to have an objective measure of yields, a critical step for assessing the validity of loss assessments based on farmers’ own smartphone pictures.


1. Can farmers provide a time series of pictures to be used in loss assessment?

The first prerequisite for such an insurance system to be feasible is that farmers be willing and able to take pictures of their fields regularly and with a sufficient level of quality. Out of the 592 farmers encouraged to take pictures, 475 farmers (80.2 percent) uploaded at least one valid picture during the season. Of the farmers who took pictures, the large majority (more than 83 percent) took at least six pictures throughout the season—or roughly one picture per month— while more than 59 percent of them took pictures twice a month or more.

Greenness levels and other features of the pictures are most comparable across time when lighting conditions are held constant. This is why farmers were asked to take pictures between 10am and 2pm; however, if they were unable to make it in that window, the app allowed them to take a picture at a different time. Farmers took pictures across a broader range of times than initially requested, in part because most plots are not located close to the farmer’s house, meaning that they generally visit their plots only at certain times of the day. Loss assessment algorithms will have to factor in this ground reality. Finally, we present the number of farmers who took at least one picture in a given calendar week throughout the season.

The pattern is encouraging, with sustained submissions from an average of 200 farmers weekly, except for the beginning of the season (when the wheat plants had not started growing yet and when farmers were facing technical challenges with the app) and the post-harvest period (when farmers no longer had to take pictures).

Figure 3: Number of uploaded pictures for farmers who took pictures

Figure 4: Number of pictures by time of the day

Figure 5: Number of farmers uploading at least one picture by week

In summary, while farmers did not strictly follow the requested protocol, they were able to submit a substantial number of pictures for loss assessment. Moving forward, a more flexible protocol for the number of pictures required per week will be needed, especially early in the season when the crop is not showing yet. It could also be important to make the benefits of taking pictures more salient to farmers, as they may not see the direct value of doing so in the absence of damage. In this regard, bundling the insurance product with other services, such as agro-advisory services(for instance, irrigation advice) and pest detection could help boost farmers’ interest in taking pictures more regularly, which will improve outputs from loss assessment. Overall, however, farmers appear able and willing to take pictures for loss assessment purposes themselves

2. To what extent is damage visible in pictures? What type of damage is visible?

The second prerequisite entails that damage arising from different types of hazards be plainly visible in a picture taken at a distance of a few meters. Initial conversations with local wheat agronomists indicated that pictures would be able to capture most — though not all — hazards. Certain events such as lodging (the bending of the wheat plant due to winds and wet, loose soil), hail, or certain common wheat diseases such as yellow rust would indeed be visible. Other events, such as blight or high temperatures, which can affect grain filling without showing a direct effect on the external aspect of the plant, would be much more difficult to identify. Farmers’ perceptions measured through surveys were in agreement with this. To cover farmers from high temperatures, we combined picture-based insurance with a weather-index based insurance product. Figure 6 shows a box plot of the loss assessments, ordered by the median assessment within a site. A few interesting patterns can be seen from this figure. The level of agreement between experts is quite high for low levels of damage (under 20 percent). Even for sites with more severe damage (over 20 percent), most experts agree over the approximate region in which the damage occurs.

These results are indicative of crop experts being able to identify crop losses from direct visual inspection of pictures.The above results could potentially be improved through a loss detection algorithm that considers not only the pictures themselves but also the development of greenness and texture indices over time, as well as close-up pictures and localized weather information. Texture indices can measure how upright the crop is and are thereby a potential way to capture lodging and hail storms occurring too late in the growing season for the crop to recover. Coarser low-cost satellite imagery currently available would be unable to detect such events, and even higher-resolution microsatellites may lack the advantage of close-up ground-based view angles. In addition, microsatellite imagery is less well-suited for the insurance problem at hand due to its cost, visibility issues (i.e. clouds), and reduced interpretability of the images.

Figure 7: Yields for farmers in different PBI payout categories

It is important to note that for insurance purposes, it is not necessary to translate pictures into precise estimates of crop damage, particularly at very low levels of damage (when insurance does not pay out). As a final way to assess whether the pictures contain sufficient information for loss assessment, we compare yields – measured through crop cutting experiments – for farmers with different levels of insurance payouts (median expert loss assessment below 5 20 percent, between 20 and 50 percent, and above 50 percent).
Figure 7 shows the average yield for farmers without a PBI claim to be around 20 quintals per acre, with a significantly lower average yield of 18 and 10 quintals per acre for, respectively, farmers in the first and second or third PBI payout categories. In other words, through visual inspection of the pictures, experts were able to accurately identify yield losses at a scale that is relevant for insurance purposes.

3. How does PBI compare to index insurance?

A final question is whether picture-based loss assessment adds value to existing cost-effective ways of estimating crop damage. An alternative that has gained popularity over the past decades is weatherbased index insurance, which bases its payouts on the value of an index measured from a weather station. Removing the need for farmer’s involvement and the back-end system necessary for PBI, this alternative is certainly cheaper than traditional indemnity insurance; however, it measures weather at a location often several kilometers away from a farmer’s plot and depends on identifying an appropriate relationship between weather and crop growth, which may not always reflect a farmer’s true losses. Here, we compare the performance of picture-based loss assessment with that of a weather index-based assessment in order to understand whether the additional costs of a PBI approach can be justified.

The weather-index based insurance (WBI) product covered the insured farmer from higher-than-normal night temperatures and (unseasonal) excess rainfall at the end of the wheat season, from February to April. The indices relied on daily minimum temperature and rainfall collected at 25 nearby weather stations (located within five kilometers of the study villages) and were developed based on Focus Group Discussions with farmers and Key Informant Interviews with local wheat agronomists. For each index, triggers were rounded values of the 70th percentiles in historical records for one weather station in Haryana and one weather station in Punjab.

Figure 8: Yields for farmers without and with WBI payouts

Figure 8 compares the average yields from crop cutting experiments across farmers who would and would have not received payouts from this WBI product. Given the effort and care that was put into designing the weather index-based product, the result is very disappointing. The yields from farmers for whom WBI would have triggered a payout are virtually indistinguishable from those for whom no payout would have been triggered. This is indicative of a very large degree of overall basis risk, where actual losses do not correspond with insurance payouts. Having to pay farmers in years that they have normal yields will be reflected in the price of the insurance product. At the same time, these results indicate the value that picture-based loss assessment can have by reducing the degree of basis risk characteristic of more standard WBI products.


Picture-based crop insurance is a new approach to improve smallholder farmers’ access to affordable but high-quality insurance. By leveraging increasing smartphone ownership among smallholder farmers, and relying on automated image processing techniques, the goal of PBI is to combine key advantages of index insurance – fast and inexpensive claims processing – with those of indemnity insurance – low basis risk and easy-to-understand products. The picture-based insurance project is the first formative evaluation of this potentially game-changing insurance approach.

Based on the formative evaluation of this novel PBI product, we find that farmers are largely able to follow picture-taking protocols. Further, agronomists and farmers agree that the most important risks in wheat production are plainly visible in pictures, and experts are indeed able to detect such damage in the pictures. Moreover, picture-based insurance payouts are better correlated with yields than weather-index based insurance payouts, indicating that the pictures captured crop damage better than the indices behind the weather index-based insurance product — despite careful efforts in designing this latter product.

The study also highlights areas for further research. First, although farmers are able and willing to take enough pictures for loss assessments, there is room for improvement. In the formative evaluation, communication was one-way, from the farmer to the project. Future efforts could concentrate on bundling picture-based insurance with picture-based agro-advisory or pest-detection service in order to make the process more inclusive and make the benefits of taking pictures more salient to farmers.

Second, loss verification through agronomic experts is not an economically viable model in the long run; automated image processing algorithms need to be developed to lower the costs of loss assessment through pictures. This formative evaluation collected valuable initial data to develop such algorithms, although more data will be required to improve performance. In this regard, research efforts to collect pictures of crops along with measures of damage and yields can prove to be very valuable.

Third, this note focused on the technical feasibility of PBI. A second project note describes in detail the feasibility of the scheme from an economic point of view. That note addresses two questions raised earlier: 1. Do farmers strategically reduce efforts or tamper with pictures to receive payouts when they have PBI coverage (in other words, does PBI induce moral hazard)? 2. Does PBI increase the demand for crop insurance? Formative evaluation results are promising in this respect and highlight the costs and benefits of the PBI approach.

PBI has the potential to bring about important changes in the way that insurance is offered to smallholder farmers. Our first results indicate that this is a promising approach that will help protect farmers from the increasing risk of extreme natural hazards and calamities posed by climate change. Being able to rely on this protection could help farmers better invest in their farms, improve their yields, become more resilient, and move toward a better future.


Take-away messages

  • Farmers are able and willing to take enough pictures of sufficient quality for loss assessment. Bundling this with agro-advisory services can potentially improve compliance.
  • Damage is visible from smartphone pictures and can be quantified by agronomic experts. This paves the way for algorithms that automate loss assessment procedures.
  • Picture-based loss assessments are strongly correlated with yields and improve upon weather index-based measures that were carefully designed to capture damage.