Protecting vines with AI

Their interest in AI and machine learning brought the two then-school students to the federal AI school competition, BWKI, in 2019, where they met and began working together on ideas to shape the real-world using AI tools. Their heart's desire was quickly found: "Where we both come from, pesticides are used widely in vineyards, polluting our environment and harm human health," explains Mario. Their goal was to significantly reduce the use of pesticides by means of AI. Together they developed the app "Vine Leaf Diseases and AI", with which they won the special award for sustainability first prize at the BWKI competition in 2020. The app enables winegrowers to detect and identify leaf diseases on grapevines through mobile phone images and thus reduce the use of pesticides.

The approach was image recognition and processing. Maria-Theresa Licka and Mario Schweikert started collecting a lot of image data of pest infestations, nutrient deficiencies, and healthy vine leaves. They visited and observed vineyards during different vegetation periods and took photos with various smartphone models. "In order to recognize patterns correctly, we created an image processing pipeline, at the end of which is a “TensorFlow” model based on machine learning," says Mario. Numerous image data matrices from the wide-ranging database are used for training. However, the final model should not contain too many layers and parameters to achieve fast performance. Vineyards often have limited network connectivity. For this reason, the team wanted the app to run offline. "We compressed our data pipeline to balance efficiency and accuracy," explains Maria-Theresa. They used the “tflite” format, which compresses the model and optimizes it for edge computing on the mobile device.

The data processing pipeline is the heart of the app’s AI. First, the image data is scaled to 512 by 512 pixels, as the machine learning model can already detect minor discolorations with this resolution, but also needs a reasonable classification time. Subsequently, the image data is normalized in a range from -1 to 1. "At this point, we would like to add a segmentation algorithm in the future, which we are currently developing to focus on the grape leaf," says Mario. In the next step, the AI checks whether the image content is a grape leaf or not to minimize application errors. Then it is fed into the multi-class machine learning model. To train this, the system uses data augmentation to further diversify the dataset by changing the image parameters of the training data, such as angle or brightness. "We also use transfer learning to reduce the number of parameters that can be trained," says Maria-Theresa and adds: "By activating the last layer with the SoftMax function, we get a class probability that allows us to detect multiple infestations of a vine leaf."

This is how the app works:

• The user takes a photo of a potentially infected vine leaf.
• The app provides feedback on which leaf diseases, pest infestations or nutrient deficiencies have been detected.
• The app reports the type of pest infestation, image data, location and time stamp of the photo to the server.
• The local spread of leaf diseases is thus detected at an early stage and made available to the winegrowers as a geographical map in the app.
• The winegrowers can thus take suitable measures against the pest infestation at an early stage.
• Large-scale spread and the use of pesticides can be avoided.

The app can be downloaded from the Google Play Store and Apple App Store, is free of charge and available in several languages. "We now have more than 750 users worldwide," says Mario. Professional winegrowers use it just as much as hobby winegrowers. While the users at the start of the app were mainly from Germany, the app's geographical map shows that it is now also used in the Netherlands, Spain, Luxembourg, and Italy as well as the USA and countries in Africa. Maria-Theresa and Mario are not resting on their laurels, however. "One of our next steps is to train the AI for rare pests in the vineyard and to provide useful advice, especially for amateur winegrowers, on which pesticides should be used to combat infestations," says Mario. Maria-Theresa adds, "We want to make the app more popular so that it is used more often around the world, as this can significantly reduce the use of pesticides."