The SynField system has been installed in a cultivation area in the outskirts of Zalenieki, Latvia. This is the first smart agriculture system installed within the AgriDataValue project. SynField provides the capability to monitor and record the environmental and soil conditions prevailing in the area. The installation includes a meteorological station, a soil temperature sensor and a humidity sensor. The meteorological station records the ambient temperature, relative humidity, wind intensity, wind direction and rainfall. SynField smart agriculture systems will be installed in many European countries to collect valuable data and measure key environmental parameters.
The AgriDataValue technologies will be fully tested and validated during the AgriDataValue project lifetime through the projects’ Use Cases (UC) in 9 countries, representing more than 181,000ha with 25 types of crops that span from southwest to northeast Europe, outdoor and greenhouse crops, organic and non-organic production, and more than 2,000 animals of 5 types. In addition, more than 4,200 farmers will provide insights and more than 89,000 will be directly informed.
Within the projects’ Use Cases, SynField smart agriculture systems will be utilized to collect valuable data in order to optimize the quality and quantity of the crop production and increase environmental sustainability. Innovative technologies such as SynField will contribute to reducing the wasted irrigation water, fertilizers, pesticides, and consumed energy.
AgriDataValue participated at the ‘’Open company day’’ event hosted by the project’s partner Inagro in Beitem, Belgium. The event was attended by more than 3000 visitors, including farmers. During the event, a demonstration of the precision agriculture technologies, that will be incorporated within the project’s pilot activities was performed. More specificaly, unwanted herbs in a celeriac field detection using drones scenario and demo has been presented.
The domain of agriculture faces many challenges such as disease and pest infestation, suboptimal soil management, inadequate drainage, and irrigation, and many more. These challenges result in significant crop losses and environmental hazards, primarily due to the excessive use of chemicals. Several research have been conducted to address these issues. The field of Artificial Intelligence with its rigorous learning capabilities has become a key technique for solving different agriculture related problems. (2) Systems are being developed to assist – agricultural experts for better solutions throughout the world. (3) Among these systems, it is widely recognised that Artificial Intelligence (AI) techniques will play a key role in the immediate foreseeable future. In parallel, also the introduction of algorithms able to explain AI algorithms (also known as eXplainable AI or XAI) is deemed necessary. An interesting analysis on this subject was done by Masahiro Ryo (4) from whose paper the following diagram is taken, illustrating the AI/XAI usage trend.
The importance of adopting an explainable AI approach is also critical when the end users are farmers or agricultural companies. By providing an explanation of the AI’s decision-making process, farmers can gain better understanding of its functioning, increase their level of trust in the technology, and make informed decisions about how to optimize their operations. This can result in significant cost savings, as farmers can target their resources on the areas that will yield the greatest benefits.
Man and Machine Gap
In the past decade, we have witnessed a massive development in Machine Learning and related fields. Due to the increased availability of big datasets and generally more accessible powerful computing resources Deep Learning techniques experienced a boost that revitalized Machine Learning. The resulting super-human performance of AI technologies in solving complex problems has made AI extremely popular. As a matter of fact, however, this increase in performance has been achieved using increasingly complex models that are not interpretable by humans (Black Box AI). Furthermore, in specific contexts closely tied to human experiences, the utilization of AI algorithms is still viewed as risky and lacking in trustworthiness. For example, in healthcare AI applications that support clinicians’ decisions by suggesting a specific treatment for a patient based on predictions of complications. Or, in banking field, AI mechanisms that suggest whether a customer who has applied for a loan is creditworthy or not. In these contexts, it is essential that decision-makers have access to information that can explain why a patient risks a specific complication or why an applicant should not get a loan. But not only that: a mechanical maintenance operator will for instance be able to make little use of information about a future system failure if he or she does not also have indications of which component is beginning to degrade, just as a farmer should know in detail which s suboptimal management will cause a predicted crop failure so that he or she can take informed and targeted action.
The XAI bridge
Contributing to these results, XAI technology has grown exponentially in the AI domain, dealing with the development of algorithms able to explain AI algorithms, thus closing the gaps. Its relevance has become evident not only in academia, but also in industry and institutions. The European Union in the AI Act (1) has explicitly declared the ‘interpretability’ of AI to be a fundamental characteristic for all AI systems, and indispensable for those AI solutions deemed to be high-risk, to be used in European soil.
Explainable AI, also known as Interpretable AI, or Explainable Machine Learning (XML), either refers to an AI system over which it is possible for humans to maintain oversight, or to the methods to achieve this.
Main XAI techniques
In AI, Machine learning (ML) algorithms can be categorized as white-box or black-box. White-box models, sometimes called glass box models, provide results that are understandable to experts in the domain. Black-box models, on the other hand, are any AI systems whose inputs and operations aren’t visible to the user, or another interested party. The goal of the XAI is to ‘open up’ black boxes and make them as like a white box as possible. XAI algorithms follow the two principles of:
Transparency: when the processes of extracting features from the data and generating labels can be described and motivated by the designer.
Interpretability: possibility of comprehending the ML model behaviour and presenting the underlying basis for decision-making in a human-understandable way.
If such algorithms fulfil these principles, they provide a basis for justifying decisions, tracking them and thereby verifying them, improving the algorithms, and exploring new facts.
Over the years, numerous methods have been introduced to describe the operation of more ‘Black-Box’ AI algorithms, for example Deep Neural Networks. These methods are characterized along two axes:
Local and global methods: as the name suggests, these are methods used to explain different aspects of the AI model behaviour. Local explanations explain single model decisions, while global explanations characterize the general behaviour of a model (e.g., a neuron, a layer, an entire network). In some cases, global explanation is derived from local explanations, but this is not necessarily true for all artificial intelligence models.
Post-hoc and Ante-hoc Methods: Post-hoc “after this event” methods are those methods that provide the explanation after the model has been trained with a standard training procedure; examples of such methods are LIME, BETA, LRP. Ante-hoc methods are those that are interpreted immanently in the system, i.e., they are transparent by nature in the sense that these methods introduce a new network architecture that produces an explanation as part of its decision.
There have been several attempts to explain the prediction of the AI models. The most active among them has been on the problem of feature attribution. The feature attribution aims to explain which part of the input is most responsible for the output of the model. For example, in Computer Vision tasks detecting plant diseases, heatmaps can show the region of the most damaged leaf area of the image that affects mostly the output. Other techniques include feature visualization, interpretability by design.
These studies have produced many tools and frameworks for developing XAI systems. A short view of them could be:
LIME: Uses interpretable feature space and local approximation with sparse K-LASSO.
Anchors: Model agnostic and rule based, sparse, with interactions.
Graph LIME: Interpretable model for graph networks from N-hop neighbourhood.
XGNN: Post-hoc global-level explanations for graph neural networks.
Shap Flow: Use graph-like dependency structure between variables
XAI in AgriDataValue Project
As part of the AgriDataValue project, which aims to drive digital transformation in agriculture at the European level, a specialized module is being designed and created to increase trust and confidence of end-users in this technological ecosystem that will make massive use of AI.
The Human Explainable AI Conceptual Framework Component will fully integrate AgriDataValue data platform and Federated Machine Learning component offering two main business services to the users/citizen users:
The ability to investigate the identity of an already trained AI model (what algorithm it is, what data it was trained on, what processing steps the data underwent, etc.).
The possibility of being able to interpret a model’s decision on one or more data.
Conclusions
AI eXplainability is still object of a lively and active research that has produced, so far, many development tools and frameworks to address the difficult task of explaining the predictions of AI models, in particular those produced by Black-Box models. It is quickly becoming one of the main areas of development of the future of the AI systems. The research on XAI subject is still progressing and, in the meantime the existing methods and frameworks start being applied in application domain fields, including Agriculture, to comply with stronger and stronger interest of government bodies and Legislators. One point on which all seem to agree is that the AI of the future should conform to human values, ethical principles, and legal requirements so to ensure the privacy, security, and safety of the human users.
References
(1) “Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts, COM/2021/206 final, in https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=celex:52021PC0206.
(2) Benos et al., 2021 “Machine Learning in Agriculture: A Comprehensive Updated Review”, MDPI.
(3) Gouravmoy Bannerjee, Uditendu Sarkar et al., 2018 “Artificial Intelligence in Agriculture: A Literature Survey”, ISSN 2319 – 1953.
(4) Masahiro Ryo, 2022, “Explainable artificial intelligence and interpretable machine learning for agricultural data analysis”, KeAi Communications Co.
The agricultural sector stands at a crossroads between traditional practices and technological innovation. The AgriDataValue project is spearheading this revolution, particularly by steering semantic interoperability tasks within the agricultural data space. This initiative marks a significant impact in adopting and integrating the DEMETER’s Agriculture Information Model with the IDS Information Model, setting a precedent in the realm of data communication and management in agriculture.
Breaking New Ground in Data Economy
The core of the AgriDataValue project is to unleash the potential of data in agriculture, ensuring that it can be seamlessly shared, understood, and utilized to its fullest potential. This necessitates a common language within the data economy, particularly in agricultural spaces, to facilitate the unambiguous exchange and interpretation of data across various systems and organizations.
This is where semantic interoperability comes in. It is the glue that holds the communication between IT systems together, allowing for precise data interpretation, irrespective of the originating system’s structure or language. By ensuring data semantics are consistent across diverse systems, we pave the way for enhanced collaboration and innovation among stakeholders in the agricultural sector.
DEMETER’S AIM and IDS Information Model
The AgriDataValue project’s novelty comes from its integration of the DEMETER Agriculture Information Model (as a sector-specific information model) with the domain-agnostic IDS Information Model. The former provides a robust framework specifically tailored for the agricultural context, encompassing precise terms, structures, and standards critical to this domain.
On the other hand, the IDS Information Model serves as a universal dialect in the data economy, transcending sectors and establishing a standardized mode of data exchange and interpretation. It’s a comprehensive suite that defines common concepts, ensuring every component within a data ecosystem can effectively ‘speak’ and ‘understand’ the language of data.
When these two models converge within the scope of the AgriDataValue project, it creates a symbiotic relationship that amplifies their individual strengths. This work means stakeholders will be able to initiate a semantically interoperable, coherent, and meaningful data exchanges.
The Role of Vocabulary Hubs – Nurturing an Agricultural Data Lexicon
Central to this integration is the role of vocabulary hubs, serving as collaborative beacons within the IDS framework. These hubs are repositories and management platforms, crucial for hosting, maintaining, and documenting additional vocabularies specific to agriculture, curated from the DEMETER model and beyond.
These vocabularies, akin to dictionaries in the data realm, dispel ambiguity, ensuring information consistency across diverse systems and contexts. They are dynamic, evolving with new terms, and adaptations reflecting the growing needs and complexities of the agricultural sector.
While the path towards full semantic interoperability in agriculture is full of challenges, ranging from technical inconsistencies to organizational hurdles, the industry needs pioneers to create some usable best practices, so that they can be ready-to-use solutions to be adopted by all stakeholders of the agriculture sector.
The AgriDataValue project is a testament to the power of collaboration, standardization, and innovation. By bridging the DEMETER Agriculture Information Model with the IDS Information Model, the project is not just facilitating seamless communication in the data space today; it is setting the stage for the future of agriculture – a future where data-driven decisions propel the industry forward to new heights of success and sustainability. Join us as we continue to cultivate this digital frontier, transforming challenges into opportunities, and ideas into realities.
The Open Geospatial Consortium (OGC) and the International Data Spaces Association (IDSA) https://internationaldataspaces.org/ (partner of AgriDataValue consortium) have signed a Memorandum of Understanding (MoU) that outlines how they will together contribute to a flourishing data economy through the creation and development of standards for data spaces that ensure sovereign, interoperable, and trusted data sharing. OGC is an international non-profit consortium aiming to make geospatial (location) information and data services FAIR – Findable, Accessible, Interoperable, and Reusable. IDSA is an international non-profit association that follows a user-driven approach to create a global standard for international data spaces and interfaces based on sovereign data sharing. The two organizations have already identified AgriDataValue as a relevant to their work project. The AgriDataValue project, throughout its lifespan, will boost data sovereignty and will not just offer an open Agri-environment Platform of Platforms for capturing, processing in-situ and upgrading data, but also data sovereignty tools.
AgriDataValue adopts a DevSecOps approach for development, integration, testing and deployment.
The integration framework and tools are covered in this blog post, along with an indication of which ones AgriDataValue used to create the project’s CI/CD pipeline. Development, security, and operations, or DevSecOps, automate the integration of security at each stage of the software development lifecycle, from initial design through integration, testing, deployment, and software delivery [1].
The term “DevSecOps” suggests that security is integrated into the Agile and DevOps practices that development organizations adopt. Instead of relying on Quality Assurance (QA) testing at the end of the development cycle, on production, security issues are addressed as they arise in this viewpoint. Therefore, by automating the delivery of secure software without slowing the software development cycle, DevSecOps speeds up both the software development and release cycles.
The best practices for DevSecOps, as identified by IBM in July 2020 [1] may be summarized as follows:
Shift left: It encourages integrating security into all processes from software development to delivery from the start and moving security from the right, or the end of the DevOps process, to the left. Particularly, early involvement of cybersecurity experts in the design, development, and validation processes can help to implement security as software components are constructed. In the early stages of the software lifecycle, security risks and vulnerabilities can then be discovered and properly addressed.
Security education: To achieve the intersection of engineering and compliance with an organization’s security measures, software engineers must receive security training. The terms threat models, compliance checks, vulnerability tests, and implementing security controls should be understood by developers.
Culture: The implementation of DevSecOps requires a culture of security within organizations or even teams because it will enable people to comprehend and carry out their responsibilities in the DevSecOps lifecycle. In fact, the culture of security will involve people, communication, processes, and technology so that the chosen technological tools and the necessary software security are integrated into organizational processes.
Traceability, auditability, and visibility: These principles should serve as a guide for companies adopting DevSecOps in order to ensure greater insight into a more secure environment. Tracking configuration items that are well documented and distributed appropriately within the company will make it easier for DevSecOps to function there.
DevSecOps, as approached in AgriDataValue, is illustrated in Figure 1. According to the figure, this DevSecOps approach has the following stages: “Plan,” “Create,” “Verify,” “Package,” “Release,” “Configure,” “Detect,” “Respond,” “Predict,” and “Adapt.” These stages are correlated in a continuous workflow.
DevSecOps, as approached in AgriDataValue
The terms “Plan” and “Create” refer to the design and development of software, respectively. The terms “Verify”, “Package,” and “Release” refer to Continuous Integration and Continuous Delivery, which are handled by automated CI/CD tools, specifically GitLab in AgriDataValue. The remaining steps of the “Ops” section deal with production-level QA testing and the procedures for relaying their results back to the design and development phases in a continuous, circular interaction. Monitoring and analytics tools will make it possible to properly log throughout the entire software lifecycle during the DevOps process. Last but not least, security is integrated into every step of this DevOps cycle, implementing security by design across the individuals, teams, and technologies involved in the creation and release of the AgriDataValue software.
In an era where data drives the world, it is not a surprise that agriculture, too, is embracing the immense possibilities of data-driven decision-making through smart farming and agri-environmental monitoring. In this context, AgriDataValue project has been born to enhance smart-farming capacities, competitiveness, and fair income by introducing a fully distributed platform of platforms based on cutting-edge technology, such as complex deep learning-based decision support systems to revolutionize the food supply chain.
As the storage and computing capabilities go forward, deep learning, generative models, and AI in general have taken the spotlight. However, in the agriculture domain, there are several challenges that complicate training these AI models.
1. The Data Dilemma: Despite the increase in agri-environmental monitoring, in several use cases the scarcity of data harms the generalization capabilities of the models.
2. Representation Reality: In instances where data is accessible, it often lacks in representing the diverse realities of agriculture: geographic location, crop types and farming practices introduce biases that hinder the effectiveness of AI solutions.
3. Privacy Predicament: The presence of sensitive and private data makes entities reluctant to share their insights with other entities.
4. Non-technical end users: In agriculture, the end-users are often farmers or technicians with limited model interpretation experience. This poses a challenge as complex AI models often operate as black boxes, requiring enhanced efforts for comprehension and trust-building.
AgriDataValue’s Game-Changing Solution
How will AgriDataValue tackle these challenges?
Federated Learning (FL) is a Machine Learning approach that prioritizes privacy and decentralization. It enables models to be trained on data distributed across multiple devices without the need of sharing any sensitive information. Following this paradigm, devices or clients train their AI models with their local data and share their model updates with a central sever that aggregates them. This places FL at the forefront of innovation across diverse privacy-sensitive fields, such as medicine, finances, and agriculture.
In this context, AgriDataValue will go one step further and deploy a Hierarchical Federated Learning (HFL) approach that optimizes the resource usage by introducing a structured hierarchy into the FL process. This hierarchical arrangement allows for multi-level communications, making it a highly scalable solution and adaptable to domains like agriculture, with a high variety of data sources. The hierarchy is composed by a root server, that generates the global model, the end-users or edge devices, and several intermediate layers of servers. These intermediate server layers aggregate the model updates coming from lower layers and send the aggregated models to other aggregators positioned above in the hierarchy. After the final aggregation performed by the root server, the global model is forwarded down to the hierarchy layers.
Figure 1. Hierarchical Federated Learning example with 3 layers.
In this way, AgriDatavalues solution proposes the fusion of HFL with privacy-preserving algorithms and explainable AI (XAI), to overcome the challenges arising when constructing reliable decision systems rooted in trustworthy and comprehensible AI. In doing so, AgriDataValue platform of platforms aspires to generate AI models from diverse data sources, all while efficiently safeguarding the privacy of sensitive information. These solutions position themselves as a pioneering force in driving innovation and research within the agricultural sector.
AgriDataValue is designed and structured to significantly contribute to the overall EU agricultural development. Targeting the new Common Agricultural Policy (CAP) implementation, the project will drive developments in Precision Farming and Agri-Environmental monitoring and strengthen the Agricultural Digital Transformation at European Level. AgriDataValue has conducted a research regarding the European Union’s Common Agricultural Policy (CAP) for the period 2023-2027. The key points of the new CAP are presented in the following report.
The European Union’s CAP has long been a cornerstone of agricultural support and development. As we enter a new phase, the CAP 2023-2027 brings forth transformative changes aimed at ensuring a more sustainable, resilient, and innovative agricultural sector. With a heightened focus on environmental protection, climate action, and enhanced support for rural development, the new CAP represents a significant step forward in shaping the future of European agriculture.
1. Embracing Sustainability
Sustainability lies at the heart of the new CAP, reflecting the EU’s commitment to address environmental challenges and promote biodiversity conservation. The CAP 2023-2027 aims to promote sustainable farming practices that minimize the impact on natural resources. Farmers will be encouraged to adopt agroecological approaches, reduce chemical inputs, and embrace conservation practices to preserve soil health and protect water resources.
2. Enhancing Climate Resilience
Climate change poses unprecedented challenges to agriculture, with extreme weather events becoming more frequent and unpredictable. The new CAP places a strong emphasis on climate resilience, offering support to farmers to adapt and mitigate climate risks. Funding opportunities will be available to invest in climate-smart technologies, such as precision farming, renewable energy solutions, and carbon sequestration practices, which contribute to climate change mitigation and sustainable land management.
3. Supporting Young Farmers and Innovation
Nurturing the next generation of farmers is vital for the continuity of the agricultural sector. The CAP 2023-2027 introduces specific measures to support young farmers, facilitating their entry into agriculture and ensuring a vibrant rural future. Moreover, the new CAP encourages innovation and digitalization in agriculture by providing funding for research and the adoption of advanced technologies. This promotes a more competitive and efficient agricultural sector, capable of meeting evolving consumer demands and global challenges.
4. Strengthening Rural Development
Rural communities are the backbone of agriculture, and the new CAP seeks to strengthen their economic and social fabric. Funding will be channeled into initiatives that enhance rural infrastructures, improve access to education and healthcare, and promote local entrepreneurship. The CAP 2023-2027 will also encourage diversification in rural economies, fostering sustainable tourism, renewable energy projects, and non-agricultural activities to create more resilient rural communities.
5. Simplification and Fairness
Recognizing the complexities of the previous CAP, the new iteration aims to simplify procedures and streamline funding mechanisms. By reducing administrative burdens and enhancing transparency, the CAP 2023-2027 intends to ensure a fair distribution of support among farmers and stakeholders. Direct payments will be more targeted, focusing on environmental and social outcomes, rewarding sustainable practices and active farmers.
Conclusion
The Common Agricultural Policy 2023-2027 heralds a new era of sustainability, resilience, and inclusivity for European agriculture. With its strong commitment to environmental protection, climate action, and rural development, the CAP empowers farmers to become stewards of the land and champions of biodiversity. By embracing innovation, nurturing young farmers, and promoting sustainable practices, Europe’s agricultural sector is poised to lead the way in meeting the challenges of the 21st century. As the CAP 2023-2027 unfolds, it lays the foundation for a greener and more prosperous future for European agriculture.
The 1st Advisory Board meeting for AgriDataValue was held online on the 24th of July 2023. It was attended by external advisory board members and the consortium partners. The advisory board consists of three external experts, namely, Mrs. Marianna Faraldi (project manager at Tecnoalimenti S.C.p.A), Mr. Gregory Chatzikostas (vice president of business development at Foodscale Hub), and Prof. Federico Alvarez (associate professor at Universidad Politécnica de Madrid -UPM).
The consortium presented an overview of the project objectives, current and planned activities, as well as the project’s overall architecture. In addition, the consortium presented the project’s pilots and the associated use cases. The Advisory Board members provided inspiring and valuable feedback on the work accomplished, identifying also opportunities and potential challenges. The consortium of the AgriDataValue aims to capitalize on the fruitful discussion and experts’ recommendations that aim to safeguard the project’s smooth operation, public outreach, and impact creation.
