Inefficient Data Management Identified as Key Barrier to Adopting AI
These days, a common problem among companies is getting a hang of data management. More specifically, data management is turning out to be the top technical inhibitor to AI adoption. Some industries display more glaring obstacles to AI/ML adoption. The current agrifood data landscape, although rich in information, consists of data stored in different forms and places, making search difficult. That also makes the data unfit for AI tasks.
In this blog post, we will explore the issue of data management in 2023 and discuss potential strategies for the agricultural sector to overcome this challenge.
Data Management: The Primary Technical Barrier to Embracing AI
The Analyst Report, conducted by S&P Global Market Intelligence, rightly highlighted that “Data is the lifeblood of AI, but if left unchecked, it can quickly become a deterrent.” As enterprises build growing databases and expand data architectures, they are finding them buckling under the strain of AI workloads.
Thus, somewhat surprisingly, data management emerged as the foremost concern among the challenges indicated by respondents in the survey. Out of over 1,500 surveyed people, 32% of them put data management in the first place when it comes to inhibitors of their company’s AI/ML application deployments. Following closely, security claimed the second position, with 26% of respondents listing it as the top challenge. Seeing how cybersecurity dominated the internet conversation for the previous years, it seems the focus is shifting with the onset of generative AI. Compute performance is listed third on the list.
The survey observed responses from AI practitioners and decision-makers in large and medium enterprises. But it can be useful information for businesses and organisations of all sizes, as the majority of them are collecting and storing more data than ever. The information from the new report provides a useful pathway to thinking holistically about data management.
Knowledge Lakes Hold the Answer to Data Management?
With the rapid development of IoT, AI, 5G, and other digital-related advancements, organisations are generating large amounts of data. If their information remains digitally scattered, that leaves the data architecture underdeveloped.
That is why data lakes are necessary. These large repositories organise raw data into its native format. It is mostly left to the data analyst to curate them later. But even so, the data architecture maintains the data life cycle from its source via processing, AI and ML analytics, and, of course, data lake storing.
However, the needs of many organisations exceed the capabilities of data lake architecture. This is where we bring Knowledge Lakes into our conversation. These are contextualised Data Lakes which are quicker to sort raw data and derive insights.
That is a fairly novel principle – contextualising data by using AI and ML. That approach improves the usability and interoperability of data, data quality management, and metadata management. One of the current trends in digital technology is the development of systems for niche data needs.
Such a system is the planned STELAR Knowledge Lake Management System. Its main goal is to enable, support, automate the creation, and manage FAIR and AI-ready data in the common European data space, specifically in the agrifood data space. The foundation would be a set of state-of-the-art open-source components acting as a platform, while connected to it would be modular and scalable sets of data management tools. These tools would be prolific in turning raw data lakes into knowledge lakes of data assets that are given semantic context and annotated.
Entities requiring access to data related to food safety, food security, and sustainability would experience the advantages offered by these data management technologies. More precisely, the STELAR endeavor will be tested in three real-life pilot scenarios in the following areas:
- Risk prevention in food supply lines
- Early crop growth predictions
- Timely precision farming interventions
Let’s put these benefits into numbers. The planned data management tools could have the following effect:
- Saving up to 50% of time spent on risk prevention decisions.
- Border recalls and rejections cost a company €10 million on average, and better risk prediction can lower those costs.
- A 10% reduction of recalled and rejected food can end up saving a company €1 million.
Other Key Findings
- 69% of surveyed experts said their organisation has at least one AI project in the pipeline.
- Just 28% of respondents said their organisations reached enterprise scale, with AI projects in the pipeline and driving business value.
- Artificial Intelligence was regarded as a cost-saving lever before but it has now shifted to a revenue driver, with 69% of surveyed respondents now leveraging AI and/or ML to come up with new revenue streams.
- 77% of those surveyed for the report said their data architectures impact their sustainability performance.
About the Report
The report was conducted by the renowned S&P Global Market Intelligence and commissioned by WEKA. Overall, it highlighted the opportunities and potential issues organisations may come across during their digital transformation journeys. It also gave a glimpse into the steps these organisations may take to adopt AI in the future.
Even though the STELAR project explored SMEs (small and medium-sized enterprises) in the EU agricultural landscape, it is still worth looking into how decision-makers and AI practitioners view progress in their research organisations and medium and large enterprises. The study encompassed respondents across APAC, EMEA, and North America. For more insight such as this one, follow STELAR and its Blog.
