Gartner, Inc., a leading research and advisory firm, has highlighted the top trends that are shaping the future of data science and machine learning (DSML). As the industry continues to grow and evolve, the significance of data in artificial intelligence (AI) becomes increasingly important, especially with the growing interest in generative AI investments.
According to Peter Krensky, Director Analyst at Gartner, DSML is transitioning from a focus solely on predictive models to a more inclusive, dynamic, and data-centric discipline. This transformation is driven by the enthusiasm surrounding generative AI. While there are emerging risks, there are also numerous new capabilities and use cases for data scientists and their organizations.
One of the key trends identified by Gartner is the shift towards cloud data ecosystems. Data ecosystems are transitioning from self-contained software or blended deployments to comprehensive cloud-native solutions. Gartner predicts that by 2024, 50% of new system deployments in the cloud will be based on a cohesive cloud data ecosystem rather than manual integration.
Another trend is the rise of Edge AI, which enables data processing at the point of creation, providing real-time insights, identifying new patterns, and meeting stringent data privacy requirements. Gartner forecasts that by 2025, over 55% of data analysis by deep neural networks will occur at the edge system, compared to less than 10% in 2021. Organizations are encouraged to identify applications and AI training needed to move towards edge environments near IoT endpoints.
Responsible AI is also a crucial trend highlighted by Gartner. Responsible AI aims to ensure that AI is a positive force rather than a threat to society. It involves making the right business and ethical choices when adopting AI, including considerations of societal value, risk, trust, transparency, and accountability. Gartner predicts that by 2025, 1% of AI vendors will concentrate the majority of pretrained AI models, making responsible AI a significant concern.
The concept of Data-Centric AI is gaining momentum as well. It represents a shift away from a model and code-centric approach to a more data-focused strategy for building better AI systems. Solutions such as AI-specific data management, synthetic data, and data labeling technologies tackle data challenges related to accessibility, volume, privacy, security, complexity, and scope. Gartner expects that by 2024, 60% of data for AI will be synthetic, simulating reality and future scenarios, compared to just 1% in 2021.
Accelerated AI investment is driving the adoption of AI solutions in organizations and industries. Gartner predicts that by the end of 2026, more than $10 billion will have been invested in AI startups relying on foundation models—large AI models trained on extensive data. A recent Gartner poll revealed that 45% of executive leaders increased their AI investments due to the hype surrounding ChatGPT. Additionally, 70% of organizations are exploring generative AI, while 19% are already piloting or implementing it.
The trends identified by Gartner offer valuable insights into the future of DSML and provide guidance for organizations aiming to leverage data science and machine learning effectively. As the industry continues to evolve, it is essential to embrace cloud data ecosystems, explore Edge AI capabilities, prioritize responsible AI practices, adopt a data-centric approach, and invest in AI to stay competitive in an increasingly data-driven world.
By staying informed and embracing these trends, organizations can harness the power of DSML to drive innovation, uncover valuable insights, and navigate the complexities of AI in a responsible and efficient manner. As the world relies more on data and AI, understanding and leveraging these trends will be crucial in shaping a successful future for businesses and society as a whole.
