Revisiting the Death of Data Science

And what the Gen-AI revolution has taught us for 2024

Every trendy thing comes with the concern of its future obsolescence. Trends capture us. They draw us in with their novelty and promises of belonging, coolness, and for the data scientist, their value. We don the latest models, adopt the buzzwords, chase the viral experiences — all in the pursuit of that feeling of “in-ness.” The ability to proudly profess that “I am a data scientist” and feel confident that it still means something.

But within this expanding bubble lurks the shadow of obsolescence. It whispers doubts: “Will this new skill be next week’s punchline?” “Will I, armed with yesterday’s algorithms, become a fossil of a bygone era of technology?” The fear of being “unvalued,” ostracized from the ever-shifting sands of artificial intelligence, bears a potent influence over our continued search for identity.

Back in 2021, I wrote an article addressing my thoughts on whether data science was dead.

Is Data Science Truly Dead?

And since that time, we have seen an acceleration of changes in the field. Data scientists are no longer as cool as we once were. The trend is artificial intelligence, and the buzzwords are generative, composite, and multimodal. With the rise of generative AI, we are stoked with new concerns over the future for data science practitioners. So herein I revisit my article and the question it posed; If it wasn’t then, is data science dead now?

The Rise of [More Capable] Machines

If data scientists were concerned about being automated into extinction prior to 2022, before the general release of ChatGPT, then post its release the concerns appear even more palpable, in part, because they feel more achievable. With Gen-AI, automation and pre-built solutions have advanced even further than I predicted. Tools have become more sophisticated and accessible, allowing more people without extensive data science skills to build models. And now, those models can be built using natural language directions further reducing the need for technical programming skills.

However, these tools still require human oversight and judgment, so while they enable citizen data scientists, they have not eliminated the need for experienced professionals. Intelligent automation is quickly becoming the next trend. Data scientists who embrace it, who become the architects of these intelligent systems, who bridge the gap between human intuition and machine efficiency — they will not be the replaced, but the irreplaceable.

To do this, we need to continue our search for integration. As individual users, the tools of Gen-AI that are made accessible through nicely packaged web interfaces have limits in their ability to bring broader value to the business problems they can solve. Learning to embed those tools effectively and efficiently into the fabric of our code with API integrations will serve as true differentiators in the crowded data science workforce.

The Constantly Evolving Toolkit

But learning to integrate with these new, more capable machines isn’t the only cog in the wheel of the data science toolkit. I was right that new innovations would continue emerging at a rapid pace, but I was not expecting that pace to go into hyperdrive with the release of Gen-AI. Thus, more practitioners now need to leverage capabilities like MLOps, prompt design, and composite AI coupled with the cloud, automation, and containerization I highlighted back in 2021.

For the data scientists of old, learning wasn’t something we left locked behind the thin paper veil of our degrees. It has defined our success in the applied world and is a lesson we must continue to heed. The reality remains: the most adaptable data scientists are those who constantly learn and integrate the latest advancements into their workflows.

In the evolving landscape of data science, a critical skill for future practitioners remains a comprehensive understanding of the challenges associated with deploying advanced AI tools. As these tools enhance their capabilities, their inherent complexity grows in tandem. This heightened complexity not only translates into increased computational demands but also often entails greater associated costs. Furthermore, many of these cutting-edge tools exist beyond the confines of organizational firewalls, giving rise to concerns regarding data privacy, ethical considerations, and practical applicability. To navigate this terrain effectively, future data scientists must grasp these limitations thoroughly, enabling them to assess the potential of these tools in addressing real-world problems.

The Expectation of a User Experience

But before we get to the future, there is still a past we must contend with. And in that past, data science wasn’t always about seamless user experience. Models were buried in code, serialized, and saved on servers. Accessing them required some understanding of how data flows and what probabilities mean.

But the future is different. Today, models are delivered directly to users. With Gen-AI all that’s required to leverage these models is an internet connection and a browser. This shift demands a new kind of data scientist — one who thinks beyond just the accuracy of their models and considers the entire user journey.

Thus, when it comes to the user experience, expectations have increased around impact and value. It’s no longer sufficient to just develop models — data scientists must make those models usable and accessible through well-designed applications and interfaces. A user-centric mindset focused on outcomes rather than just accuracy metrics has become essential. With more people now able to take advantage of AI tools, the masses are becoming more educated on both their promise and their limitations. Building interfaces that enable solutions will continue to be essential and thus the data scientists of the future would benefit from building skills with UI frameworks like Streamlit and RShiny to bring their stakeholders a greater sense of control and intractability. I would emphasize that this experience is no longer important, it is an expectation.

Shouting from the Rooftops of the Trade

As Gen-AI has taken hold, data literacy efforts have stepped up as organizations realize the potential of this trend. With everyone from CEOs to entry-level employees now able to leverage AI in their day-to-day work, more and more people are becoming AI-aware. But awareness doesn’t necessarily translate into understanding. Data scientists play a key role in spreading knowledge, mentoring citizen data scientists, and serving as the connective tissue between technical systems and business needs. Soft skills like communication and translation are now just as vital as hard technical competencies.

And I reiterate, data science remains relevant because it isn’t a mere tool in the toolkit but rather a way of thinking about and tackling problems. Data scientists are adept at understanding how machine learning and AI models help us to handle uncertainty more intelligently in problem spaces. Without an intuition for how these models work, realizing their true potential will be limited at best, and applied irresponsibly at worst.

What Should We Be Prepared for in the Future?

Looking ahead, I predict data science generalists will need to continue specializing into more defined roles to keep pace with AI’s exponential growth. On one end, AutoML Engineers will focus solely on building and maintaining automated systems. On the other, Solution Engineers will concentrate on integrating models into products and ensuring they solve real issues. Data Scientists will sit in the middle — blending technical depth with business understanding to lead impactful analytics initiatives. Overall, human oversight and judgment will remain essential even as much of the work becomes increasingly automated. The future data scientist must embrace new innovations while never losing sight of the human element.

The take home? I still don’t think data scientists have anything to worry about. In short, and to answer the question…no…data science isn’t dead and in fact, is arguably even more important in this newly developing AI integrated world. And our ongoing battle with concerns of obsolescence may indeed hold a deeper truth about the human condition. It reminds us that nothing, no matter how trendy, is permanent. So let us not define ourselves by the hottest trend, but by the values and passions that weather the storms.

Like engaging to learn about data science, career growth, life, or poor business decisions? Learn more about me here.

Additional resources:

• 2023 in Review: Recapping the Post-ChatGPT Era and What to Expect for 2024
• 7 Key Data Science Trends For 2024-2027
• 2024 Data + AI Predictions
• Data Science 2024: Where Cats, Coffee, and Code Collide

Revisiting the Death of Data Science was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.