Prompt Engineering, Agents, and LLMs: Kickstart a New Year of Hands-On Learning about AI

Prompt Engineering, AI Agents, and LLMs: Kick-Start a New Year of Learning

And just like that, it’s 2024. Welcome back to the Variable — and to a new year of learning.

Great articles come in all shapes and sizes, and we love how on an average day at TDS, we get to publish excellent hands-on guides right next to nuanced explorations of cutting-edge research. Our first newsletter of the year reflects that balance: we’ve brought together some of our best and most thought-provoking recent articles, and hope they’ll give you just the right dose of inspiration to get things going in January and beyond. (If you’ve been mostly offline during the holidays, it’s also a great opportunity to catch up with some of our December must-reads.)

Let’s get started.

• How I Won Singapore’s GPT-4 Prompt Engineering Competition
  In an excellent (and splash-making) TDS debut, Sheila Teo recounts her recent experience of winning a prestigious competition centered around prompt-engineering techniques. She offers fresh, useful insights about the various approaches to this nascent discipline, which “blends both art and science.”
• Is ChatGPT Intelligent? A Scientific Review
  Spoiler alert: the answer is no, ChatGPT isn’t, in fact, intelligent. Oren Matar’s thorough overview of research into this question goes beyond simplistic binaries, though. It discusses the challenges of assessing the performance of large language models and the AI tools they power—especially given how magical their performance can seem to anyone who isn’t familiar with their inner workings.
• On Why Machines Can Think
  Tackling a similar theme from a different angle, Niya Stoimenova unpacks the reasoning skills LLMs display and the limitations they’re still facing. Niya encourages us to adopt a more level-headed, balanced perspective on this topic, especially given that “people are both over-exaggerating and under-representing the thinking capabilities of AI models.”

• Can LLMs Replace Data Analysts? Building an LLM-Powered Analyst
  To help us switch gears from the theoretical to the actionable, we turn to Mariya Mansurova’s experiment, whose goal is to build an LLM-based tool that can complete common workflows that data analysts currently handle. It walks us through the planning and execution stages, and sets the stage for Mariya’s follow-up guide, which delves deeper into the realm of LLM agents.
• Develop Your First AI Agent: Deep Q-Learning
  Staying on the topic of AI agents and how to work with them, Heston Vaughan’s debut article stood out as a comprehensive, patient guide to developing one from scratch. In this case, the specific context is a reinforcement learning playground, which is a great starting point for people who are new to the topic and eager to dive right in.
• Unlocking Decision-Making: AI Bridges Theoretical Frameworks with Technological Advancements
  Smart decision-making is the end goal of much of data scientists’ work; Stephanie Shen looks at how this process usually unfolds, and examines the ways in which new and powerful AI tools are enhancing the decision support systems at our disposal.
• 3 Music AI Breakthroughs to Expect in 2024
  With so much news about text- and image-generating tools like ChatGPT and Midjourney, recent advances in music-focused AI have stayed largely under our collective radar. Max Hilsdorf’s latest foray into the topic—a look into what the near future might hold for music creators, performers, and listeners—offers a fascinating glimpse at music embeddings, new applications, and other emerging trends.
• Revisiting the Death of Data Science
  The impending doom of data science as a field and as a career path has been announced time and again over the years. Brandon Cosley revisits this question through the lens of generative-AI technologies, and offers a grounded, cautiously optimistic perspective on the ways the latter will affect the workflows and toolkits of data practitioners.

Our latest cohort of new authors

Every month, we’re thrilled to see a fresh group of authors join TDS, each sharing their own unique voice, knowledge, and experience with our community. December was no exception—despite a long holiday season, we still welcomed fantastic new writers, including Daniel Bakkelund, Mike Perrotta, Tigran Hayrapetyan, Matteo Consoli, Nick Gerend, Kateryna Herashchenko, Ella Pham, João Felipe Guedes, George Miloshevich, Tea Mustać, Mike Cvet, K Bahavathy, Amber Roberts, Maxwell Wulff, Elahe Aghapour & Salar Rahili, Harminder Singh, Rafael Guedes, Pablo Piskunow, Rishabh Raman, Kamil Raczycki, Heston Vaughan, John Andrews, Sheila Teo, Yevhen Kralych, Carla Pitarch Abaigar, Jesper Alkestrup, Alessandro Tomassini, Jan Philip Wahle, Sarthak Sarbahi, Bhavin Jawade, Kate Yurkova, Wenqi Glantz, Matthew Andres Moreno, Gyorgy Kovacs, Livia Ellen, Iulia Brezeanu, Vincent Koc, and Alberto Paderno, among others.

Thank you for supporting the work of our authors! If you enjoy the articles you read on TDS, consider becoming a Friend of Medium Member: it’s a brand-new membership level that offers your favorite authors bigger rewards for their top-notch writing.

Until the next Variable,

TDS Editors

Prompt Engineering, Agents, and LLMs: Kickstart a New Year of Hands-On Learning about AI was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.