2024 Survival Guide for Machine Learning Engineer Interviews

A year-end summary for junior-level MLE interview preparation

Job-seeking is hard!

In today’s market, job-seeking for machine learning-related roles is more complex than ever. Even though public reports claim that the job demand for machine learning engineers (MLE) is fast growing, the fact is that the market has turned toward an employer’s market over the past few years. Finding an ML job in 2020, 2022, and 2024 could be completely different experiences. What’s more, a few factors contribute to the disparities of job-seeking difficulties across geography, domain, as well as seniority level:

• Geography: According to the People in AI report, the top cities hiring in North America in 2024 are the Bay Area, NYC, Seattle, etc. If we use the ratio between professionals# and post# to evaluate the success rate of finding a job, then the success rate in the Bay Area is 3.6%. However, if you live in LA or Toronto, the demand is much lower, which causes the success rate to drop to 1.4%, only 40% compared to the Bay Area. The success rate could be even lower if you live in other cities.
• Domain: The skill sets needed for ML Engineer roles vary widely for each domain. Take deep learning models as an example; CV usually uses models like ResNet, Yolo, etc., while NLP involves understanding RNN, LSTM, GRU, and Transformers; Fraud Detection uses SpinalNet; LLM focuses on the knowledge of Llama and GPT; Recommendation System consists of the understanding of Word2Vec and Item2Vec. However, not all domains are hiring the same number of ML Engineers. If we search the tags in the system design case studies from Evidently AI, the tag CV corresponds to 30 use cases, Fraud Detection corresponds to 29, NLP corresponds to 48, LLM corresponds to 81 and Recommendation System corresponds to 82. The Recommendation System has almost 2.7 times more use cases than the CV. This ratio might be highly biased and not truly reflect the actual situation in the job market. Still, it shows the likelihood of more opportunities in the Recommendation System for ML Engineers.
• Seniority level. According to the 365 DataScience report, although 72% of the postings don’t explicitly state the YOE required, engineers with 2–4 YOE are in the highest demand. This means you’ll likely face more difficulty getting an entry-level job offer. (The reason that most demand is on engineers with 2–4 YOE could be explained by the fact that MLE was not a typical role five years ago, as explained in this blog post.)

This article will summarize the materials and strategies for MLE interview preparation. But please remember, this is just an empirical list of information I gathered, which might or might not work for your background or upcoming interviews. Hopefully, this article will shed some light or guidance on your career-advancing journey.

Before the Interview — What to Expect?

The interview journey could be extended, painful and lonely. When you start applying for jobs, there are things you should think and plan accordingly:

• Interview timeline
• Types of roles
• Types of companies
• Domain
• Location

Interview timeline. The timeline for each company is different. For companies of smaller sizes (<500), usually in pre-seed or series A/B, the timeline is generally faster, and you can expect to finish the application process within a few weeks. However, for companies of larger sizes (> 10k or FAANG), from application submission to the final offer stage, it can vary from 3–6 months, if not longer.

Types of roles. I would refer to Chip Huyen’s Machine Learning Interview book for a more detailed discussion of different ML-related roles. The role of MLE could come under different names, such as machine learning engineer, machine learning scientist, deep learning engineer, machine learning developer, applied machine learning scientist, data scientist, etc. At the end of the day, a typical machine learning engineer role is end-to-end, which means you’ll start by talking to product managers (sometimes to customers) and defining the ML problem, preparing the dataset, designing and training the model, defining the evaluation metrics, and serving and scaling the model, and keeping improving the outcome. Sometimes, companies mix the titles, e.g., MLE with ML Ops. It’s the responsibilities that matter, not the titles.

Types of companies. Again, Chip Huyen’s Machine Learning Interview book discusses the differences between application and tooling companies, large companies and startups, and B2B and B2C companies. Moreover, it’s worth considering whether the company is public or private, whether it’s sales-driven or product-oriented. These concepts should not be overlooked, especially if you’re looking for your first industry job, as they will build your “lens of career,” which we’ll discuss later in the interview strategy section.

Domain. As mentioned in the introduction section, the jobs of MLE could fall into different domains like recommendation systems and LLMs, and you need to spend time preparing for the fundamental knowledge. You need to identify one or two domains you’re most interested in to maximize your chances; however, preparing for all different domains is almost impossible as it will disperse your energy and attention and get you under-prepared.

Location. Beyond all the points above, location is a serious matter. Looking for MLE jobs will be even more difficult unless you live in high-demand areas like the Bay Area or NYC. If relocating is impossible, you probably need to plan for a longer timeline to get a satisfying job offer; however, if you leave the relocating option open, applying to opportunities in high-demanding areas is probably a good idea.

During the Interview — How to Prepare?

Once you start the application process and start to get interviews, there are a few things you need to search and prepare for:

• Interview format
• Referrals, networking
• LinkedIn or Portfolio
• Interview resources and materials
• Strategies: planning, tracking, evolving, prompting, estimating your level, wearing your “lens of career,” getting an interview partner, red flags
• Accepting the offer

Interview format

The interview format varies among different companies. No two companies have the same interview format for the MLE role, so you must do your “homework” on researching the format in advance. For example, even for FAANG companies, Apple is known for its startup-style interview format, which varies from team to team. On the other hand, Meta tends to have a consistent interview format at the company level, comprising one or two leet code rounds and ML system design rounds. Usually, the recruiter would give detailed information about large companies’ interview format, so you won’t be surprised. However, the process could be less structured for smaller companies and change more frequently. Sometimes, smaller companies replace leet code with other coding questions and only lightly touch the modelling part instead of having an entire ML system design session. You should search for information on free websites like Prepfully, Glassdoor, Interview Query, or other paid websites for a comprehensive understanding of the interview format and process to prepare better in advance. Lastly, don’t be limited by interview format as it’s not a standard test — there could be behaviour elements during technical interviews and technical questions during your hiring manager round. Be prepared, but be flexible and ready to be surprised.

Referrals and networking

Many web articles would exaggerate the benefit of referrals, but having a referral is just a shorter path for you to get past the recruiter round and land directly to the second round (usually the hiring manager round). Besides having referrals, it’s almost equivalently essential to network in person, e.g., use hackathon opportunities to talk to companies, go to in-person job fairs, and participate in offline volunteer events sponsored by companies you’re interested in. Please don’t rely on referrals or networking to get a job, but use them as opportunities to increase your probability of getting more conversations from recruiters and hiring managers to maximize your interview efficiency.

LinkedIn or Portfolio. LinkedIn and Portfolio are just advertising tools that help recruiters understand who you are beyond the textual information in your resume. As a junior MLE, it would help to include course projects and Kaggle challenges in your GitHub repository to show more relevant experience; however, at you get more senior level, toy projects make less sense, but PR in large-scale open source projects, insightful articles and analysis, tutorials on SOTA research or toolboxes, will make you stand out from the rest of the candidates.

Interview resources and materials

Generally speaking, you need materials covering the five domains: i) coding, ii) behaviour, iii) ML/Deep learning fundamentals, iv) ML system design, and v) a general MLE interview advice book.

i) Coding. If you’re not a Leet Code expert, then I would recommend starting with the following resources:

• NeetCode
• Cracking the Coding Interview: 189 Programming Question…
• Coding Interview Patterns: Nail Your Next Coding Interv…

ii) Behaviour:

• Behavioral Interviews for Software Engineers: All the M…
• The Staff Engineer’s Path: A Guide for Individual Contr…

iii) ML/Deep learning fundamentals:

Deep Learning: Foundations and Concepts

iv) ML system design:

• Machine Learning System Design Interview
• Designing Machine Learning Systems: An Iterative Proces…

v) A general MLE interview advice book:

• Introduction to Machine Learning Interviews Book
• Inside the Machine Learning Interview: 151 Real Questio…

You also need to have a handful of interview partners — these days, you can subscribe to online interview preparation services (don’t use the costly ones which charge you thousands of dollars; there are always cheaper replacements) and pair up with other MLE candidates for skill and information exchange.

Strategies: planning, tracking, evolving, prompting, interview for one level up, wearing your “lens of career,” getting an interview partner, red flags

Planning, tracking, and evolving. Ideally, as described in this article, you should get at least a handful of recruiter calls and categorize your interviews to different interest levels. For one thing, the job market is constantly changing, and someone can rarely plan for the best strategy in the first interview. For the other thing, you’ll learn and grow during the interview process, so you’ll become different from where you were a few months ago at the beginning of the job-seeking stage. So, even if you’re the most talented candidate on the market, it’s essential to spread out your conversations over a few months and start with the conversations that you’re least interested in to familiarize yourself with the market and sharpen your interview skills, and leave the most important ones to the later stage. Track your progress, feedback, and thoughts during your interview process. Set specific learning goals and evolve with your interviews. You might never have had the chance to touch on GenAI knowledge in the past few years, but you could utilize the interview process to learn from online courses and build small side projects. The best thing is to get a job after the interview, and the second best thing is to learn something useful even if you don’t get the job offer. If you keep learning from every interview, eventually, it will vastly increase your chances of getting the next job offer.

Prompting. This is the age of LLM, and you should utilize it wisely. Look for the keywords in the job descriptions or responsibilities. If there is an interview involving “software engineering principles,” then you can prompt your favourite LLM to give you a list of software engineering principles for machine learning for preparation purposes. Again, the prompt answer shouldn’t be your sole source of knowledge, but it can compensate for some blind spots from your daily reading sources.

Interview for one level up. Sometimes, the boundaries between levels are blurry. Unless you’re an absolute beginner in this field, you can always try the opportunities that are one level above and prepare for the down level at the job offer stage. If you’re interviewing for senior level, preparing or applying to staff-level opportunities doesn’t hurt. It doesn’t always work, but sometimes it can open doors for you.

Wear your “lens of career”. Don’t just go to an interview without thinking about your career. Unless you desperately need this job for a specific reason, ask yourself, where does this job fit into your overall career map? This question matters from two perspectives: first, it helps you choose the company that you want to go to, e.g., one startup might offer higher salaries in the short run, but if it doesn’t prioritize sound software engineering principles, then you’ll lose the opportunity to grow into a promising career in the long run; second, it helps to diagnose the outcome of the interview, e.g., your rejections are mostly from startups, but eventually you landed in offers from well-known listed companies, then you’ll realize the rejections don’t mean you’re not a qualified MLE, but because interviewing at startups require different skills and those don’t belong to your career path.

Partner up. Five years ago, there was no such thing as finding an interview partner. But these days, there are interview services all over the internet ranging from extremely high cost (which I don’t recommend) to a few hundred dollars. Remember, it’s a constantly shifting market, so nobody knows the whole picture. The best way to gain information is to partner with your non-competitive peers (e.g., you’re in the CV domain, and your partner is in the recommendation system domain) to practice and improve together. Better than just partnering, you should seek to partner up — look for people with a higher seniority level while you can still offer something useful for them. You might ask, how is it possible? Why would someone more senior than me want to practice with me together? Remember, nobody is perfect, and you can consistently offer others something. There are senior software engineers who would like to become MLE, and you can trade your ML knowledge for their software engineering best practices. There are product managers who need ML-related input, and you can ask for behavioural practice in return. Even for people with no industry experience at the entry-level, you can still ask for coding practice in return or listen to their life stories and get inspired. As an MLE, especially at the senior/staff level, you need to demonstrate leadership skills, and the best leadership skill you can demonstrate is to collect the professionals at different levels to help achieve the goal you’re chasing after — your dream offer.

Red flags. Some red flags, like asking you to overwork directly or ghosting the interviews, are explicit. However, some red flags are more subtle or deliberately disguised. For example, your hiring manager might politely explain their situation and wish you “didn’t have high expectations at the beginning and decide to leave in a few months” — it sounds so considerate. Still, it shadows the fact that the turnover is high. The best strategy for avoiding red flags involves reading Glassdoor reviews and learning about company culture during the interview. Specifically, “culture” doesn’t mean the “culture claims” defined on the company website but the actual dynamics between you and the team. Are the interviewers only asking prepared questions without trying to understand your problem-solving skills? When you throw a question, can the interviewer catch that question and give an answer that helps you to understand the company’s value better? Lastly, always remember to use your gut feelings and decide whether you like your future team. After all, if you decide to take the job offer, you’re facing these people eight hours per day for the next few years; if your gut feeling tells you that you don’t like them, then it won’t be happy anyway.

Accepting the offer. Once you’re done with all the frustration, all the disappointments, and all the hard work, it’s time to talk about the offer. Many web threads discuss the necessity of negotiating the offer, but I suggest being cautious, especially in this employer’s market. If you want to negotiate, the best practice is to have two comparable offers and prepare for the worst case. Also, websites like Levels.fyi and Glassdoor should be used to research the compensation range.

After the Interview — What to Do Next?

Congratulation! Now you’ve accepted your offer and are ready to start your new journey, but is there anything else you can do?

Summarize your interview process. The interview journey was long and painful, but you also gained a lot of things from the journey! Now you’re relaxed and happy, and it’s the best opportunity to reflect, be grateful to the people who helped you during the interview, and share some of the information with those still struggling with their interview process. Besides, you must have collected a lot of notes and to-do lists during the past few months but never had time to sit down and organize them, and now is the best opportunity to do that!

Plan your career path. Your self-understanding could have changed during the past few months; now, you’ve better understood your learning ability and problem-solving ability under pressure. You have talked to so many startups and big techs and start to have a better picture of where you are in the next five years. If you’re at the mid-to-senior level, then you have probably talked to many people at the staff level and gotten a better idea of what you will work on in the next stage of your career. This is the time for all this planning!

Keep learning. If you’re from an academic background, you’re probably used to reading papers and learning about SOTA ML techniques. However, the role of MLE is more than that of the academy. It combines research, applied ML practices and software engineering to make a real business impact. Now is a good time to think about the best strategy to keep learning from multiple sources to keep yourself up-to-date.

Get ready for the new role. You have talked to the other MLEs in your new company and know what models or tech stack they’re using. In sporadic cases, you already know this tech stack very well, but most times, you need to learn many new things for the new role. Make a plan for how you will learn them and set small milestones to achieve the goals. Besides, learn about your new company, explore its homepage and understand its business goals. this will help to set a good tone when you start the new job and talk to your new colleagues.

After all, the interview journey is different for everyone. Your level of experience, focus on the domain, long-term career goal, and personality all form a unique interview journey. Hopefully, this article will shed some light on the interview preparation materials and strategies. And I hope everyone will eventually land on their dream offer!

Acknowledgement: Special thanks to Ben Cardoen and Rostam Shirani for proofreading and insightful suggestions that contributed to the final version of this article.

References

• Sophie Magnet, “The Most In-Demand Machine Learning Engineer Skills in 2024,” link: https://365datascience.com/career-advice/career-guides/machine-learning-engineer-skills
• Evidently AI, “ML and LLM system design: 500 case studies to learn from,” link: https://www.evidentlyai.com/ml-system-design
• Sam Jones, “2024 Job Market for Machine Learning Engineers,” link: https://www.peopleinai.com/blog/2024-job-market-for-machine-learning-engineers
• Jennifer Robertson, “Why It’s So Difficult to Find Senior Machine Learning Engineers,” link: https://blog.qwasar.io/blog/why-its-so-difficult-to-find-senior-machine-learning-engineers
• Chip Huyen, “Introduction to Machine Learning Interviews Book,” link: https://huyenchip.com/ml-interviews-book/
• NeetCode Roadmap, link: https://neetcode.io/roadmap
• Gayle Laakmann McDowell, “Cracking the Coding Interview: 189 Programming Questions and Solutions,” 2015
• Alex Xu, “Coding Interview Patterns: Nail Your Next Coding Interview,” 2024
• Alejandro Companioni, “On job-hunting in 2024 as a machine learning engineer,” link: https://acompa.net/mle-job-hunt-2024.html
• Peng Shao, “Inside the Machine Learning Interview: 151 Real Questions from FAANG and How to Answer Them”, 2023
• Chip Huyen, “Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications,” 2022
• Ali Aminian, Alex Xu, “Machine Learning System Design Interview,” 2023
• Christopher M. Bishop, Hugh Bishop, “Deep Learning: Foundations and Concepts,” 2023
• Tanya Reilly, “The Staff Engineer’s Path: A Guide for Individual Contributors Navigating Growth and Change,” 2022
• Melia Stevanovic, “Behavioral Interviews for Software Engineers: All the Must-Know Questions With Proven Strategies and Answers That Will Get You the Job,” 2023

2024 Survival Guide for Machine Learning Engineer Interviews was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.