Concerns about the environmental impacts of Large Language Models (LLMs) are growing. Although detailed information about the actual costs of LLMs can be difficult to find, let’s attempt to gather some facts to understand the scale.
Generated with ChatGPT-4o
Since comprehensive data on ChatGPT-4 is not readily available, we can consider Llama 3.1 405B as an example. This open-source model from Meta is arguably the most “transparent” LLM to date. Based on various benchmarks, Llama 3.1 405B is comparable to ChatGPT-4, providing a reasonable basis for understanding LLMs within this range.
Inference
The hardware requirements to run the 32-bit version of this model range from 1,620 to 1,944 GB of GPU memory, depending on the source (substratus, HuggingFace). For a conservative estimate, let’s use the lower figure of 1,620 GB. To put this into perspective — acknowledging that this is a simplified analogy — 1,620 GB of GPU memory is roughly equivalent to the combined memory of 100 standard MacBook Pros (16GB each). So, when you ask one of these LLMs for a tiramisu recipe in Shakespearean style, it takes the power of 100 MacBook Pros to give you an answer.
Training
I’m attempting to translate these figures into something more tangible… though this doesn’t include the training costs, which are estimated to involve around 16,000 GPUs at an approximate cost of $60 million USD (excluding hardware costs) — a significant investment from Meta — in a process that took around 80 days. In terms of electricity consumption, training required 11 GWh.
The annual electricity consumption per person in a country like France is approximately 2,300 kWh. Thus, 11 GWh corresponds to the yearly electricity usage of about 4,782 people. This consumption resulted in the release of approximately 5,000 tons of CO₂-equivalent greenhouse gases (based on the European average), , although this figure can easily double depending on the country where the model was trained.
For comparison, burning 1 liter of diesel produces 2.54 kg of CO₂. Therefore, training Llama 3.1 405B — in a country like France — is roughly equivalent to the emissions from burning around 2 million liters of diesel. This translates to approximately 28 million kilometers of car travel. I think that provides enough perspective… and I haven’t even mentioned the water required to cool the GPUs!
Sustainability
Clearly, AI is still in its infancy, and we can anticipate more optimal and sustainable solutions to emerge over time. However, in this intense race, OpenAI’s financial landscape highlights a significant disparity between its revenues and operational expenses, particularly in relation to inference costs. In 2024, the company is projected to spend approximately $4 billion on processing power provided by Microsoft for inference workloads, while its annual revenue is estimated to range between $3.5 billion and $4.5 billion. This means that inference costs alone nearly match — or even exceed — OpenAI’s total revenue (deeplearning.ai).
All of this is happening in a context where experts are announcing a performance plateau for AI models (scaling paradigm). Increasing model size and GPUs are yielding significantly diminished returns compared to previous leaps, such as the advancements GPT-4 achieved over GPT-3. “The pursuit of AGI has always been unrealistic, and the ‘bigger is better’ approach to AI was bound to hit a limit eventually — and I think this is what we’re seeing here” said Sasha Luccioni, researcher and AI lead at startup Hugging Face.
And now?
But don’t get me wrong — I’m not putting AI on trial, because I love it! This research phase is absolutely a normal stage in the development of AI. However, I believe we need to exercise common sense in how we use AI: we can’t use a bazooka to kill a mosquito every time. AI must be made sustainable — not only to protect our environment but also to address social divides. Indeed, the risk of leaving the Global South behind in the AI race due to high costs and resource demands would represent a significant failure in this new intelligence revolution..
So, do you really need the full power of ChatGPT to handle the simplest tasks in your RAG pipeline? Are you looking to control your operational costs? Do you want complete end-to-end control over your pipeline? Are you concerned about your private data circulating on the web? Or perhaps you’re simply mindful of AI’s impact and committed to its conscious use?
SLM can be a smarter choice!
Small language models (SLMs) offer an excellent alternative worth exploring. They can run on your local infrastructure and, when combined with human intelligence, deliver substantial value. Although there is no universally agreed definition of an SLM — in 2019, for instance, GPT-2 with its 1.5 billion parameters was considered an LLM, which is no longer the case — I am referring to models such as Mistral 7B, Llama-3.2 3B, or Phi3.5, to name a few. These models can operate on a “good” computer, resulting in a much smaller carbon footprint while ensuring the confidentiality of your data when installed on-premise. Although they are less versatile, when used wisely for specific tasks, they can still provide significant value — while being more environmentally virtuous.
Smaller is smarter was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
While most Cyber Monday sales are just extensions of discounts from Black Friday, they still present a good opportunity to grab a solid notebook at a decent price. To help you avoid wasting cash, we’ve researched reviews, checked price histories and rounded up a few Cyber Monday laptop deals that are genuinely cheaper than usual.
The offers still bring several notebooks we’ve tested and recommended down to (or at least close to) their best prices to date. Apple’s latest MacBook Air is available for a low of $844, for one, while the newest Microsoft Surface Laptop is $300 off. The top recommendation in our Chromebook buying guide is on sale as well, as is our pick for the best Windows notebook for those on a budget.
We’ll update this post if prices change and more deals pop up over the rest of the day. Just keep in mind that many of the deals below are for pre-configured models; if you want to spec out your machine more precisely, it’s worth buying directly from a manufacturer. You’re also more likely to run into longer shipping times than we saw last week.
Apple MacBook Air (13-inch, M3) for $844 at Amazon ($255 off MSRP): The latest MacBook Air is our pick for the best laptop you can buy. It ticks nearly all the boxes for those who prefer macOS, from its premium design and comfy keyboard to its lengthy battery life and capable performance for everyday use. We gave it a score of 90 in our review. A recent Bloomberg report said that Apple will likely release a minor refresh early next year, but this is a new low for the current base model, which includes Apple’s M3 chip, 16GB of RAM and a 256GB SSD. If you want more storage, the 512GB variant is down to an all-time low of $1,049. Also at Best Buy, but for $55 more.
Apple MacBook Air (15-inch, M3) for $1,044 at Amazon ($255 off): The 15-inch MacBook Air is more or less identical to the 13-inch version in terms of features and performance, but it has a more spacious display and better speakers. It is about 0.6 pounds heavier, though. This is the best price to date for the base model with an M3 chip, 16GB of RAM and a 256GB SSD, but a version with double the storage space is available for a new low of $1,234 as well. The latter is also at Best Buy for $1,299.
Apple MacBook Air (13-inch, M2) for $799 at Amazon ($200 off): If you’re on a tighter budget, the last-generation MacBook Air gets you most of the things we love about the newer M3 model at a lower price. The main sacrifices are a slightly lesser chip, technically slower storage in the base model and no ability to connect to two external monitors. This deal is for the entry-level variant, which includes an M2 chip, 16GB of RAM and 256GB of storage — it’s $50 more than the all-time low we saw during Black Friday but still $200 less than buying from Apple directly. Also at Best Buy.
Apple MacBook Pro (14-inch, M4 Pro) for $2,099 at Amazon ($300 off):The recently updated MacBook Pro is more laptop than most people need, but it’s still a powerhouse for coders, media editors and those with more demanding needs. We gave it a 92 in our review earlier this month. Just keep in mind that there’s no USB-A port and, as with all Apple laptops, effectively no way to upgrade the device after purchase. This is the largest discount yet for the configuration with Apple’s M4 Pro chip, 24GB of RAM and a 1TB SSD. A version with 512GB of storage is $250 off and down to a low of $1,749 as well. Also at B&H.
Apple MacBook Pro (16-inch, M4 Pro) for $2,199 at Amazon ($300 off): The 16-inch MacBook Pro adds a roomier display and a couple extra hours of battery life to the same excellent performance, build quality and mini-LED display of its smaller counterpart. The downside is that it’s heavier at 4.7 pounds. This is an all-time low for the config with an M4 Pro chip, 24GB of RAM and a 512GB SSD. Also at B&H.
Microsoft Surface Laptop 7 (13.8-inch) for $899 at Amazon ($300 off): We gave the latest Surface Laptop a review score of 88 earlier this year, and we currently recommend it as a great mainstream ultraportable in our laptop buying guide. The caveat is that it runs on an ARM processor, which performs well but may not work with every app or peripheral you use. If you can live with that, though, the bright 120Hz display, upscale aluminum design and long battery life all impress. This configuration includes a Snapdragon X Plus chip, 16GB of RAM and a 512GB SSD. It was technically available for $800 at Micro Center last week, but only with in-store pick up. Otherwise, this matches the best price we’ve seen. If you prefer a bigger screen, a 15-inch version with a quicker Snapdragon X Elite chip and a 256GB SSD is down to a low of $1,053. Also at Best Buy.
HP Pavilion Aero 13 for $600 at HP ($300 off): The HP Pavilion Aero is the budget pick in our laptop buying guide. It feels a bit cheap, and its webcam is poor, but it’s thin and light, it has a decent port selection and its IPS display is solid for the money. This config with a Ryzen 5 8640U chip, 16GB of RAM and a 512GB SSD should be fast enough for typical web browsing, word processing and light photo editing. This is within $20 of the best price we could find for the 2024 model since its release.
Lenovo ThinkPad X1 Carbon (Gen 12) for $1,099 at Lenovo ($500 off): The ThinkPad X1 Carbon is an honorable mention in our laptop buying guide, and it earned a score of 87 in our review earlier this year. Its performance wasn’t quite top-of-the-line in benchmark testing, but it was still fast enough to handle the kind of web browsing, media editing and even light gaming that most people do with a notebook. The design is reasonably thin (0.6 inches), light (2.4 pounds) and highly durable, with a healthy array of ports (HDMI 2.1 included) and the kind of bouncy, responsive keyboard we’ve come to love from the ThinkPad line. The matte display virtually eliminates glare as well, though the 1080p webcam is just OK, and the side-mounted power button is somewhat awkward to press. This is the lowest price we’ve seen for the config with Intel’s Core Ultra 5 135U chip, 16GB of (non-upgradeable) RAM, a 512GB SSD and a 14-inch 1,920 x 1,200 display. Just enter the coupon code X1G12TDDEAL at checkout. If you need more power, another model with a faster Core Ultra 7 155U chip and 32GB of RAM is similarly discounted at $1,391. Use the code HOLIDAYTHINKDB for that one.
Lenovo Flex 5i Chromebook Plus for $349 at Best Buy ($150 off): The Flex 5i Chromebook Plus tops our guide to the best Chromebooks. We like that its IPS touchscreen has a tall 16:10 aspect ratio, as it makes the device better-suited to word processing and reading web pages. It runs just fine for the basic tasks you’re meant to do with Chrome OS, and both its keyboard and trackpad feel reliable. Its battery life is mediocre, however, and at 3.6 pounds it’s fairly heavy for a 14-inch notebook. This ties the lowest price we’ve seen for the config with an Intel Core i3-1315U chip, 8GB of RAM and 128GB of eMMC storage.
Alienware M18 R2 gaming laptop for $2,300 at Dell ($500 off): The M18 R2 is an iteration of the top 18-inch pick in our gaming laptop buying guide. It’s an absolute tank, but it’s put together well, and this configuration can blow through new games at high settings in 1080p or 1440p. It comes with a 165Hz 2,560 x 1,600 panel, an Intel Core i9-14900HX chip, 32GB of RAM, a 1TB SSD and an NVIDIA GeForce RTX 4080 GPU. This deal is $100 more than the price we saw on Black Friday, though it’s still a good ways off the notebook’s usual going rate. A lower-spec model is on sale for $1,900, though that one will run into issues sooner with more demanding games. Also at eBay via Dell.
ASUS ROG Strix G16 (2024) gaming laptop for $1,615 at Amazon ($285 off): While not a top pick in any of our guides, this configuration of the ROG Strix G16 should be a solid value for those in search of a 16-inch all-rounder gaming notebook. It includes an Intel Core i9-14900X chip, 16GB of RAM, a 1TB SSD and an NVIDIA GeForce RTX 4070 GPU, which should be enough to run many games at high-ish settings without much hassle. The overall build doesn’t feel cheap, and the 2,560 x 1,600 240Hz display is both sharp and fast. The RAM and storage are user-upgradeable, too, though the battery life and 720p webcam aren’t ideal. This deal matches an all-time low. Another model with a 1080p panel, a Core i7-13650HX chip and a RTX 4060 GPU is on sale for a low of $1,100, but you’ll have to turn down game settings more frequently to get decent frame rates with that one.
Apple iPad Pro (11-inch, M4) for $849 at Amazon ($100 off, clip $50 coupon): The iPad Pro is still overkill for most people, but its class-leading OLED display, remarkably slim frame and powerful M4 chip make it a tremendous tablet. It can work as a laptop replacement for certain creative types, but you’ll have to live with iPadOS’ continued shortcomings in multitasking, file management and other computer-y tasks. If you’re thinking of making the switch — or if you’ve just got money to burn and want a more portable companion to your current laptop — this matches the lowest price we’ve seen for the latest 11-inch model. Just clip the $50 on-page coupon. If you’re willing to pay for a roomier display, the 13-inch iPad Pro is also on sale for a low of $1,099.
This article originally appeared on Engadget at https://www.engadget.com/deals/the-best-cyber-monday-laptop-deals-for-2024-save-on-notebooks-from-apple-microsoft-lenovo-and-more-195505288.html?src=rss
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
cookielawinfo-checkbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.