Balancing sustainability in AI training: Addressing compute resources and environmental impact

In the rapidly advancing field of artificial intelligence, the training of AI models has become a cornerstone of technological progress. However, this progress comes with significant sustainability challenges.  

The process of training AI models requires immense compute resources, which in turn leads to substantial energy consumption and environmental impact. This article delves into the sustainability challenges associated with AI training, examining the compute resources required and the environmental footprint, while also exploring potential solutions to mitigate these impacts. Using examples like the Meta Llama models, we will highlight the importance of balancing technological advancement with environmental responsibility. 

1. Compute resources and storage needs: AI training involves processing vast amounts of data, which requires significant compute resources and storage capacity. For example, the Meta Llama 3.1 model has 405 billion parameters and requires 213GB of storage. The storage needs for AI training are substantial, as large datasets must be stored and accessed quickly to train models effectively. 
2. Environmental impact and storage efficiency: The environmental impact of AI training is partly due to the energy consumption of data centres, which house the storage infrastructure. Efficient storage solutions can help reduce the overall energy consumption and carbon footprint. For instance, the Meta Llama 3.2 Vision model required 2.02 million hours of training and emitted 584 tonnes of CO2 equivalent (tCO2e). Optimising storage efficiency can contribute to more sustainable AI training practices.
3. Optimising AI models and storage requirements: Reducing the size of AI models while maintaining their performance can decrease storage requirements. The Meta Llama 3.2 model was optimised to 3 billion parameters, reducing its size to just 2GB. Similarly, the lightweight version of the Meta Llama 3.2 Vision model has 11 billion parameters and requires only 8GB of storage. These optimisations help lower the storage demands for AI training.
4. AI Inferencing and memory needs: AI inferencing, which involves running trained models to make predictions or decisions, also requires substantial memory and storage. For example, running a model with 7 billion parameters requires approximately 8GB of DRAM. Optimising AI models for inferencing, such as through model quantisation, can help reduce memory and storage requirements.
5. Data centre architecture: Data centres are being rearchitected to handle neural computational workloads more efficiently, balancing higher compute density with lower power consumption. This includes optimising storage infrastructure to support the high data throughput and low latency required for AI training and inferencing.
6. Increased storage for AI-denerated data: AI generates large amounts of data, which must be stored and managed efficiently. For example, generative AI models like large language models (LLMs) and diffusion models produce significant amounts of output data. This drives the need for high-capacity, high-performance storage solutions to handle the increased data volume. 

The sustainability challenges of AI training are intricately linked to storage. Efficient storage solutions play a crucial role in reducing the environmental impact of AI training, optimising AI models, and managing the large volumes of data generated by AI applications. 

To read more about Futuresource Consulting’s comprehensive coverage in this area, including their storage media quarterly HDD, SSD, and DRAM trackers please contact Imad at imad.sarwar@futuresource-hq.com.

Press Contact: Nicola Finn, Marketing Manager, Futuresource Consulting, nicola.finn@futuresource-hq.com.