In today’s world, powerful language models are transforming how we interact with technology. But have you ever wondered about the differences in performance when running these models on a Apple GPU versus a CPU? This blog post dives into a fascinating experiment comparing the speeds of Microsoft Phi-4, a 9 billion parameter language model (Phi-4) on both a MacBook Pro’s CPU and the Apple GPU.
For the purposes of the test, I’m running all of this on a MacBook Pro M3 with 18GB Ram and 512GB SSD. This is a reasonably powerful computer, but far from high spec in this day and age.
The setup involved using N8N, a platform for deploying AI applications, and Docker containers to run the Phi-4 model. For details around how I set this up have a look at this link. The experiment focused on generating 500 words of text in response to a prompt.
If you wish to follow along with this experiment, then have a look at this video here:
CPU Performance: A Slower Journey
Running the Phi-4 model locally on the CPU proved challenging. Initially, an “out of system memory” error message popped up, indicating that the 11.2GB required by the model exceeded the available RAM. After increasing the allocated memory in Docker to 11.4GB, the process began. The CPU usage spiked as the 11 cores worked to process the request. However, generating 500 words took 168 seconds.
Apple GPU Performance: A Facster Experience
Switching to the GPU changed the story. The same Phi-4 model executed the task significantly faster. The GPU spiked whilst it handles the task, which completed in 73 seconds. Therefore being noticeably quicker than the CPU run. This highlights the inherent advantage GPUs have in handling computationally intensive tasks like language modeling.
The following stats were collected from the N8N output:
CPU Performance
- 168305ms – 168 seconds
- ~715 Tokens
GPU Performance
- 73679ms – 73 seconds
- ~732 Tokens
Why the Difference?
GPUs are specifically designed for parallel processing and highly complex geometry processing, making them ideal for the massive matrix calculations involved in training and running language models. CPUs, while powerful, are optimised for sequential processing. This architectural difference explains the significant performance gap observed in this experiment.
If you want to learn more about the high level concepts have a look at this great blog post from Pure: https://blog.purestorage.com/purely-educational/cpu-vs-gpu-for-machine-learning/
Key Takeaways:
- GPUs excel at handling computationally intensive tasks like language modeling.
- The choice between CPU and GPU depends on the specific application and resource constraints.
- For demanding tasks like running large language models, GPUs offer a substantial performance boost.
This experiment provides valuable insights into the capabilities of both CPUs and GPUs in the realm of AI. As language models continue to evolve and grow in complexity, understanding these performance differences will be crucial for developers and researchers alike.
No Responses