So, I’ve been super curious about diving deeper into the world of large language models (LLMs) and decided to give it a shot right here on my MacBook M3 Pro with 18GB of RAM. Let me walk you through my adventure as I explore running DeepSeek R1 locally—and running language models locally is a game-chager for me.
DeepSeek R1 – Are you Sure?
So the models I’m using are fine tuned by DeepSeek R1 and downloaded via Ollama. This is the only realistic way of running a version of DeepSeek locally as the full model would require somewhere in the region of 1TB of VRAM, which unless you are hosting some kind of monster server farm at home, then it’s not really an option to run the full-fat DeepSeek R1 at home. For those interested, you can access DeepSeek via API and pay to play, but that’s not what I want to be doing. So I’ll be making the use of the tools at my disposal.
Why Local AI? The Power of Running Models on My Desktop
Over the last couple of years I’ve been a big fan of experimenting with AI tools. But there’s something incredibly satisfying about doing everything right here in my own environment. This means being able to run models locally without worrying about data privacy or cloud dependencies.
I chose DeepSeek R1 for a couple of reasons:
- Performance: With its 18GB RAM and powerful M3 Pro chip, my MacBook is set to handle reasonably demanding tasks. Plus, unlike the free versions of OpenAI etc the model isn’t limited when there is high demand on the servers.
- Versatility: I wanted to test out different model sizes and see how they perform in real-world scenarios. Smaller models like 1.5B parameters are perfect for tasks where speed is key and the potential quality trade off is accepted.
I’ve produced the following video which runs you through the whole testing process:
The Test began: Speed Matters (Even if it’s My Own Brain!)
To kick things off, I decided to write a brief prompt about artificial intelligence and see how each model fared in terms of speed versus output quality.
- First Attempt: Running the 7B parameter model took me around 30 seconds. This was to produce a 500 word ‘essay’ on the subject of articificial intelligence. I haven’t paid too much attention to the qulaity of the output, as this will get tested over the next few weeks – where I will start to find out the relative strengths and weaknesses of the 7B parameter model.
- Second Attempt: Using a slightly different 500 word prompt, I saw a noticable improvement—about 14 seconds faster! Again, the quality has not been looked into, but this was better than I’d expected.
A Closer Look: Comparing the Competition
I wanted to make sure I wasn’t letting the bigger model slide by in terms of performance. So, I compared against a previous test of running Phi-4 on the same MacBook. You can read and watch about the results here.
- 500 Words Test: DeepSeek R1 handled it in 22 seconds, while Phi-4 took around 36 seconds for the same prompt. The difference is impressive, which leads me to want to experiment more and work with it to determine it’s relative qualities.
Use Cases: Beyond Just Writing with DeepSeek R1
Okay, so I’m all about practical applications. Here’s why running these models locally is a big deal for me:
- Content Creation: Need to rewrite a piece of content? DeepSeek R1 can handle that in under one minute; definitely a time-saver and you should always proof read every first draft, regardless of model you are using.
- Personal Projects: Whether it’s analysing data, writing essays, or brainstorming ideas, having these tools right here means I’m not reliant on external platforms and I can build my own knowledge base locally.
Privacy and Security: A Win-Win
Another huge plus is the privacy aspect. Running models locally means all the data stays within my own networks. So there is no need to worry about third-party servers or data leaks. This is especially important for sensitive projects or personal tasks where you don’t want to run the risk of data getting exposed to others.
Final Thoughts
In a nutshell, running DeepSeek R1 on my MacBook M3 Pro has been an a worth while experience. It’s fast, seemingly efficient, and opens up a whole new realm of possibilities for me. Plus, being able to control data privacy works for my use cases.
So whether you’re a casual AI enthusiast or someone who works with sensitive data, I highly recommend giving local AI a try. Who knows? Your next breakthrough idea could be right here on your desktop!
No Responses