Exploring Deepseek coder R1

Photo by Florian Olivo on Unsplash

I started exploring DeepSeek Coder R1 for all my coding work. My goal is to evaluate its performance and determine whether we can use it for the company and make it offline. This would save lots of resources.

Deepseek R1

Setting Up DeepSeek Coder

I began by cloning the DeepSeek Coder repository from GitHub (DeepSeek-Coder GitHub) and setting it up on a desktop with a small GPU. Given that DeepSeek Coder offers different model sizes, I wanted to experiment with what our current setup could handle.

One key observation was that the smaller model could be run locally on a laptop using Ollama, but the accuracy might be limited. This is a trade-off we need to consider for practical usage.

Scaling Up: Testing on Different Hardware

For more complex tasks, we considered loading the 8B model on a decent-sized server using only the CPU via Ollama or utilizing small GPUs. This seemed feasible, but we wanted to push the limits further.

A conversation with Ragavendra led to the decision to bring a desktop with an A100 Nvidia GPU to the office. While it isn’t a massive setup, it provides enough power to test larger models like the 14B parameter model locally.

Cloud Experiments and Cost Efficiency

If the tests on the desktop go well, the next step would be to request approval from CTO to use an AWS g4dn.4xlarge GPU instance. This setup is powerful enough to run a 70B parameter model via Ollama, giving us insights into how DeepSeek Coder performs under high loads.

Since AWS allows for on-demand usage, we plan to experiment and shut down the instance immediately after testing to keep costs minimal. This way, we get the insights we need without overspending on cloud resources.

Next Steps

• Complete local testing on the desktop with an A100 GPU.
• Evaluate model performance for coding and automation tasks.
• Determine whether an AWS GPU instance is required for large-scale deployment.
• Optimize costs by only using cloud resources for targeted experiments.

This exploration is an exciting step toward leveraging AI-powered coding solutions for our projects . With DeepSeek Coder R1, we’re looking to enhance automation, streamline workflows, and push the boundaries of AI-assisted development.

I will give more updates with testing results shortly