What configuration is required to deploy deepseek-r locally?

The larger the number of parameters, the more complex the model is usually, and the stronger its performance, but the computational resource requirements and training costs are also higher.

1. DeepSeek 1.5B (1.5 billion parameters)
   Applicable scenarios: Lightweight tasks such as text generation, simple Q&A, etc.

CPU deployment:

Memory: 16GB RAM

Storage: 10GB hard drive space (for models and cache)

CPU： Modern processors with 4 or more cores (such as Intel i5 or AMD Ryzen 5)

GPU deployment (optional):

GPU: NVIDIA GTX 1660 or higher (4GB video memory)

Memory: 16GB RAM

Storage: 10GB hard drive space

1. DeepSeek 7B (7 billion parameters)
   Applicable scenarios: Medium complexity tasks such as text generation, translation, summarization, etc.

CPU deployment:

Memory: 32GB RAM

Storage: 20GB hard drive space

CPU： Modern processors with 8 or more cores (such as Intel i7 or AMD Ryzen 7)

GPU deployment (recommended):

GPU: NVIDIA RTX 3060 or higher (12GB video memory)

Memory: 32GB RAM

Storage: 20GB hard drive space

1. DeepSeek 14B (14 billion parameters)
   Applicable scenarios: Complex tasks such as long text generation, complex Q&A, etc.

CPU deployment:

Memory: 64GB RAM

Storage: 40GB hard disk space

CPU： Modern processors with 16 or more cores (such as Intel Xeon or AMD Ryzen 9)

GPU deployment (recommended):

GPU: NVIDIA RTX 3090 or higher (24GB video memory)

Memory: 64GB RAM

Storage: 40GB hard disk space

1. DeepSeek 32B (32 billion parameters)
   Applicable scenarios: High performance tasks such as large-scale text generation, complex reasoning, etc.

CPU deployment:

Memory: 128GB RAM

Storage: 100GB hard disk space

CPU： Server level processors with 32 or more cores (such as AMD EPYC or Intel Xeon)

GPU deployment (recommended):

GPU: NVIDIA A100 or higher (40GB video memory)

Memory: 128GB RAM

Storage: 100GB hard disk space

1. DeepSeek 70B (70 billion parameters)
   Applicable scenarios: Top performance tasks such as large-scale language understanding, complex reasoning, etc.

CPU deployment:

Memory: 256GB RAM

Storage: 200GB hard drive space

CPU： Server level processors with 64 or more cores

GPU deployment (recommended):

GPU： Multiple NVIDIA A100 (80GB VRAM) or H100

Memory: 256GB RAM

Storage: 200GB hard drive space

1. DeepSeek 671B (671 billion parameters)
   Applicable scenarios: ultra large scale tasks, such as ultra long text generation, complex scientific research, etc.

CPU deployment:

Memory: 1TB RAM or higher

Storage: 1TB hard drive space

CPU： Multi channel server level processors (such as AMD EPYC or Intel Xeon)

GPU deployment (recommended):

GPU： Multiple NVIDIA H100 or A100 (80GB video memory)

Memory: 1TB RAM or higher

Storage: 1TB hard drive space

General recommendations
GPU acceleration:

For 7B and above models, it is strongly recommended to use GPU acceleration, especially NVIDIA's high-performance graphics cards (such as RTX 3090, A100, H100).

The larger the video memory, the larger the batch size supported and the faster the speed.

Memory requirements:

The larger the number of model parameters, the higher the memory requirement. If there is insufficient memory, the inference speed will significantly decrease, and it may even fail to run.

Storage requirements:

Model files are usually large, especially for models of 14B and above, requiring sufficient hard disk space to store the model and cache.

Distributed deployment:

For models of 32B and above, it is recommended to use multi GPU or multi node distributed deployment to share computing pressure.

Cloud services:

If the local hardware is insufficient, you can consider deploying with cloud services such as AWS, Google Cloud, Azure, and choose high-performance GPU instances as needed.