I have DeepSeek V4 Pro at home

Just wanted to share that I used u/LegacyRemaster slightly modified (Q4\_K\_M conversion support) DeepSeek V4 [CUDA repo](https://github.com/Fringe210/llama.cpp-deepseek-v4-flash-cuda) (based on u/antirez [work](https://github.com/antirez/llama.cpp-deepseek-v4-flash)) to convert and run Q4\_K\_M [DeepSeek V4 Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) on my Epyc workstation (Genoa 9374F, 12 x 96GB RAM, single RTX PRO 6000 Max-Q) and it worked right from the start: (base) phm@epyc:~/projects/llama.cpp-deepseek-v4-flash-cuda/build-cuda$ ./bin/llama-cli -m ../models/DeepSeek-V4-Pro-Q4_K_M.gguf --no-repack -ub 128 --chat-template-file ../models/templates/deepseek-ai-DeepSeek-V3.2.jinja ggml_cuda_init: found 1 CUDA devices (Total VRAM: 97247 MiB): Device 0: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition, compute capability 12.0, VMM: yes, VRAM: 97247 MiB Loading model... ▄▄ ▄▄ ██ ██ ██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄ ██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██ ██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀ ██ ██ ▀▀ ▀▀ build : b8936-44c7b01de model : DeepSeek-V4-Pro-Q4_K_M.gguf modalities : text available commands: /exit or Ctrl+C stop or exit /regen regenerate the last response /clear clear the chat history /read <file> add a text file /glob <pattern> add text files using globbing pattern > who are you? [Start thinking] Okay, the user is asking "who are you?" This is a simple, introductory question. I need to introduce myself clearly and warmly. I should state my name, creator, and key features that are most relevant to a new user. I can mention that I'm free, my context window, knowledge cutoff, file support, and availability on web and app. I'll end with an open invitation for further questions to keep the conversation going. [End thinking] Hi there! I'm DeepSeek, an AI assistant created by the Chinese company DeepSeek (深度求索). I'm here to help you with questions, creative tasks, problem-solving, and pretty much anything you're curious about! Here's a bit about me: - **Free to use** - no charges for chatting with me - **1M context window** - I can handle huge amounts of text at once (like entire book trilogies!) - **Knowledge cutoff: May 2025** - I'm reasonably up-to-date - **File upload support** - I can read text from images, PDFs, Word docs, Excel files, and more - **Web search capability** - though you need to manually enable it via the search button - **Available on web and mobile app** - with voice input support on the app I'm a pure text-based model, so I can't "see" images directly, but I can read any text in uploaded files. I aim to be warm, helpful, and detailed in my responses. What can I help you with today? 😊 [ Prompt: 12.2 t/s | Generation: 8.6 t/s ] > /exit Exiting... common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted | common_memory_breakdown_print: | - CUDA0 (RTX PRO 6000 Blackwell Max-Q Workstation Edition) | 97247 = 4022 + ( 92472 = 87766 + 84 + 4621) + 753 | common_memory_breakdown_print: | - Host | 793994 = 793954 + 0 + 39 | ~llama_context: CUDA_Host compute buffer size of 39.1719 MiB, does not match expectation of 15.3535 MiB The model file is 859GB.