@xiaomovps: After a company starts using AI, they quickly hit several hard problems: whether data can be externalized, whether costs can be controlled, and whether to build internal models themselves. This article documents a very real weekend operation—remotely connecting to the company's DGX Spark and hands-on running Ling-3.0-flash. Not stopping at 'can it run', but…

X AI KOLs Timeline News
ai-deployment local-inference data-privacy cost-control model-optimization hardware-setup

Summary

This article documents the complete process of the author remotely connecting to the company's DGX Spark server on the weekend to successfully deploy the Ling-3.0-flash model, including selection, deployment, performance testing, and integration with development tools, and shares insights on local deployment as a controllable intermediate state.

After a company starts using AI, they quickly encounter several hard problems: whether data can be externalized, whether costs can be controlled, and whether to build internal models themselves. This article documents a very real weekend operation—remotely connecting to the company's DGX Spark and hands-on running Ling-3.0-flash. Not stopping at 'can it run', but clearly writing out the complete path of selection, deployment, speed testing, and integration with development tools. The core judgment is straightforward: this machine is more suitable for running sparse MoE, rather than large Dense models. Ling has a total of 124B parameters, with only about 5.1B activated each time, which perfectly utilizes the unified memory. For the inference engine, vLLM was directly used, instead of the lighter Ollama—at this level of hardware, the difference in concurrency and throughput is too significant. The most interesting part is the final reflection: local deployment is not the endpoint, but a controllable intermediate state. External APIs handle quick calls, while internal machines manage sensitive data and continuous inference. Let AI assist as much as possible in what it can, but it's best to understand the process principles yourself; otherwise, troubleshooting issues becomes very difficult. For those who are hesitating about whether to implement local models for their team, this article is much more useful than simple benchmarks.
Original Article
View Cached Full Text

Cached at: 08/18/26, 12:26 PM

After companies start using AI, they quickly run into several hard questions: can data be transferred externally, can costs be controlled, and should we develop our own internal models?

This article documents a real weekend operation—remotely connecting to the company’s DGX Spark and personally getting Ling-3.0-flash up and running. It goes beyond merely testing “if it can run” and clearly outlines the complete path from model selection, deployment, speed testing, to integrating development tools.

The core judgment is straightforward: this machine is better suited for running sparse MoE (Mixture of Experts) rather than large Dense models. Ling has a total of 124 billion parameters, but only about 5.1 billion are activated each time, which perfectly utilizes the unified memory. For the inference engine, vLLM was chosen directly over the lighter Ollama—on hardware at this level, the differences in concurrency and throughput are too significant.

The most insightful part is the final reflection: local deployment isn’t an end goal but rather a controllable intermediate state. External APIs handle quick calls, while internal machines manage sensitive data and continuous inference. Let AI assist where it can, but it’s best to understand the underlying processes and principles yourself—otherwise, troubleshooting becomes difficult.

For those considering whether to implement local models for their teams, this article offers far more practical value than mere benchmark scores.

Similar Articles

Ling-3.0-flash MXFP4 released and running locally on one DGX Spark.

Reddit r/LocalLLaMA

Ling-3.0-flash MXFP4, a quantized model, has been released and runs locally on a single DGX Spark, achieving ~80 tok/s decoding and 2,500-3,500 tok/s long-input prefilling, enabling private on-device inference for coding, agents, and offline batch jobs.

@zhixianio: After receiving the new machine, I began an 'ascetic' practice of forcing myself to use local models for common tasks. I thought it would be painful, but both speed and quality greatly exceeded my expectations: Model: Qwen3.6-35B-A3B-oQ6-fp16-mtp, Running: oMLX, with N…

X AI KOLs Timeline

The author uses the Qwen3.6-35B-A3B model and oMLX tool on the new local machine for daily tasks, finding that both speed and quality far exceed expectations, even outperforming remote LLMs in PA and coding scenarios, demonstrating a significant improvement in on-device AI capabilities.