超低成本20GB显存，448GB/s带宽，仅需100美元。

以下是用100美元显卡所能达到的上限：你可以拥有3个并发用户，充足的上下文，速度优于或接近于那些显存更少但价格高出4倍以上的显卡组合。 0.00.008.388 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.008.391 I device_info: 0.00.089.439 I - CUDA0 : NVIDIA P102-100 (10144 MiB, 10013 MiB free) 0.00.197.645 I - CUDA1 : NVIDIA P102-100 (10144 MiB, 10013 MiB free) 0.00.197.656 I - CPU : Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz (128396 MiB, 128396 MiB free) 0.00.197.728 I system_info: n_threads = 6 (n_threads_batch = 6) / 12 | CUDA : ARCHS = 600,610,750,860,890 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.197.764 I srv init: running without SSL 0.00.197.849 I srv init: using 11 threads for HTTP server 0.00.198.515 I srv start: binding port with default address family 0.00.199.823 I srv llama_server: loading model 0.00.199.902 I srv load_model: loading model '/models/Qwen3.6-35B-A3B-UD-IQ4_XS.gguf' 0.00.199.906 I common_init_result: fitting params to device memory ... 0.00.199.907 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.00.987.288 W common_fit_params: failed to fit params to free device memory: n_gpu_layers already set by user to 99, abort 0.23.223.625 W llama_context: n_ctx_seq (32768) < n_ctx_train (262144) -- the full capacity of the model will not be utilized 0.23.481.073 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.23.570.914 I srv load_model: initializing slots, n_slots = 3 0.23.598.842 W srv load_model: speculative decoding will use checkpoints 0.23.598.851 W common_speculative_init: no implementations specified for speculative decoding 0.23.598.852 I slot load_model: id 0 | task -1 | new slot, n_ctx = 32768 0.23.598.854 I slot load_model: id 1 | task -1 | new slot, n_ctx = 32768 0.23.598.854 I slot load_model: id 2 | task -1 | new slot, n_ctx = 32768 0.23.598.961 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.23.598.963 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.23.598.964 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.23.598.965 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192 0.23.598.985 I srv init: idle slots will be saved to prompt cache upon starting a new task 0.23.628.848 I init: chat template, example_format: '<|im_start|>system You are a helpful assistant<|im_end|> <|im_start|>user Hello<|im_end|> <|im_start|>assistant Hi there<|im_end|> <|im_start|>user How are you?<|im_end|> <|im_start|>assistant <think> </think> ' 0.23.666.546 I srv init: init: chat template, thinking = 0 0.23.666.572 I srv llama_server: model loaded 0.23.666.575 I srv llama_server: server is listening on http://127.0.0.1:5802 0.23.666.579 I srv update_slots: all slots are idle 0.48.181.695 I srv operator(): Chat format: peg-native 0.48.182.094 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1 0.48.182.101 I srv get_availabl: updating prompt cache 0.48.182.111 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.48.182.123 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 98304 tokens, 8589934592 est) 0.48.182.128 I srv get_availabl: prompt cache update took 0.02 ms 0.48.182.302 I slot launch_slot_: id 2 | task 0 | processing task, is_child = 0 0.48.182.309 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.48.182.311 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.48.186.009 I srv operator(): Chat format: peg-native 0.48.189.081 I srv operator(): Chat format: peg-native 0.49.483.103 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1 0.49.483.111 I srv get_availabl: updating prompt cache 0.49.483.116 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.49.483.119 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 98304 tokens, 8589934592 est) 0.49.483.120 I srv get_availabl: prompt cache update took 0.01 ms 0.49.483.178 I slot launch_slot_: id 1 | task 2 | processing task, is_child = 0 0.49.483.179 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.49.483.181 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 0.49.483.181 I srv get_availabl: updating prompt cache 0.49.483.182 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.49.483.183 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 98304 tokens, 8589934592 est) 0.49.483.183 I srv get_availabl: prompt cache update took 0.00 ms 0.49.483.215 I slot launch_slot_: id 0 | task 3 | processing task, is_child = 0 0.51.242.275 I slot create_check: id 0 | task 3 | created context checkpoint 1 of 32 (pos_min = 1376, pos_max = 1376, n_tokens = 1377, size = 62.813 MiB) 0.51.367.765 I slot create_check: id 1 | task 2 | created context checkpoint 1 of 32 (pos_min = 670, pos_max = 670, n_tokens = 671, size = 62.813 MiB) 0.51.367.773 I slot print_timing: id 2 | task 0 | prompt processing, n_tokens = 1377, progress = 1.00, t = 3.19 s / 432.28 tokens per second 0.51.480.037 I slot create_check: id 2 | task 0 | created context checkpoint 1 of 32 (pos_min = 1376, pos_max = 1376, n_tokens = 1377, size = 62.813 MiB) 0.56.647.801 I slot print_timing: id 0 | task 3 | n_decoded = 100, tg = 23.30 t/s, tg_3s = 23.30 t/s 0.56.653.219 I slot print_timing: id 2 | task 0 | n_decoded = 100, tg = 23.30 t/s, tg_3s = 23.30 t/s 0.56.692.218 I slot print_timing: id 1 | task 2 | n_decoded = 100, tg = 23.54 t/s, tg_3s = 23.54 t/s 0.59.655.679 I slot print_timing: id 0 | task 3 | n_decoded = 171, tg = 23.43 t/s, tg_3s = 23.60 t/s 0.59.661.606 I slot print_timing: id 2 | task 0 | n_decoded = 171, tg = 23.42 t/s, tg_3s = 23.60 t/s 0.59.702.608 I slot print_timing: id 1 | task 2 | n_decoded = 171, tg = 23.56 t/s, tg_3s = 23.58 t/s 1.02.659.591 I slot print_timing: id 0 | task 3 | n_decoded = 242, tg = 23.49 t/s, tg_3s = 23.64 t/s 1.02.665.066 I slot print_timing: id 2 | task 0 | n_decoded = 242, tg = 23.49 t/s, tg_3s = 23.64 t/s 1.02.705.486 I slot print_timing: id 1 | task 2 | n_decoded = 242, tg = 23.58 t/s, tg_3s = 23.64 t/s 1.03.253.784 I slot print_timing: id 0 | task 3 | prompt eval time = 2873.48 ms / 1381 tokens ( 2.08 ms per token, 480.60 tokens per second) 1.03.253.789 I slot print_timing: id 0 | task 3 | eval time = 10897.06 ms / 256 tokens ( 42.57 ms per token, 23.49 tokens per second) 1.03.253.791 I slot print_timing: id 0 | task 3 | total time = 13770.54 ms / 1637 tokens 1.03.253.792 I slot print_timing: id 0 | task 3 | graphs reused = 253 1.03.253.924 I slot release: id 0 | task 3 | stop processing: n_tokens = 1636, truncated = 0 1.03.259.600 I slot print_timing: id 2 | task 0 | prompt eval time = 4178.32 ms / 1381 tokens ( 3.03 ms per token, 330.52 tokens per second) 1.03.259.605 I slot print_timing: id 2 | task 0 | eval time = 10898.93 ms / 256 tokens ( 42.57 ms per token, 23.49 tokens per second) 1.03.259.606 I slot print_timing: id 2 | task 0 | total time = 15077.26 ms / 1637 tokens 1.03.259.607 I slot print_timing: id 2 | task 0 | graphs reused = 253 1.03.259.741 I slot release: id 2 | task 0 | stop processing: n_tokens = 1636, truncated = 0 1.03.288.482 I slot print_timing: id 1 | task 2 | prompt eval time = 2960.66 ms / 1381 tokens ( 2.14 ms per token, 466.45 tokens per second) 1.03.288.486 I slot print_timing: id 1 | task 2 | eval time = 10844.49 ms / 256 tokens ( 42.36 ms per token, 23.61 tokens per second) 1.03.288.487 I slot print_timing: id 1 | task 2 | total time = 13805.15 ms / 1637 tokens 1.03.288.488 I slot print_timing: id 1 | task 2 |