Qwen3.5

Benchmark

The following three-machine comparison is based on the full formal matrix of lzc_thor running on 2026-03-20T05-47-33Z, retaining only the complete results from correct usage, without including experimental single-request data such as p1, cublas, or mmq.

Model	Best Parallel	Orin	Thor T4000	Thor T5000
Qwen3.5-0.8B-Q4_K_M	16	1326.86 tok/s / TTFT 1.94 s / 42.54 W	1311.50 tok/s / TTFT 1.88 s / 21.72 W	1783.26 tok/s / TTFT 1.37 s / 30.35 W
Qwen3.5-2B-Q4_K_M	16	989.57 tok/s / TTFT 2.91 s / 46.43 W	960.84 tok/s / TTFT 3.01 s / 22.46 W	1291.47 tok/s / TTFT 2.17 s / 32.93 W
Qwen3.5-4B-Q4_K_M	16	431.96 tok/s / TTFT 7.16 s / 48.48 W	442.65 tok/s / TTFT 7.07 s / 23.95 W	622.09 tok/s / TTFT 4.87 s / 35.87 W
Qwen3.5-9B-Q4_K_M	16	318.65 tok/s / TTFT 10.85 s / 56.49 W	314.55 tok/s / TTFT 11.23 s / 24.60 W	450.86 tok/s / TTFT 7.71 s / 35.85 W
Qwen3.5-27B-Q4_K_M	16	95.80 tok/s / TTFT 44.38 s / 56.82 W	90.82 tok/s / TTFT 48.44 s / 25.32 W	150.03 tok/s / TTFT 25.49 s / 36.91 W
Qwen3.5-35B-A3B-Q4_K_M	16	192.59 tok/s / TTFT 27.41 s / 43.12 W	181.57 tok/s / TTFT 30.34 s / 18.74 W	271.35 tok/s / TTFT 19.09 s / 31.05 W
Qwen3.5-122B-A10B-IQ2_S	16	65.14 tok/s / TTFT 46.02 s / 42.48 W	60.92 tok/s / TTFT 50.95 s / 19.49 W	76.55 tok/s / TTFT 43.03 s / 26.25 W

Latest T4000 SM110 Retest

2026-03-20T17-10-00Z's latest sm110 maxn retest has now completed Thor T4000, and is therefore listed separately to avoid mixing in incomplete T5000 / Orin data into the same comparison table.

Model	Best Parallel	Thor T4000
Qwen3.5-0.8B-Q4_K_M	16	1313.47 tok/s / TTFT 1.88 s / 21.73 W
Qwen3.5-2B-Q4_K_M	16	960.84 tok/s / TTFT 3.01 s / 22.84 W
Qwen3.5-4B-Q4_K_M	16	442.74 tok/s / TTFT 7.07 s / 23.95 W
Qwen3.5-9B-Q4_K_M	16	314.46 tok/s / TTFT 11.23 s / 24.60 W
Qwen3.5-27B-Q4_K_M	16	91.00 tok/s / TTFT 48.40 s / 25.33 W
Qwen3.5-35B-A3B-Q4_K_M	16	181.36 tok/s / TTFT 30.35 s / 18.74 W
Qwen3.5-122B-A10B-IQ2_S	16	60.67 tok/s / TTFT 51.23 s / 19.49 W

Notes

• Comparison data from three machines: /home/catdog/lzc_thor/docs/runs/2026-03-20T05-47-33Z
• Source of the latest sm110 retest: /home/catdog/lzc_thor/docs/runs/2026-03-20T17-10-00Z
• Qwen3.5-397B-A17B-IQ1_S did not form valid benchmarks with T4000 and Orin in the most recent test, thus it is not included in the official benchmark table.
• Power consumption column records the maximum power observed during testing.