Qwen3 VL Embedding 2B on Thor

Use the official vllm container image on the thor machine to test Qwen/Qwen3-VL-Embedding-2B. The current container image combination cannot run successfully.

Test Configuration

• Device: thor
• Container image: nvcr.io/nvidia/vllm:26.01-py3
• Container vllm version: 0.13.0+faa43dbf.nv26.01
• Model: Qwen/Qwen3-VL-Embedding-2B
• Try path:
• Offline LLM(..., runner="pooling")
  • HTTP vllm serve --runner pooling
• Try parameters:
  • --gpu-memory-utilization 0.65 --max-model-len 8192 --enforce-eager
  • --gpu-memory-utilization 0.5 --max-model-len 8192 --enforce-eager
• dtype=float16 also performed additional validation
• Dataset: datasets/qwen3_vl_embedding.jsonl
• Synthesized resource: scripts/prepare_qwen3_vl_assets.py

Conclusion

• The official container image can recognize the model and enter the initialization process of runner=pooling.
• When gpu-memory-utilization=0.65 is encountered, startup fails immediately due to insufficient idle GPU memory.
• Reduced GPU memory usage to 0.5 and proceeded to the visual encoding profile stage, but initialization still failed.
• The root cause is not in the script parameters, but in the current official container image's vllm + torch combination: the visual path in Qwen3-VL-Embedding-2B reaches torch.nn.functional.conv3d and triggers RuntimeError: GET was unable to find an engine to execute this computation.
• float16 further validation yielded the same result, confirming this is not an isolated issue with the bfloat16 path.

Failure Summary

• 0.65 configuration error:
  • Free memory on device (32.52/59.9 GiB) on startup is less than desired GPU memory utilization (0.65, 38.94 GiB).
• 0.5 and float16 configuration errors:
  • RuntimeError: GET was unable to find an engine to execute this computation
  • Triggered at the qwen3_vl.py visual encoding path's F.conv3d(...) stage

Telemetry

• Sample count: 22
• Maximum temperature: 32.593 C
• Maximum power consumption: 6887 mW
• Maximum fan PWM: null
• Maximum GPU utilization: null

Artifacts

Remote result directory:

/home/nvidia/bench-results/qwen3-vl-embedding-vllm

Only telemetry files have been generated; offline results and HTTP probe are not available.

/home/nvidia/bench-results/qwen3-vl-embedding-vllm/telemetry.jsonl
/home/nvidia/bench-results/qwen3-vl-embedding-vllm/telemetry.summary.json