What I just uploaded

I just put Nemotron-Terminal-32B-NVFP4 on Hugging Face! It is a 4-bit NVFP4 quant of nvidia/Nemotron-Terminal-32B. NVIDIA built the base on Qwen3-32B for terminal tasks, bash scripting and agent work. The model is at https://huggingface.co/kleinpanic93/Nemotron-Terminal-32B-NVFP4.

How I made it

It is post-training quantization with NVIDIA Model Optimizer 0.41.0, run on the GB10 in my DGX Spark. The run took 1478 seconds. Calibration used 512 samples of synthetic random data. Attention stays in BF16 and the lm_head is left out. The group size is 16 and the KV cache is not quantized.

The file metadata shows real packed 4-bit weights. The checkpoint is 20.67 GB in five safetensors shards. The card says about 20 GB against about 64 GB for BF16. The whole upload took about an hour. The base model is dense.

Run it

The card gives this command:

vllm serve kleinpanic93/Nemotron-Terminal-32B-NVFP4 \
  --quantization modelopt_fp4 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.72 \
  --trust-remote-code \
  --enable-chunked-prefill \
  --enable-prefix-caching

NVFP4 kernels need NVIDIA Blackwell, such as GB10 or GB200, or newer. It does not run on Hopper or older. The license is the NVIDIA open model license.

What is not done

There are no evals. I have not measured the quality or the speed of this quant.

What is next

If you run it, tell me how it does. I want to know.

P.S. Thanks for reading. Consider subbing to the RSS feed.
Klein Panic