feat(refit): load native MXFP8 weights into vLLM

[seonjinn]

Summary

Native MXFP8 training stores each quantized parameter as E4M3 values plus E8M0 scales. vLLM refit must place both tensors into the model's existing parameter storage before the normal weight-processing step runs.

This PR adds a version-aware vLLM refit adapter and binds ordered components from #3907 to their live destinations:

weight       -> parameter value storage
weight_scale -> parameter scale storage

The adapter restores loader-owned state before each refit, receives every component, runs vLLM's post-load processing once, and discards temporary state after success or failure.

What changes

• Add one lifecycle adapter for vLLM weight restore, load, process, and cleanup.
• Resolve native MXFP8 value and scale destinations without replacing parameter objects.
• Load each already-resharded local expert component in one wrapped loader call, rather than one call per expert. Keep the owned receive copy, gate/up slice boundaries, duplicate/missing-component checks, and native vLLM processing.
• Validate destination shape, dtype, placement, and component completeness before transfer.
• Preserve speculative decoding ownership: external draft.* weights stay on the external drafter path, co-trained MTP weights stay on the policy stream, and disk-loaded MTP weights are not overwritten.
• Finalize the main model, HPC state, and co-trained MTP drafter in that order.
• Reject unsupported native MXFP8 layouts early, including expert tensor parallelism.
• Keep BF16 and existing one-component refit paths unchanged.

vLLM's layerwise finalizer already reprocesses attention modules, including static FP8 KV-cache scales, so this PR does not run a second full-model post-load pass.

Dependencies

• Stacked on feat(refit): support ordered NCCL Reshard components #3907 for ordered refit components and feat(refit): export native MXFP8 training weights #3908 for the Megatron producer.
• End-to-end Ray validation uses fix(vllm): detect if quantize is mxfp8 from worker config #3653 for worker-config MXFP8 detection and fix(vllm): apply FP8 patches with RayExecutorV2 #3754 for RayExecutorV2 patch ordering; this PR does not duplicate those changes.
• This PR uses the existing NeMo-RL refit entry points and does not include the broader reload refactor from feat: Refit with vLLM reload_weights API #3651.

Tests

• Added lifecycle tests for repeated refit, corrected retry after setup failure, cleanup ownership, and post-load processing.
• Added destination tests for value/scale binding, geometry checks, completion fences, and unsupported layouts.
• Earlier Linux focused result: 42 refit-adapter tests passed. Existing MTP/drafter routing tests cover co-trained MTP, disk-loaded MTP, and Eagle3 gating.
• Added grouped value/scale byte checks and a Blackwell vLLM 0.25.1 test that compares repeated refits against independently initialized models, including padding and stable runtime pointers.
• The batching implementation passed the integration branch's 52 adapter tests on GB200 and a 20-step Qwen3-30B-A3B Async-1off run with fp8_param=true: W&B. That branch also contains lookup/selective-initialization optimizations not included in this update; it is not an exact-head validation of this PR. The port at fe415f92 has not yet been rerun on GB200.
• Ruff lint, Ruff format, and git diff --check pass locally. Full vLLM-backed tests are requested through CI.

[seonjinn]

/ok to test 7ec80f3

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: dfc57ce (PR #3909 from sna/native-mxfp8-vllm-receiver)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: c96dfd3 (PR #3909 from sna/native-mxfp8-vllm-receiver)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: e89c95a (PR #3909 from sna/native-mxfp8-vllm-receiver)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 4d9428f (PR #3909 from sna/native-mxfp8-vllm-receiver)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[seonjinn]

Additional GB200 validation completed on head 4d9428f21db2629499e95733c99a864e44632432.

Pinned dependencies:

• Megatron-Bridge: fcea83a855b20642ccb3d276706ce96834f7bad2
• Megatron-LM: bde6af2eb29325649dd71d41e584acb9782706f1
• TransformerEngine: 5e52befd5262c06289106338c308079d6adb391f
• vLLM: 0.26.0

Results:

• Dependency pins: 11 passed
• NVRx import and async cleanup: 5 passed
• Native MXFP8 source and refit source: 63 passed
• Precision matcher and NCCL metadata: 2 passed
• Megatron-Bridge export and Megatron-LM selective storage: 5 passed
• vLLM refit adapter: 49 passed
• vLLM backend selection: 1 passed
• NCCL reshard lifecycle and guards: 5 passed

All selected tests passed. The vLLM tests used a separate pinned environment because the repository intentionally marks the mcore and vllm extras as conflicting.