Imported from NatTuck/vllm-mi50-workspace (AGENTS.md). Install upstream with npx skills add NatTuck/vllm-mi50-workspace. Copyright stays with the author.

vllm for AMD gfx906 - Agent Guidelines

Problem Summary

Build vllm with Triton support for AMD gfx906 GPUs (MI50, MI60, Vega20, Radeon VII). These older AMD cards lack bfloat16 support and require specific patches to both Triton and vllm.

Target Environment: ROCm 7.12, Python 3.12.3

Reference Forks:

• refs/triton-gfx906/ - Working Triton build for gfx906
• refs/vllm-gfx906/ - Working vllm build for gfx906

(Don't change the reference forks. They are read-only references.)

Main Challenge: Patch current main branches of triton/ and vllm/ with gfx906 support from the reference forks while maintaining compatibility with ROCm 7.12.

Workspace Layout

/home/nat/Apps/src/vllm-mi50/
├── triton/                    # Main Triton source (v3.6.0)
├── vllm/                      # Main vllm source (latest)
├── refs/
│   ├── triton-gfx906/        # Reference Triton fork (v3.4.0-3.5.0 with gfx906)
│   └── vllm-gfx906/          # Reference vllm fork (gfx906 patches)
├── .venv/                     # Python virtual environment (to be created)
├── TODO.md                    # Detailed implementation plan
└── AGENTS.md                  # This file

Source Control

(TODO)

Work Rules

1. Don't Disable or Remove Tests

Keep all existing tests intact. If a test fails:

• Document the failure
• Investigate the root cause
• Fix the underlying issue
• Do NOT skip, disable, or delete tests to make builds pass

Tests are our safety net - they catch regressions and verify fixes actually work.

2. Don't Panic and Break Working Code

Once something works (e.g., Triton builds, vllm loads a model):

• Stop and verify it's actually working
• Document what made it work
• Move on to the next problem

If you encounter a new difficult problem:

• Don't randomly change the working parts
• Isolate the new problem
• Fix it without breaking what already works

Example: If Triton smoke tests pass but vllm fails to load a model, don't start changing Triton - focus on vllm-specific issues.

Quick Reference

Key gfx906 Limitations:

• No bfloat16 support (use float16/float32)
• Limited FP8 support
• Requires specific matrix operation layouts

Build Order:

1. Environment setup → 2. Triton → 3. Triton tests → 4. vllm → 5. vllm tests

Critical Files to Patch:

• triton/setup.py - AMD backend enablement
• triton/third_party/amd/backend/compiler.py - gfx906 architecture support
• vllm/platforms/rocm.py - gfx906 detection
• vllm/model_executor/layers/ - Matrix ops, bfloat16 workarounds

Getting Help

If stuck:

1. Check refs/triton-gfx906/ and refs/vllm-gfx906/ for working implementations
2. Review TODO.md for detailed plan and known issues
3. Test incrementally - don't make multiple big changes at once