# Instructions for the ROCm MTP and VEC investigation

## Purpose

This directory stores the patches, launch notes, validation logs, and session
history for the ROCm MTP fitting and HIP VEC flash-attention work.

The patch author and maintainer is eaman. Historical Qwen and Sol labels refer
only to LLM-assisted development series and must not be presented as authorship.
Use “Eaman patch” for the current project and keep old Qwen/Sol names only when
identifying legacy branches, filenames, logs, or historical stages.

## Repository and branch policy

- This notes repository is `/home/eaman/llama/bug`.
- Keep this repository on `master`.
- The source repository is `/home/eaman/llama/llama.cpp`.
- Keep the source repository on `eaman` when working on this investigation.
- Do not commit or modify `master` in `llama.cpp` unless the user explicitly asks.
- Do not push or create pull requests.

## Canonical artifacts

- `summary_eaman.md` is the main technical record and current status document.
- `status` is the short operational snapshot.
- `sol4.patch` and `rocm_improvement.patch` are historical complete-patch names.
- `latest_rocm_improvement_<base>.patch` is the current cumulative Eaman patch:
  ROCm fitting/pipeline improvements, MTP Compact Rollback, and HIP VEC control.
- `mtp_compact_rollback_<base>.patch` is the standalone MTP Compact Rollback
  patch without the cumulative ROCm, pipeline, HIP VEC, or MoE changes.
- `latest_rs_cumulative_*` and `mtp_rs_only_*` are superseded historical exports;
  do not use their old recurrent-state names for current artifacts or controls.
- `logs/` contains validation logs.
- `old/patches/` contains superseded patches such as `sol2.patch` and `sol3.patch`.
- `llama_scripts/` contains active local launchers.
- `dual_gpu_context_balancing_guide.md` is the reusable manual procedure for
  converting asymmetric dual-GPU headroom into context with tensor overrides.
- Do not recreate the removed `vec_update.md` or `think_pipeline_off_f16.sh` files.

## Current source state

- Source branch: `eaman`.
- Current `eaman` tip is merge `9ee6674f`, integrating local `latest` mainline
  commit `662a0b01`. Recovery branch `eaman-pre-latest-662a0b01` preserves the
  previous verified cumulative tip `527be7e6`.
- Standalone branch `eaman-mtp-compact-rollback` is merge `cdc3d024` on exact
  base `662a0b01`; recovery branch
  `eaman-mtp-compact-rollback-pre-662a0b01` preserves `6cb89357`. Its public
  control is `--spec-mtp-cr-depth` with environment
  variable `LLAMA_ARG_SPEC_MTP_CR_DEPTH`.
- Current exports are `mtp_compact_rollback_662a0b01.patch` and cumulative
  `latest_rocm_improvement_662a0b01.patch`.
- Combined HIP/Vulkan builds validated both branches on pinned ROCm core-7.14 /
  HIP 7.14.60850 and isolated ROCm core-10.0 / HIP 7.15.26333. Standalone build
  1362 and cumulative build 1376 linked, enumerated both GPUs, passed the parser,
  and passed all four generated-model/recurrent tests, including DSV4.
- The complete cumulative build 1279 binary/library directory remains deployed
  at `/home/eaman/llama/bin_vulkan`. The canonical source `build/bin` is now the
  pinned core-7.14 build 1376; it has not been deployed. The prior deployment
  used by the already-running Vulkan process remains preserved at
  `/home/eaman/llama/bin_vulkan_previous`.
- The initial Q4/Q4 MTP VEC run reached 147,712 context tokens and completed
  startup plus short generation. The former open-ended long-prompt and
  delayed-OOM follow-up is archived under `old/` and is no longer a current
  refresh acceptance criterion.

### MTP Compact Rollback

- MTP Compact Rollback is integrated into production `eaman` and also exported
  independently for mainline testing.
- It includes on-device replay checkpoints, exact aligned per-device fitter
  reservation, startup-pinned per-slot allocation, sticky and catchable host
  fallback, fragmented-layout host fallback, and validated two-slot execution.
- `--hip-fa-force-vec on|off` belongs only to the cumulative Eaman patch. It
  defaults to `off`; `on` forces the supported quantized-KV HIP VEC path.
  This dispatch control is HIP-specific, not a Vulkan or CUDA forcing control.
- Existing matched dense/MoE depth tests, a 1,024-token replay run, direct
  two-slot reservation, and concurrent two-slot ROCm execution passed on the
  preceding base. Startup scratch-fit caching is an optional efficiency
  improvement rather than a correctness prerequisite.

## Validation workflow

1. Confirm both repository branches and clean worktrees before acting.
2. Apply patches and build in a temporary detached worktree; never build by
   modifying the user's `master` checkout.
3. A CPU/default build checks general compilation.
4. A HIP build is needed to compile `ggml/src/ggml-cuda/fattn.cu` and the
   `fattn-vec` instances.
5. Report separately whether the HIP backend compiled and whether the final
   application link completed.
6. Remove temporary worktrees after validation.
7. Record new measured results in `summary_eaman.md` and keep `status` concise.

## Daily mainline refresh

- Use the repo-local `$update-llama-eaman` skill in
  `.agents/skills/update-llama-eaman/` for the recurring update workflow.
- Preserve and commit the previous known-good Eaman patch on `eaman` before advancing
  the local `latest` branch.
- Fast-forward `latest` from `origin/master`; do not update or modify the local
  source `master` branch.
- Merge `latest` into `eaman`, adapt the Eaman patch delta, build the HIP/Vulkan binaries,
  and generate `latest_rocm_improvement_<base>.patch` from `latest..eaman`.
- The standalone patch header must record its exact llama.cpp base, the build
  number reported by the freshly built binary, and the tested ROCm/Vulkan
  environment.
- Validate clean application and tracked-tree equality in a temporary detached
  worktree before updating checksums and current-state documentation.
- After every successful refresh, keep only the new `eaman-pre-latest-<base>`
  recovery branch. Delete older matching branch names with `git branch -d` only
  after verifying that each tip is an ancestor of current `eaman`; retain and
  report any branch that fails the ancestry check.

## Documentation rules

- Use paths from the current layout: logs are under `logs/`, and old patches are
  under `old/patches/`.
- Keep historical results labeled as historical; do not present the initial VEC
  result as a completed stability benchmark.
- Update checksums and commit identifiers when replacing the canonical patch.
- Keep `summary_eaman.md` for detailed history and `AGENTS.md` for stable workflow
  rules; do not duplicate large experiment logs here.