Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permute the pass pipeline to coalesce before setting up the matmul #2956

Merged
merged 1 commit into from
Dec 9, 2024

Conversation

alexbaden
Copy link
Contributor

@alexbaden alexbaden commented Dec 6, 2024

Required for #2834

Two reasons to do this - one, it properly tags the layouts with their memory order very early in the TTGIR pipeline. And two, it moves our TTGIR pipeline closer to upstream. I am splitting the change to isolate any regressions or undesired behavior caused by this change vs changing the DPAS layouts in #2834.

cc #2354

@alexbaden alexbaden marked this pull request as ready for review December 6, 2024 17:22
@alexbaden
Copy link
Contributor Author

Benchmark run: https://benchmarks.glados.intel.com/d/1pXX4hUSz/microbenchmarks?orgId=1&var-tag=ci%7Cci-adv%7Cci-dflt%7Cmovecoalesce&var-bench=All&var-device=Intel%28R%29%20Data%20Center%20GPU%20Max%201550&var-compiler=triton&var-compiler=xetla&var-compiler=onednn&var-backend=All&var-baseline_backend=xetla-ci-XPU%201550&var-target_backend=triton-ci-XPU%201550

Results appear to be in line with main, all deviations in the plots are within a min/max run from the last few days. This is ready for review.

@etiotto
Copy link
Contributor

etiotto commented Dec 6, 2024

I wonder whether FlashAttention (using the default codegen path + dev IGC) is ok. @alexbaden ?

@alexbaden
Copy link
Contributor Author

Is that covered by the attn plot in the microbenchmarks link I posted above?

@whitneywhtsang
Copy link
Contributor

Is that covered by the attn plot in the microbenchmarks link I posted above?

No, need to run with igc-18251 runner.

@alexbaden alexbaden merged commit cacfe10 into main Dec 9, 2024
6 checks passed
@alexbaden alexbaden deleted the alex/move_coalesce branch December 9, 2024 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants