Code

Enable Flash Attention#1065

Code

Merged

IlyasMoutawwakil merged 24 commits intohuggingface:mainhuggingface/optimum-intel:mainfrom jiqing-feng:varlenjiqing-feng/optimum-intel:varlenCopy head branch name to clipboardJan 14, 2025

+182-39

Commits

Commits on Dec 11, 2024

• use real varlen attn

  Show description for 6d21075
  jiqing-feng

  committed
  6d21075

  Copy full SHA for 6d21075

Commits on Dec 12, 2024

• optimize gpt2 by using linear instead of conv1D

  Show description for b792875
  jiqing-feng

  committed
  b792875

  Copy full SHA for b792875

• Merge branch 'huggingface:main' into varlen

  jiqing-feng

  authored
  422134f

  Copy full SHA for 422134f

• fix usage without pkv

  Show description for 36884cb
  jiqing-feng

  committed
  36884cb

  Copy full SHA for 36884cb

• use sdpa for no cache forward

  Show description for d061e69
  jiqing-feng

  committed
  d061e69

  Copy full SHA for d061e69

• fix format

  Show description for 31c635a
  jiqing-feng

  committed
  31c635a

  Copy full SHA for 31c635a

• fix sdpa

  Show description for 73a5ef7
  jiqing-feng

  committed
  73a5ef7

  Copy full SHA for 73a5ef7

• revert shape for sdpa

  Show description for f9c021b
  jiqing-feng

  committed
  f9c021b

  Copy full SHA for f9c021b

• fix sdpa precision, still have error

  Show description for d069407
  jiqing-feng

  committed
  d069407

  Copy full SHA for d069407

Commits on Dec 13, 2024

• fix sdpa shape

  Show description for 2c54045
  jiqing-feng

  committed
  2c54045

  Copy full SHA for 2c54045

• upgrad minimum torch version to 2.5

  Show description for bce9aa9
  jiqing-feng

  committed
  bce9aa9

  Copy full SHA for bce9aa9

• rm pdb

  Show description for 72ac9e6
  jiqing-feng

  committed
  72ac9e6

  Copy full SHA for 72ac9e6

Commits on Dec 16, 2024

• fix non patch path

  Show description for 3fdb3a5
  jiqing-feng

  committed
  3fdb3a5

  Copy full SHA for 3fdb3a5

Commits on Dec 18, 2024

• Merge branch 'main' into varlen

  jiqing-feng

  authored
  7e20b86

  Copy full SHA for 7e20b86

Commits on Dec 25, 2024

• Merge branch 'huggingface:main' into varlen

  jiqing-feng

  authored
  c1bd7f7

  Copy full SHA for c1bd7f7

Commits on Jan 13, 2025

• Merge branch 'huggingface:main' into varlen

  jiqing-feng

  authored
  fb71c2e

  Copy full SHA for fb71c2e

Commits on Jan 14, 2025

• use varlen if flash attn not available

  Show description for 6186aaf
  jiqing-feng

  committed
  6186aaf

  Copy full SHA for 6186aaf

• revert ipex version change

  Show description for cbc232b
  jiqing-feng

  committed
  cbc232b

  Copy full SHA for cbc232b

• fix flash attn check

  Show description for 4dd2e44
  jiqing-feng

  committed
  4dd2e44

  Copy full SHA for 4dd2e44

• prefill attn

  Show description for 372d3f8
  jiqing-feng

  committed
  372d3f8

  Copy full SHA for 372d3f8

• fix cache

  Show description for daddabf
  jiqing-feng

  committed
  daddabf

  Copy full SHA for daddabf

• refactor attention

  Show description for 95b7043
  jiqing-feng

  committed
  95b7043

  Copy full SHA for 95b7043

• use flash attn for decode

  Show description for 71aa6b0
  jiqing-feng

  committed
  71aa6b0

  Copy full SHA for 71aa6b0

• fix dtype

  Show description for 9211803
  jiqing-feng

  committed
  9211803

  Copy full SHA for 9211803