Code

[Zerobubble] Merge Main.#6107

Code

Merged

duanjunwen merged 190 commits intohpcaitech:feature/zerobubblehpcaitech/ColossalAI:feature/zerobubblefrom duanjunwen:dev/zero_bubbleduanjunwen/ColossalAI:dev/zero_bubbleCopy head branch name to clipboardNov 5, 2024

+1,685-834

Commits

Commits on Jul 1, 2024

• fp8 operators for compressed communication

  Show description for f5a52e1
  BurkeHulk

  authored
  f5a52e1

  Copy full SHA for f5a52e1

Commits on Jul 4, 2024

• Merge branch 'hpcaitech:main' into feature/fp8_comm

  BurkeHulk

  authored
  6991819

  Copy full SHA for 6991819

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for e17f835
  pre-commit-ci[bot]

  committed
  e17f835

  Copy full SHA for e17f835

Commits on Jul 10, 2024

• fix typo

  GuangyaoZhang

  committed
  dbfa7d3

  Copy full SHA for dbfa7d3

Commits on Jul 12, 2024

• fix scaling algorithm in FP8 casting

  BurkeHulk

  committed
  1e19594

  Copy full SHA for 1e19594

• support fp8 communication in pipeline parallelism

  BurkeHulk

  committed
  e881901

  Copy full SHA for e881901

• add fp8_communication flag in the script

  BurkeHulk

  committed
  6601874

  Copy full SHA for 6601874

• Merge remote-tracking branch 'origin/feature/fp8_comm' into feature/fp8_comm

  Show description for 1f1b856
  BurkeHulk

  committed
  1f1b856

  Copy full SHA for 1f1b856

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 51f916b
  pre-commit-ci[bot]

  committed
  51f916b

  Copy full SHA for 51f916b

Commits on Jul 16, 2024

• Merge pull request hpcaitech#5885 from BurkeHulk/feature/fp8_comm

  Show description for 9470701
  BurkeHulk

  authored
  9470701

  Copy full SHA for 9470701

• shardformer fp8

  GuangyaoZhang

  committed
  457a0de

  Copy full SHA for 457a0de

Commits on Jul 17, 2024

• fix rebase

  GuangyaoZhang

  committed
  5a310b9

  Copy full SHA for 5a310b9

• remove all to all

  GuangyaoZhang

  committed
  6a20f07

  Copy full SHA for 6a20f07

Commits on Jul 18, 2024

• Merge pull request hpcaitech#5899 from BurkeHulk/SP_fp8

  Show description for d0bdb51
  GuangyaoZhang

  authored
  d0bdb51

  Copy full SHA for d0bdb51

• fix shardformer fp8 communication training degradation

  GuangyaoZhang

  committed
  5b969fd

  Copy full SHA for 5b969fd

• Merge pull request hpcaitech#5921 from BurkeHulk/fp8_fix

  Show description for 62661cd
  GuangyaoZhang

  authored
  62661cd

  Copy full SHA for 62661cd

Commits on Jul 24, 2024

• [fp8] support all-gather flat tensor (hpcaitech#5932)

  ver217

  authored
  5fd0592

  Copy full SHA for 5fd0592

Commits on Aug 2, 2024

• [fp8] add fp8 comm for low level zero

  ver217

  committed
  ae486ce

  Copy full SHA for ae486ce

• [test] add zero fp8 test case

  ver217

  committed
  91e596d

  Copy full SHA for 91e596d

• Merge pull request hpcaitech#5961 from ver217/feature/zeor-fp8

  Show description for c297e21
  BurkeHulk

  authored
  c297e21

  Copy full SHA for c297e21

Commits on Aug 5, 2024

• [Feature] llama shardformer fp8 support (hpcaitech#5938)

  Show description for 53cb960
  GuangyaoZhang

  authored
  53cb960

  Copy full SHA for 53cb960

Commits on Aug 6, 2024

• [FP8] rebase main (hpcaitech#5963)

  Show description for 0c10afd
  

  18 peopleauthored
  0c10afd

  Copy full SHA for 0c10afd

• [fp8]support all2all fp8 (hpcaitech#5953)

  Show description for afb26de
  

  flybird11111

  and

  pre-commit-ci[bot]

  authored
  afb26de

  Copy full SHA for afb26de

Commits on Aug 7, 2024

• [fp8] add fp8 linear (hpcaitech#5967)

  Show description for 76ea164
  ver217

  authored
  76ea164

  Copy full SHA for 76ea164

• [fp8] support fp8 amp for hybrid parallel plugin (hpcaitech#5975)

  Show description for ccabcf6
  ver217

  authored
  ccabcf6

  Copy full SHA for ccabcf6

• fix (hpcaitech#5976)

  flybird11111

  authored
  7739629

  Copy full SHA for 7739629

Commits on Aug 8, 2024

• [Feature]: support FP8 communication in DDP, FSDP, Gemini (hpcaitech#5928)

  Show description for b480eec
  

  BurkeHulk

  and

  pre-commit-ci[bot]

  authored
  b480eec

  Copy full SHA for b480eec

• [test ci]Feature/fp8 comm (hpcaitech#5981)

  Show description for 4b9bec8
  flybird11111

  authored
  4b9bec8

  Copy full SHA for 4b9bec8

Commits on Aug 9, 2024

• [fp8] support gemini plugin (hpcaitech#5978)

  Show description for 8241c0c
  ver217

  authored
  8241c0c

  Copy full SHA for 8241c0c

• [fp8] use torch compile (torch >= 2.3.0) (hpcaitech#5979)

  Show description for e4aadee
  botbw

  authored
  e4aadee

  Copy full SHA for e4aadee

• [fp8]Moe support fp8 communication (hpcaitech#5977)

  Show description for f1a3a32
  

  flybird11111

  and

  pre-commit-ci[bot]

  authored
  f1a3a32

  Copy full SHA for f1a3a32

Commits on Aug 12, 2024

• [fp8] support hybrid parallel plugin (hpcaitech#5982)

  Show description for b2483c8
  wangbluo

  authored
  b2483c8

  Copy full SHA for b2483c8

Commits on Aug 13, 2024

• [fp8] refactor fp8 linear with compile (hpcaitech#5993)

  Show description for 0978080
  ver217

  authored
  0978080

  Copy full SHA for 0978080

Commits on Aug 14, 2024

• [fp8] support asynchronous FP8 communication (hpcaitech#5997)

  Show description for 597b206
  

  flybird11111

  and

  pre-commit-ci[bot]

  authored
  597b206

  Copy full SHA for 597b206

Commits on Aug 15, 2024

• [fp8] update torch.compile for linear_fp8 to >= 2.4.0 (hpcaitech#6004)

  botbw

  authored
  88fa096

  Copy full SHA for 88fa096

• [fp8] linear perf enhancement

  botbw

  committed
  1a2e90d

  Copy full SHA for 1a2e90d

• [fp8]update reduce-scatter test (hpcaitech#6002)

  Show description for 20722a8
  flybird11111

  authored
  20722a8

  Copy full SHA for 20722a8

Commits on Aug 16, 2024

• [fp8] add use_fp8 option for MoeHybridParallelPlugin (hpcaitech#6009)

  wangbluo

  authored
  3f09a61

  Copy full SHA for 3f09a61

• [fp8] zero support fp8 linear. (hpcaitech#6006)

  Show description for 0a51319
  flybird11111

  authored
  0a51319

  Copy full SHA for 0a51319

Commits on Aug 17, 2024

• merge

  wangbluo

  committed
  4cf79fa

  Copy full SHA for 4cf79fa

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 81272e9
  pre-commit-ci[bot]

  committed
  81272e9

  Copy full SHA for 81272e9

Commits on Aug 19, 2024

• fix the merge

  wangbluo

  committed
  02636c5

  Copy full SHA for 02636c5

• Merge branch 'fp8_merge' of https://github.com/wangbluo/ColossalAI into fp8_merge

  wangbluo

  committed
  52289e4

  Copy full SHA for 52289e4

• fix the merge

  wangbluo

  committed
  1a5847e

  Copy full SHA for 1a5847e

• fix the merge

  wangbluo

  committed
  3353042

  Copy full SHA for 3353042

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 64aad96
  pre-commit-ci[bot]

  committed
  64aad96

  Copy full SHA for 64aad96

• fix the merge

  wangbluo

  committed
  4c82bfc

  Copy full SHA for 4c82bfc

• Merge branch 'fp8_merge' of https://github.com/wangbluo/ColossalAI into fp8_merge

  wangbluo

  committed
  0d8e82a

  Copy full SHA for 0d8e82a

• fix

  wangbluo

  committed
  12b4401

  Copy full SHA for 12b4401

• fix

  wangbluo

  committed
  2eb3683

  Copy full SHA for 2eb3683

• fix the merge

  wangbluo

  committed
  88b3f06

  Copy full SHA for 88b3f06

• fix

  wangbluo

  committed
  1f703e0

  Copy full SHA for 1f703e0

Commits on Aug 20, 2024

• fix

  wangbluo

  committed
  5382311

  Copy full SHA for 5382311

• fix

  wangbluo

  committed
  f7acfa1

  Copy full SHA for f7acfa1

• fix

  wangbluo

  committed
  2ee6235

  Copy full SHA for 2ee6235

• fix

  wangbluo

  committed
  2e4cbe3

  Copy full SHA for 2e4cbe3

• fix merge

  wangbluo

  committed
  2d362ac

  Copy full SHA for 2d362ac

Commits on Aug 21, 2024

• fix the merge

  wangbluo

  committed
  eb5ba40

  Copy full SHA for eb5ba40

• fix

  wangbluo

  committed
  193030f

  Copy full SHA for 193030f

• fix

  wangbluo

  committed
  6aface9

  Copy full SHA for 6aface9

• fix

  wangbluo

  committed
  698c8b9

  Copy full SHA for 698c8b9

• fix

  wangbluo

  committed
  8b8e282

  Copy full SHA for 8b8e282

Commits on Aug 22, 2024

• [fp8] Merge feature/fp8_comm to main branch of Colossalai (hpcaitech#6016)

  Show description for eea37da
  

  17 peopleauthored
  eea37da

  Copy full SHA for eea37da

• Merge pull request hpcaitech#6023 from wangbluo/fp8_merge

  Show description for d77e66a
  wangbluo

  authored
  d77e66a

  Copy full SHA for d77e66a

• fix

  wangbluo

  committed
  971b16a

  Copy full SHA for 971b16a

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for a292554
  pre-commit-ci[bot]

  committed
  a292554

  Copy full SHA for a292554

• Merge pull request hpcaitech#6024 from wangbluo/fix_merge

  Show description for afe845f
  wangbluo

  authored
  afe845f

  Copy full SHA for afe845f

• Merge branch 'main' into feature/fp8_comm

  ver217

  authored
  caab4a3

  Copy full SHA for caab4a3

Commits on Aug 23, 2024

• Update train_dpo.py

  flybird11111

  authored
  0bc9a87

  Copy full SHA for 0bc9a87

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3b0df30
  pre-commit-ci[bot]

  committed
  3b0df30

  Copy full SHA for 3b0df30

• Update low_level_zero_plugin.py

  flybird11111

  authored
  9e76764

  Copy full SHA for 9e76764

• Merge pull request hpcaitech#6029 from hpcaitech/flybird11111-patch-1

  Show description for 0bf46c5
  wangbluo

  authored
  0bf46c5

  Copy full SHA for 0bf46c5

Commits on Aug 26, 2024

• fix

  wangbluo

  committed
  dae3999

  Copy full SHA for dae3999

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 80d24ae
  pre-commit-ci[bot]

  committed
  80d24ae

  Copy full SHA for 80d24ae

• Merge pull request hpcaitech#6033 from wangbluo/fix

  Show description for 4a6f31e
  wangbluo

  authored
  4a6f31e

  Copy full SHA for 4a6f31e

Commits on Aug 27, 2024

• Merge pull request hpcaitech#6012 from hpcaitech/feature/fp8_comm

  Show description for 17904cb
  ver217

  authored
  17904cb

  Copy full SHA for 17904cb

• [CI] Remove triton version for compatibility bug; update req torch >=2.2 (hpcaitech#6018)

  Show description for d383449
  Edenzzzz

  authored
  d383449

  Copy full SHA for d383449

Commits on Aug 28, 2024

• [plugin] hotfix zero plugin (hpcaitech#6036)

  Show description for cc1b0ef
  ver217

  authored
  cc1b0ef

  Copy full SHA for cc1b0ef

• [Colossal-LLaMA] Refactor latest APIs (hpcaitech#6030)

  Show description for 4a68efb
  

  TongLi3701

  and

  pre-commit-ci[bot]

  authored
  4a68efb

  Copy full SHA for 4a68efb

• add fused norm (hpcaitech#6038)

  TongLi3701

  authored
  0d3a85d

  Copy full SHA for 0d3a85d

Commits on Aug 29, 2024

• [FP8] unsqueeze scale to make it compatible with torch.compile (hpcaitech#6040)

  GuangyaoZhang

  authored
  e96a076

  Copy full SHA for e96a076

Commits on Sep 2, 2024

• [colossalai/checkpoint_io/...] fix bug in load_state_dict_into_model; format error msg (hpcaitech#6020)

  Show description for e9032fb
  

  flymin

  and

  pre-commit-ci[bot]

  authored
  e9032fb

  Copy full SHA for e9032fb

Commits on Sep 3, 2024

• [Hotfix] Remove deprecated install (hpcaitech#6042)

  Show description for c650a90
  TongLi3701

  authored
  c650a90

  Copy full SHA for c650a90

• [fp8] optimize all-gather (hpcaitech#6043)

  Show description for c3b5caf
  ver217

  authored
  c3b5caf

  Copy full SHA for c3b5caf

• [fp8] fix linear hook (hpcaitech#6046)

  ver217

  authored
  26e5539

  Copy full SHA for 26e5539

Commits on Sep 9, 2024

• [fp8] disable all_to_all_fp8 in intranode (hpcaitech#6045)

  Show description for 5ce6dd7
  

  BurkeHulk

  and

  pre-commit-ci[bot]

  authored
  5ce6dd7

  Copy full SHA for 5ce6dd7

Commits on Sep 10, 2024

• [release] update version (hpcaitech#6041)

  Show description for b3db105
  ver217

  authored
  b3db105

  Copy full SHA for b3db105

• [Feature] Split cross-entropy computation in SP (hpcaitech#5959)

  Show description for 8fd25d6
  

  Edenzzzz

  and

  pre-commit-ci[bot]

  authored
  8fd25d6

  Copy full SHA for 8fd25d6

• [hotfix] moe hybrid parallelism benchmark & follow-up fix (hpcaitech#6048)

  Show description for c54c4fc
  botbw

  authored
  c54c4fc

  Copy full SHA for c54c4fc

Commits on Sep 11, 2024

• [fp8] hotfix backward hook (hpcaitech#6053)

  Show description for 13946c4
  ver217

  authored
  13946c4

  Copy full SHA for 13946c4

• [doc] update sp doc (hpcaitech#6055)

  Show description for a35a078
  

  flybird11111

  and

  pre-commit-ci[bot]

  authored
  a35a078

  Copy full SHA for a35a078

Commits on Sep 13, 2024

• fix the sp

  wangbluo

  committed
  fdd84b9

  Copy full SHA for fdd84b9

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 216d54e
  pre-commit-ci[bot]

  committed
  216d54e

  Copy full SHA for 216d54e

• fix the attn

  wangbluo

  committed
  0a01e2a

  Copy full SHA for 0a01e2a

• fix

  wangbluo

  committed
  683179c

  Copy full SHA for 683179c

• fix

  wangbluo

  committed
  6eb8832

  Copy full SHA for 6eb8832

• fix

  wangbluo

  committed
  f393867

  Copy full SHA for f393867

• fix

  wangbluo

  committed
  dc03217

  Copy full SHA for dc03217

• [fp8] fix missing fp8_comm flag in mixtral (hpcaitech#6057)

  botbw

  authored
  696fced

  Copy full SHA for 696fced

• fix

  wangbluo

  committed
  0b14a55

  Copy full SHA for 0b14a55

• fix

  wangbluo

  committed
  0ad3129

  Copy full SHA for 0ad3129

• fix

  wangbluo

  committed
  b582319

  Copy full SHA for b582319

Commits on Sep 14, 2024

• [fp8] Disable all_gather intranode. Disable Redundant all_gather fp8 (hpcaitech#6059)

  Show description for f20b066
  GuangyaoZhang

  authored
  f20b066

  Copy full SHA for f20b066

• [doc] FP8 training and communication document (hpcaitech#6050)

  Show description for bdb125f
  GuangyaoZhang

  authored
  bdb125f

  Copy full SHA for bdb125f

• fix

  wangbluo

  committed
  827ef3e

  Copy full SHA for 827ef3e

• Merge pull request hpcaitech#6061 from wangbluo/sp_fix

  Show description for 37e3523
  wangbluo

  authored
  37e3523

  Copy full SHA for 37e3523

Commits on Sep 16, 2024

• fix

  wangbluo

  committed
  10e4f7d

  Copy full SHA for 10e4f7d

Commits on Sep 18, 2024

• Merge pull request hpcaitech#6064 from wangbluo/fix_attn

  Show description for 63314ce
  wangbluo

  authored
  63314ce

  Copy full SHA for 63314ce

• [moe] add parallel strategy for shared_expert && fix test for deepseek (hpcaitech#6063)

  botbw

  authored
  4fa6b95

  Copy full SHA for 4fa6b95

• [ColossalEval] support for vllm (hpcaitech#6056)

  Show description for f9546ba
  

  Camille7777

  and

  pre-commit-ci[bot]

  authored
  f9546ba

  Copy full SHA for f9546ba

Commits on Sep 19, 2024

• [release] update version (hpcaitech#6062)

  ver217

  authored
  dabc2e7

  Copy full SHA for dabc2e7

Commits on Sep 25, 2024

• release FP8 news (hpcaitech#6068)

  Show description for cbaa104
  binmakeswell

  authored
  cbaa104

  Copy full SHA for cbaa104

• fix the ring attn

  wangbluo

  committed
  cfd9eda

  Copy full SHA for cfd9eda

• fix the attn

  wangbluo

  committed
  65c8297

  Copy full SHA for 65c8297

• fix

  wangbluo

  committed
  6fb1322

  Copy full SHA for 6fb1322

• fix

  wangbluo

  committed
  91ed32c

  Copy full SHA for 91ed32c

• fix

  wangbluo

  committed
  6705dad

  Copy full SHA for 6705dad

Commits on Sep 26, 2024

• add funding news (hpcaitech#6072)

  Show description for f4daf04
  binmakeswell

  authored
  f4daf04

  Copy full SHA for f4daf04

• fix

  wangbluo

  committed
  3fab921

  Copy full SHA for 3fab921

Commits on Oct 9, 2024

• fix

  wangbluo

  committed
  3532f77

  Copy full SHA for 3532f77

• [feat] support zbv in mixtral benchmark;

  duanjunwen

  committed
  3f5bec8

  Copy full SHA for 3f5bec8

• fix

  wangbluo

  committed
  b635dd0

  Copy full SHA for b635dd0

Commits on Oct 10, 2024

• [fix] MixtralForCausalLMPolicy get_held_layer support zbv;

  duanjunwen

  committed
  9ee80fc

  Copy full SHA for 9ee80fc

• [feat] update MixtralPipelineForwards --> mixtral_model_forward; support zbv;

  duanjunwen

  committed
  72b507a

  Copy full SHA for 72b507a

• [shardformer] fix linear 1d row and support uneven splits for fused qkv linear (hpcaitech#6084)

  Show description for 646b3c5
  ver217

  authored
  646b3c5

  Copy full SHA for 646b3c5

• [feat] support MixtralPipelineForwards--> mixtral_for_causal_lm_forward for zbv

  duanjunwen

  committed
  e234dfa

  Copy full SHA for e234dfa

• fix

  wangbluo

  committed
  f98384a

  Copy full SHA for f98384a

• fix

  wangbluo

  committed
  5ecc27e

  Copy full SHA for 5ecc27e

• Update README.md (hpcaitech#6087)

  Show description for 6b2c506
  supercooledith

  authored
  6b2c506

  Copy full SHA for 6b2c506

• fix

  wangbluo

  committed
  efe3042

  Copy full SHA for efe3042

Commits on Oct 11, 2024

• [shardformer] optimize seq parallelism (hpcaitech#6086)

  Show description for dc2cdaf
  ver217

  authored
  dc2cdaf

  Copy full SHA for dc2cdaf

• fix

  wangbluo

  committed
  0002ae5

  Copy full SHA for 0002ae5

• fix

  wangbluo

  committed
  1507a75

  Copy full SHA for 1507a75

• [fix] fix llama, mixtral benchmark zbv loss none bug; update mixtral & llama policy and modeling;

  duanjunwen

  committed
  0ca16d5

  Copy full SHA for 0ca16d5

• fix the test

  wangbluo

  committed
  4e0e99b

  Copy full SHA for 4e0e99b

• fix the test

  wangbluo

  committed
  703bb5c

  Copy full SHA for 703bb5c

• [Coati] Train DPO using PP (hpcaitech#6054)

  Show description for 4c8e85e
  

  TongLi3701

  and

  pre-commit-ci[bot]

  authored
  4c8e85e

  Copy full SHA for 4c8e85e

Commits on Oct 14, 2024

• fix

  wangbluo

  committed
  e1e86f9

  Copy full SHA for e1e86f9

• fix

  wangbluo

  committed
  d891e50

  Copy full SHA for d891e50

• [feat] Linear1D_COL/ROW support zbv WeightGradStore;

  duanjunwen

  committed
  cfade4c

  Copy full SHA for cfade4c

• [feat] support use_zbv in llama, mixtral modeling; only replace Linear1D_Col/Row policy;

  duanjunwen

  committed
  a11b4b5

  Copy full SHA for a11b4b5

• [fix] fix test case; moe error in second iter

  duanjunwen

  committed
  abd4551

  Copy full SHA for abd4551

• [feat]EPMixtralSparseMoeBlock (op in MOE) support zbv;

  duanjunwen

  committed
  160e9a4

  Copy full SHA for 160e9a4

• fix

  wangbluo

  committed
  23199e3

  Copy full SHA for 23199e3

• fix

  wangbluo

  committed
  3201377

  Copy full SHA for 3201377

• fix

  wangbluo

  committed
  fe9208f

  Copy full SHA for fe9208f

• fix

  wangbluo

  committed
  8ff7d0c

  Copy full SHA for 8ff7d0c

Commits on Oct 15, 2024

• fix

  wangbluo

  committed
  3dc08c8

  Copy full SHA for 3dc08c8

• fix

  wangbluo

  committed
  6be9862

  Copy full SHA for 6be9862

• fix

  wangbluo

  committed
  fd92789

  Copy full SHA for fd92789

• fix

  wangbluo

  committed
  bc7eead

  Copy full SHA for bc7eead

• [fix] fix bwd b; now bwd w only for Layer replaced by Linear1D_Col/Row; other layer perform a fully bwd;

  duanjunwen

  committed
  9912cc8

  Copy full SHA for 9912cc8

• Merge branch 'feature/zerobubble' of github.com:hpcaitech/ColossalAI into dev/zero_bubble

  duanjunwen

  committed
  52dcc73

  Copy full SHA for 52dcc73

• fix

  wangbluo

  committed
  83cf2f8

  Copy full SHA for 83cf2f8

• Merge pull request hpcaitech#6071 from wangbluo/ring_attention

  Show description for dcd41d0
  wangbluo

  authored
  dcd41d0

  Copy full SHA for dcd41d0

• [fix] debug zbv llama test;

  duanjunwen

  committed
  90939b7

  Copy full SHA for 90939b7

Commits on Oct 16, 2024

• [Ring Attention] Improve comments (hpcaitech#6085)

  Show description for 62c13e7
  Edenzzzz

  authored
  62c13e7

  Copy full SHA for 62c13e7

• [fix] rm use_zbv flag in Shardconfig; rm debug info;

  duanjunwen

  committed
  e76308c

  Copy full SHA for e76308c

• [fix] add & fix llama test

  duanjunwen

  committed
  705b18e

  Copy full SHA for 705b18e

• [pipeline] hotfix backward for multiple outputs (hpcaitech#6090)

  Show description for cd61353
  ver217

  authored
  cd61353

  Copy full SHA for cd61353

Commits on Oct 17, 2024

• [ckpt] add safetensors util

  botbw

  committed
  2bcd0b6

  Copy full SHA for 2bcd0b6

• [chore] refactor

  botbw

  committed
  3b1d7d1

  Copy full SHA for 3b1d7d1

Commits on Oct 18, 2024

• [fp8] add fallback and make compile option configurable (hpcaitech#6092)

  ver217

  authored
  5ddad48

  Copy full SHA for 5ddad48

• [misc] fit torch api upgradation and remove legecy import (hpcaitech#6093)

  Show description for 58d8b8a
  ver217

  authored
  58d8b8a

  Copy full SHA for 58d8b8a

Commits on Oct 21, 2024

• [release] update version (hpcaitech#6094)

  ver217

  authored
  19baab5

  Copy full SHA for 19baab5

• fix lora ckpt save format (ColoTensor to Tensor)

  BurkeHulk

  committed
  b10339d

  Copy full SHA for b10339d

• pre-commit fix

  BurkeHulk

  committed
  6d6cafa

  Copy full SHA for 6d6cafa

• Merge pull request hpcaitech#6096 from BurkeHulk/hotfix/lora_ckpt

  Show description for dee63cc
  BurkeHulk

  authored
  dee63cc

  Copy full SHA for dee63cc

Commits on Oct 24, 2024

• [extension] hotfix compile check (hpcaitech#6099)

  ver217

  authored
  80a8ca9

  Copy full SHA for 80a8ca9

• [doc] sora solution news (hpcaitech#6100)

  Show description for 4294ae8
  binmakeswell

  authored
  4294ae8

  Copy full SHA for 4294ae8

• [feat] support meta cache, meta_grad_send, meta_tensor_send; fix runtime too long in Recv Bwd; benchmark for llama + Hybrid(tp+pp);

  duanjunwen

  committed
  2eca112

  Copy full SHA for 2eca112

• [MCTS] Add self-refined MCTS (hpcaitech#6098)

  Show description for 89a9a60
  

  TongLi3701

  and

  pre-commit-ci[bot]

  authored
  89a9a60

  Copy full SHA for 89a9a60

Commits on Oct 25, 2024

• [fix\ fix fail case test_shard_llama

  duanjunwen

  committed
  d0ec221

  Copy full SHA for d0ec221

• [fix] fix test_shard_llama

  duanjunwen

  committed
  cc0dfdd

  Copy full SHA for cc0dfdd

• [fix] fix llama modeling policy;

  duanjunwen

  committed
  03fa79a

  Copy full SHA for 03fa79a

Commits on Oct 28, 2024

• [fix] fix test_shard_llama ci;

  duanjunwen

  committed
  6377aa0

  Copy full SHA for 6377aa0

• [fix] fix test zerobubble

  duanjunwen

  committed
  5aee426

  Copy full SHA for 5aee426

Commits on Oct 29, 2024

• [fix] fix handle name; rm useless comments;

  duanjunwen

  committed
  fafe049

  Copy full SHA for fafe049

• [fix] fix send recv signature;

  duanjunwen

  committed
  fa3ccda

  Copy full SHA for fa3ccda

• [fix] fix comment in llama & benchmark

  duanjunwen

  committed
  982e4ee

  Copy full SHA for 982e4ee

Commits on Oct 30, 2024

• [feat] support no tensor parallel Linear in shardformer; Add test for use weightGradStore and not use WeightGradStore

  duanjunwen

  committed
  d2e05a9

  Copy full SHA for d2e05a9

Commits on Oct 31, 2024

• [fix] fix linear (no tp) ops func name;

  duanjunwen

  committed
  5f09243

  Copy full SHA for 5f09243

• [checkpointio] fix hybrid plugin model save (hpcaitech#6106)

  ver217

  authored
  c2e8f61

  Copy full SHA for c2e8f61

• [pre-commit.ci] pre-commit autoupdate (hpcaitech#6078)

  Show description for 2f583c1
  pre-commit-ci[bot]

  authored
  2f583c1

  Copy full SHA for 2f583c1

Commits on Nov 1, 2024

• Merge branch 'main' into dev/zero_bubble

  duanjunwen

  committed
  1d328ff

  Copy full SHA for 1d328ff

• Merge branch 'feature/zerobubble' of github.com:hpcaitech/ColossalAI into dev/zero_bubble

  duanjunwen

  committed
  c82c75a

  Copy full SHA for c82c75a

• [fix] fix fp8 args in HybridParallel

  duanjunwen

  committed
  3b5c314

  Copy full SHA for 3b5c314

• [fix] fix hybridparall use_fp8 config

  duanjunwen

  committed
  5b5fbcf

  Copy full SHA for 5b5fbcf

• [fix] fix use_fp8 flag

  duanjunwen

  committed
  0218e67

  Copy full SHA for 0218e67

• [fix] fix model zoo init

  duanjunwen

  committed
  8e40087

  Copy full SHA for 8e40087