Code

Hybrid Offloading for ZeRO3#5625

Code

Draft

tohtana wants to merge 3 commits intomicrosoft:mastermicrosoft/DeepSpeed:masterfrom tohtana:tohtana/hybrid_offloadtohtana/DeepSpeed:tohtana/hybrid_offloadCopy head branch name to clipboard

+195-2

Commits

Commits on Jun 6, 2024

• add hybrid offloading

  tohtana

  committed
  1bfd0c2

Commits on Jun 7, 2024

• disable fetching parameters

  tohtana

  committed
  a7f8c69

• Merge branch 'master' into tohtana/hybrid_offload

  tohtana

  authored
  9fba8e1