Issues

• [example] Add example on using Nadir Custom Optimizers with HuggingFace

  example
  OptimalFoundation/nadir#63 · Filter by author bhavnicksm opened on Jun 19, 2024
  

• Assert "valid" values for hyperparams in the Config

  OptimalFoundation/nadir#62 · Filter by author bhavnicksm opened on Jun 15, 2024
  

• Adam optimizer .eval() not working well with ScheduleFree

  bug
  OptimalFoundation/nadir#61 · Filter by author bhavnicksm opened on Jun 15, 2024
  1

• Create apply_update() fn to manage post update transformations

  OptimalFoundation/nadir#59 · Filter by author bhavnicksm opened on Jun 15, 2024
  

• Add SAdam (SoftPlus the denom)

  OptimalFoundation/nadir#58 · Filter by author bhavnicksm opened on Jun 9, 2024
  

• Add AdamD Optimizer

  OptimalFoundation/nadir#57 · Filter by author bhavnicksm opened on Jun 9, 2024
  

• Add Gradient Centralization for all the optimizers

  enhancement
  OptimalFoundation/nadir#51 · Filter by author bhavnicksm opened on Jun 8, 2024
  

• Refactor Optimizer Config API

  enhancementwontfix
  OptimalFoundation/nadir#45 · Filter by author bhavnicksm opened on Jun 1, 2024
  2

  

• Create a template to quickly add and test new optimisers

  OptimalFoundation/nadir#41 · Filter by author bhavnicksm opened on May 31, 2024
  

• Add Sophia Optimiser

  OptimalFoundation/nadir#40 · Filter by author bhavnicksm opened on May 30, 2024
  1

  

• GaLore-ify every optimiser

  enhancementhelp wanted
  OptimalFoundation/nadir#39 · Filter by author bhavnicksm opened on May 30, 2024
  

• Implement Optimizers

  enhancementhelp wanted
  OptimalFoundation/nadir#22 · Filter by author bhavnicksm opened on Mar 3, 2023
  

• Fix documentation

  OptimalFoundation/nadir#10 · Filter by author bhavnicksm opened on Feb 20, 2023
  1