feat: AdaLoRA and other improvements

Ekaterina Nikonovarequested to merge
feat/adalora into main
  • Fixed flash attention (vRAM improvements 8xa100 to 4xa100)
  • Implemented custom trainer with AdaLoRA
  • Migrated to the new GCP
  • General bug fixes and improvements

Related #512373

Edited by Ekaterina Nikonova

Merge request reports