What's this about?
LoGRA stores compact low-rank sketches of gradients for model updates and policy synchronization, with a controller that limits steps predicted to change the policy too sharply.
What exactly is a low-rank sketch?
It approximates a huge update matrix using a few dominant directions, trading a little detail for much lower memory use.
Why does this matter?
The reported 45.7% memory reduction enabled 27B training on eight GPUs where dense Adam failed. Broader models, tasks and distributed setups still need validation.