sequential-learning

The author shares early progress on Leo/PSCLS, an experimental system that learns sequence relationships and improves its story generation and metrics as it is trained on more stories.

This paper introduces Lie-bracket prediction of transfer order for sequential learning, using commutators of gradient fields to determine pairwise order and scaling to many domains. Experiments show high accuracy in predicting optimal curriculum orders for fine-tuning and instruction tuning.