seal-rg/recurrent-pretraining

Code for large-scale depth-recurrent language model training

View on index · View in 3D Map

Related repos

• large language model tutorials and training basics from scratch
• Trains language models to generate self-edits via reinforcement learning
• Single-stage text-to-speech model with adversarial learning
• Code and labs for MIT’s deep learning course
• open-source reinforcement learning for language models
• open notes and guides for training large language models
• reinforcement learning library for training large language models, supports many RL algorithms, integrates with LLM tools, scales to models with up to 671B parameters
• Code completion and generation model
• speech-to-speech pipeline using open models for voice detection, transcription, language modeling, and speech synthesis
• Code and tutorials for building large language models from scratch in PyTorch