机器学习与数据科学博士生系列论坛（第七十九期）—— A Practical Guide to Large Language Model Training
报告人： 杨潇博（北京大学）
时间：2024-11-14 16:00-17:00
地点：腾讯会议 568-7810-5726
Abstract:
Large Language Models (LLMs) have revolutionized natural language processing in recent years, demonstrating remarkable abilities in tasks ranging from text generation to reasoning. These models, trained on vast amounts of text data, have become increasingly powerful as they grow in size and complexity.
In this talk, I will introduce how to train large language models (LLMs) from scratch. I will start by introducing the basic building blocks of LLMs, the transformer architecture. Then, I will explain the typical three-stage training pipeline: from tokenizer training to pretraining, instruction tuning and finally reinforcement learning from human feedback (RLHF). Lastly, I'll discuss some practical challenges and solutions when scaling up these models, including scaling laws and various acceleration techniques.
About the Speaker:
论坛每次邀请一位博士生就某个前沿课题做较为系统深入的介绍，主题包括但不限于机器学习、高维统计学、运筹优化和理论计算机科学。
Your participation is warmly welcomed!
欢迎扫码关注北大统计科学中心公众号，了解更多讲座信息!
