58 lines
2.0 KiB
Markdown
58 lines
2.0 KiB
Markdown
<div align="center">
|
|
|
|
<img src="./images/logo.png"></img>
|
|
|
|
---
|
|
|
|
<img src="https://visitor-badge.laobi.icu/badge?page_id=jingyaogong/minimind"></img>
|
|
<img src="https://img.shields.io/github/stars/jingyaogong/minimind?style=social"></img>
|
|
<img src="https://img.shields.io/github/license/jingyaogong/minimind"></img>
|
|
<img src="https://img.shields.io/github/last-commit/jingyaogong/minimind"></img>
|
|
<img src="https://img.shields.io/badge/PRs-welcome-blue"></img>
|
|
<img src="https://img.shields.io/badge/🤗-MiniMind%20%20Collection-blue"></img>
|
|
|
|
---
|
|
|
|
</div>
|
|
|
|
|
|
|
|
<div align="center">
|
|
<h3>"The Greatest Path is the Simplest"</h3>
|
|
|
|
<a href="https://github.com/jingyaogong/minimind">🔗MiniMind Github Repo🔗</a>
|
|
</div>
|
|
|
|
|
|
|
|
* This open-source project aims to train a miniature language model **MiniMind** from scratch, with a size of just 26MB.
|
|
* **MiniMind** is extremely lightweight, approximately 1/7000 the size of GPT-3, designed to enable fast
|
|
inference and even training on CPUs.
|
|
* **MiniMind** is an improvement on the DeepSeek-V2 and Llama3 architectures. The project includes all stages of data
|
|
processing, pretraining, SFT, and DPO, and features a Mixture of Experts (MoE) model.
|
|
* This is not only the implementation of an open-source model, but also a tutorial for getting started with large
|
|
language models (LLMs).
|
|
* We hope that this project serves as a stepping stone for researchers and developers, providing an introductory example
|
|
to help them quickly get started and foster more exploration and innovation in the LLM field.
|
|
|
|
> To avoid any misunderstanding, "fastest 3 hours" refers to the requirement of using hardware with higher
|
|
specifications than the author's setup. Detailed specifications will be provided below.
|
|
|
|
|
|
|
|
<div align="center">
|
|
|
|
---
|
|
|
|
<img src="./images/streamlit.gif"></img>
|
|
|
|
<a href="https://www.modelscope.cn/studios/gongjy/minimind">ModelScope Online</a> |
|
|
<a href="https://www.bilibili.com/video/BV12dHPeqE72/?share_source=copy_web&vd_source=670c2504f88726f8cf4a21ef6147c0e8">Bilibili Video Link</a>
|
|
|
|
---
|
|
|
|
</div>
|
|
|
|
|
|
|