github
/
minimind
mirror of https://github.com/jingyaogong/minimind.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update eval_model.py 

1. Fix use_moe, which is no way to turn off currently.
2. Change default value of hidden_size to 512 to match the default value in the training script.
pull/377/head
Howard Su 2025-04-27 07:45:09 +08:00 committed by GitHub
parent 765c79c7a2
commit 5f5bf34400
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
eval_model.py
View File

@ -107,10 +107,10 @@ def main():
# MiniMind2-moe (145M)：(hidden_size=640, num_hidden_layers=8, use_moe=True) # MiniMind2-moe (145M)：(hidden_size=640, num_hidden_layers=8, use_moe=True)
# MiniMind2-Small (26M)：(hidden_size=512, num_hidden_layers=8) # MiniMind2-Small (26M)：(hidden_size=512, num_hidden_layers=8)
# MiniMind2 (104M)：(hidden_size=768, num_hidden_layers=16) # MiniMind2 (104M)：(hidden_size=768, num_hidden_layers=16)
parser.add_argument('--hidden_size', default=640, type=int) parser.add_argument('--hidden_size', default=512, type=int)
parser.add_argument('--num_hidden_layers', default=8, type=int) parser.add_argument('--num_hidden_layers', default=8, type=int)
parser.add_argument('--max_seq_len', default=8192, type=int) parser.add_argument('--max_seq_len', default=8192, type=int)
parser.add_argument('--use_moe', default=True, type=bool) parser.add_argument('--use_moe', action='store_true', help="Enable MOE")
# 携带历史对话上下文条数 # 携带历史对话上下文条数
# history_cnt需要设为偶数，即【用户问题, 模型回答】为1组；设置为0时，即当前query不携带历史上文 # history_cnt需要设为偶数，即【用户问题, 模型回答】为1组；设置为0时，即当前query不携带历史上文
# 模型未经过外推微调时，在更长的上下文的chat_template时难免出现性能的明显退化，因此需要注意此处设置 # 模型未经过外推微调时，在更长的上下文的chat_template时难免出现性能的明显退化，因此需要注意此处设置