79e36e17e8
Fix: Remove attn_mask when using is_causal=True in Flash Attention
...
PyTorch's scaled_dot_product_attention cannot use attn_mask and
is_causal=True simultaneously. This commit sets attn_mask=None
when is_causal=True to avoid parameter conflicts.
2025-10-18 21:02:46 +08:00
4e35fb9da8
[fix] update model
2025-10-17 00:09:32 +08:00
5ffde04b7c
update lora
2025-04-27 15:45:06 +08:00
29454c31af
fix bugs
2025-04-27 09:56:49 +08:00
274483cb1b
250426
2025-04-26 10:07:55 +08:00
a62faf34bd
250426
2025-04-26 10:05:47 +08:00
d9453ed9a3
update moe note
2025-04-09 17:38:31 +08:00
4a7c1c49e8
update rlaif
2025-04-05 16:06:08 +08:00
9e67798397
update generate
2025-04-05 15:53:55 +08:00
399d526fbd
add hidden state
2025-04-05 14:39:56 +08:00
ed01c5d84a
update inference
2025-04-05 12:03:04 +08:00
bf81fd5f5e
rmsnorm float convert
2025-04-01 16:03:44 +08:00
e369b33265
fix chat mask bug
2025-04-01 13:44:55 +08:00
258507ff89
delete __pycache__
2025-04-01 11:51:54 +08:00
844e79148c
update generate args
2025-02-15 23:56:09 +08:00
19b388cd87
update generate args
2025-02-15 23:55:10 +08:00
5b65bc767e
update cis init
2025-02-15 20:26:34 +08:00
58e3af0359
add minimind2
2025-02-09 23:49:47 +08:00
3ff66f7221
update model
2024-10-20 15:13:58 +08:00
772834148e
update readme
2024-10-08 23:40:29 +08:00
a87f628400
update model (fix loss bug)
2024-09-29 16:58:48 +08:00
75753ea765
Update data preprocessing methods
2024-09-27 17:19:03 +08:00
a8ae342775
Update data preprocessing methods
2024-09-27 16:19:30 +08:00
6759da45c1
update model mask
2024-09-21 20:00:25 +08:00
02297df3c1
Efficient implementation of Inference KV cache
2024-09-21 00:01:05 +08:00
9093519c37
Updated some explanations
2024-09-20 17:07:51 +08:00
ee218402cd
update some explain of the code
2024-09-20 17:04:16 +08:00
2dceaf4a92
添加注释，方便学习者快速理解
2024-09-18 21:53:39 +08:00
61cb61a46a
update minimind-v1-moe
2024-09-17 11:33:31 +08:00
8c18b324d0
update model
2024-09-16 16:59:52 +08:00
e4ad822c40
update model
2024-09-16 15:29:57 +08:00
16928c1231
update some config
2024-09-15 15:12:47 +08:00
aa5d70321f
update config
2024-09-15 15:09:21 +08:00
f3f1cc5fac
update config
2024-09-15 15:08:04 +08:00
3068e5efcc
update model/dataset.py
2024-09-14 16:09:42 +08:00
ecf6d44133
update model/dataset.py
2024-09-14 14:05:41 +08:00
8be42693f6
MiniMind first open source
2024-08-28 16:41:44 +08:00
wanghao
jingyaogong
gongjy
Ben