598 lines
20 KiB
Plaintext
598 lines
20 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fd56e0ed-d6e3-4fdd-bed2-2a0c32fb61f9",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 6-LoRA\n",
|
||
"\n",
|
||
"大语言模型的低秩适应(LoRA, Low-Rank Adaptation of Large Language Models) 是一项大模型参数高效微调技术，其可以显著减少可训练参数的数量.\n",
|
||
"\n",
|
||
"由于大模型参数量较大，直接进行全参微调需要消耗大量硬件资源，LoRA 的工作原理是将少量的新权重插入倒模型中，并且仅训练这些权重，这使得使用 LoRA 进行训练的速度更快、内存效率更高，并生成更小的模型权重.\n",
|
||
"\n",
|
||
"具体来说，LoRA 它冻结了预训练模型 W 的权重，并注入可训练的秩分解矩阵 A 与 B，在微调时，只训练降维矩阵 A 和 升维矩阵 B，微调结束后，将 AB 与 W 进行叠加.\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"其中，用随机高斯分布进行初始化 A，用 0 矩阵初始化 B，从而保证训练开始时旁路矩阵为 0 矩阵.\n",
|
||
"\n",
|
||
"具体来看，假设模型经过预训练主干的输出为 $W_0 x$，在 LoRA 微调阶段，我们可以用如下形式对输出进行表示.\n",
|
||
"\n",
|
||
"$$h=W_0x + \\Delta Wx = W_0x + BA x=(W_0 + BA)x$$\n",
|
||
"\n",
|
||
"其中, $B \\in \\mathbb{R}^{d \\times r},A \\in \\mathbb{R}^{r\\times k}$，r 为 LoRA 低秩矩阵的维数，$r << min(d, k)$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "dad11c0c-043d-44cb-a9a7-4c36c2e909da",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import torch\n",
|
||
"from torch import optim, nn"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3031399b-9342-454f-86df-9a96b8c6ff50",
|
||
"metadata": {},
|
||
"source": [
|
||
"## LoRA Adapter\n",
|
||
"\n",
|
||
"简单的来说，LoRA 矩阵就是具有一个隐藏层的全连接网络，其挂接在主干网络边侧进行参数更新，我们来看看 MiniMind 模型是如何在主干网络外部定义 LoRA 网络结构的."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "2f9bc623-f13d-4d67-bf2e-dce0784b43f8",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class LoRA(nn.Module):\n",
|
||
" def __init__(self, in_features, out_features, rank):\n",
|
||
" super().__init__()\n",
|
||
" self.rank = rank # LoRA 秩 控制低秩矩阵大小\n",
|
||
" self.A = nn.Linear(in_features, rank, bias=False)\n",
|
||
" self.B = nn.Linear(rank, out_features, bias=False)\n",
|
||
" # 矩阵 A 高斯分布初始化\n",
|
||
" self.A.weight.data.normal_(mean=0.0, std=0.02)\n",
|
||
" # 矩阵 B 全零初始化\n",
|
||
" self.B.weight.data.zero_()\n",
|
||
"\n",
|
||
" def forward(self, x):\n",
|
||
" return self.B(self.A(x))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4a3557bd-d34d-4a15-b789-22afea60a87d",
|
||
"metadata": {},
|
||
"source": [
|
||
"可以看到，LoRA 的网络结构非常简单直观，我们接下来定义一个方法，将 LoRA 网络应用到 MiniMind 模型的特定线性层."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "4d52e346-af62-40e2-a5b0-5cc0cf8ae6e8",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def apply_lora(model, rank=16):\n",
|
||
" \"\"\"将 LoRA 模块与目标模块进行绑定\"\"\"\n",
|
||
" for name, module in model.named_modules():\n",
|
||
" if isinstance(module, nn.Linear) and module.weight.shape[0] == module.weight.shape[1]:\n",
|
||
" lora = LoRA(module.weight.shape[0], module.weight.shape[1], rank=rank).to(model.device)\n",
|
||
" setattr(module, 'lora', lora) # 显式添加 LoRA 模块\n",
|
||
" original_forward = module.forward\n",
|
||
"\n",
|
||
" # 修改目标模块的 forward 函数\n",
|
||
" def forward_with_lora(x, layer1=original_forward, layer2=lora):\n",
|
||
" return layer1(x) + layer2(x)\n",
|
||
" \n",
|
||
" module.forward = forward_with_lora\n",
|
||
" # 打印 LoRA 绑定的模块名称\n",
|
||
" print(f'apply lora on module: {name}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "753a54c6-a442-4ce5-be89-3bc601bc8d26",
|
||
"metadata": {},
|
||
"source": [
|
||
"我们可以声明一个小模型，对于 LoRA 的绑定进行测试."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "f830221b-891e-4e27-b401-20b6fe5d3c59",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class TestModel(nn.Module):\n",
|
||
" def __init__(self):\n",
|
||
" super().__init__()\n",
|
||
" self.linear1 = nn.Linear(64, 512)\n",
|
||
" self.linear2 = nn.Linear(512, 512)\n",
|
||
" self.linear3 = nn.Linear(512, 64)\n",
|
||
"\n",
|
||
" @property\n",
|
||
" def device(self):\n",
|
||
" return next(self.parameters()).device\n",
|
||
" \n",
|
||
" def forward(self, x):\n",
|
||
" out = self.linear3(self.linear2(self.linear1))\n",
|
||
" return out"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1a8edbc2-f36c-4127-be3b-3c598872621a",
|
||
"metadata": {},
|
||
"source": [
|
||
"按照 apply_lora 的函数逻辑，LoRA 模块会应用在主干网络中满足 input_feature == output_feature 的模块上."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "eb24e24f-da09-4cd7-b4ce-7fe482194f75",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"apply lora on module: linear2\n",
|
||
"TestModel(\n",
|
||
" (linear1): Linear(in_features=64, out_features=512, bias=True)\n",
|
||
" (linear2): Linear(\n",
|
||
" in_features=512, out_features=512, bias=True\n",
|
||
" (lora): LoRA(\n",
|
||
" (A): Linear(in_features=512, out_features=16, bias=False)\n",
|
||
" (B): Linear(in_features=16, out_features=512, bias=False)\n",
|
||
" )\n",
|
||
" )\n",
|
||
" (linear3): Linear(in_features=512, out_features=64, bias=True)\n",
|
||
")\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"test_model = TestModel()\n",
|
||
"apply_lora(test_model)\n",
|
||
"print(test_model)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "f882e6c0-65a6-47b2-9ee6-4a3fca988636",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"del test_model"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9771475e-0c60-42db-b0cd-7e193de23f7c",
|
||
"metadata": {},
|
||
"source": [
|
||
"完成了 LoRA 模块在主干网络特定模块的绑定后，我们便可以冻结主干网络参数进行微调了，不过，考虑到主干网络权重在训练过程中并不会做任何参数更新，我们可以只保存 LoRA 模块的参数来节省内存，下面给出加载/保存 LoRA 权重的方法."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "95a453d7-2011-4575-9a51-36d91876874f",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def load_lora(model, path):\n",
|
||
" state_dict = torch.load(path, map_location=model.device)\n",
|
||
" for name, module in model.named_modules():\n",
|
||
" if hasattr(module, 'lora'):\n",
|
||
" lora_state = {k.replace(f'{name}.lora.', ''): v for k, v in state_dict.items() if f'{name}.lora.' in k}\n",
|
||
" module.lora.load_state_dict(lora_state)\n",
|
||
"\n",
|
||
"\n",
|
||
"def save_lora(model, path):\n",
|
||
" state_dict = {}\n",
|
||
" for name, module in model.named_modules():\n",
|
||
" if hasattr(module, 'lora'):\n",
|
||
" lora_state = {f'{name}.lora.{k}': v for k, v in module.lora.state_dict().items()}\n",
|
||
" state_dict.update(lora_state)\n",
|
||
" torch.save(state_dict, path)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "854b8afb-39b1-4175-8e69-4a9d5b25a3e5",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Fine-Tuning MiniMind with LoRA"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "10baf158-6653-49e3-9ba3-22b2e341730f",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"import platform\n",
|
||
"import argparse\n",
|
||
"import random\n",
|
||
"import time\n",
|
||
"import math\n",
|
||
"import warnings\n",
|
||
"import torch.distributed as dist\n",
|
||
"from contextlib import nullcontext\n",
|
||
"from torch.utils.data import DataLoader, DistributedSampler\n",
|
||
"from transformers import AutoTokenizer, AutoModelForCausalLM\n",
|
||
"from demo.model import MiniMindLM\n",
|
||
"from demo.LMConfig import LMConfig\n",
|
||
"from demo.dataset import SFTDataset\n",
|
||
"\n",
|
||
"warnings.filterwarnings('ignore')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f0b93272-80b3-45ce-9dec-4583fca8f87a",
|
||
"metadata": {},
|
||
"source": [
|
||
"### 可选参数设置\n",
|
||
"\n",
|
||
"首先，查看训练的可选参数，这些参数在实际使用时通过解析命令行进行导入，我们用 class 进行包装."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "bb9d4bac-8a30-4ca9-b293-94df3f880826",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class args:\n",
|
||
" # out_dir: str = \"out\" # pytorch 格式权重文件保存位置 我们只展示训练过程 所以不使用\n",
|
||
" epochs: int = 1 # 训练轮数\n",
|
||
" batch_size: int = 2 # pretrain 数据集仅两个样本，设置 batch 为 2\n",
|
||
" learning_rate: float = 5e-4 # 学习率\n",
|
||
" device: str = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
|
||
" dtype: str = 'bfloat16' # 16 bit 浮点数：8 bit 指数 + 7 bit 尾数\n",
|
||
" # use_wandb: bool = False # 是否使用 wandb 我们不使用\n",
|
||
" wandb_project: str = 'MiniMind-Notebook'\n",
|
||
" num_workers: int = 1 # 工作进程数\n",
|
||
" # ddp：bool = False # 单机多卡\n",
|
||
" accumulation_steps: int = 1 # 梯度累积步数\n",
|
||
" grad_clip: float = 1.0 # 梯度剪裁\n",
|
||
" warmup_iters: int = 0 # 学习率热启动\n",
|
||
" log_interval: int = 1 # 每一步打印日志 仅用于观察\n",
|
||
" # save_interval: int = 100 # checkpoint 保存点 我们不使用\n",
|
||
" local_rank: int = 1 # device 设备号\n",
|
||
" dim: int = 512 # 词嵌入维度 模型超参数\n",
|
||
" n_layers: int = 2 # MiniMind Block 数量 模型超参数\n",
|
||
" max_seq_len: int = 512 # 序列长度阈值\n",
|
||
" use_moe: bool = False # 是否启用混合专家\n",
|
||
" data_path: str = './demo/lora_data.jsonl' # 数据集路径\n",
|
||
" lora_name: str = 'lora_identity' # 根据任务保存成lora_(英文/医学/心理...)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "3b496134-ac9a-4146-a236-fd8867f45bc6",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"查看工作设备 cuda\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(f'查看工作设备 {args.device}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "32b956e6-e81b-49d9-bbf7-b78fbbc7eff5",
|
||
"metadata": {},
|
||
"source": [
|
||
"接下来，我们对分词器、MiniMindLM 和数据迭代器执行初始化."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "2fb7c880-b86e-4329-aeef-2827837425a3",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def init_model(lm_config):\n",
|
||
" tokenizer = AutoTokenizer.from_pretrained('../model/minimind_tokenizer')\n",
|
||
" model = MiniMindLM(lm_config)\n",
|
||
" moe_path = '_moe' if lm_config.use_moe else ''\n",
|
||
" # 热启动\n",
|
||
" # ckp = f'./out/rlhf_{lm_config.dim}{moe_path}.pth'\n",
|
||
" # state_dict = torch.load(ckp, map_location=args.device)\n",
|
||
" # model.load_state_dict(state_dict, strict=False)\n",
|
||
" return model.to(args.device), tokenizer"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "7f0c74d0-a191-40c1-b922-104f241aa879",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"apply lora on module: layers.0.attention.wq\n",
|
||
"apply lora on module: layers.0.attention.wo\n",
|
||
"apply lora on module: layers.1.attention.wq\n",
|
||
"apply lora on module: layers.1.attention.wo\n",
|
||
"模型位于设备：cuda:0, 词表长度：6400, DataLoader：<torch.utils.data.dataloader.DataLoader object at 0x0000018FB27CA050>\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"lm_config = LMConfig(dim=args.dim, n_layers=args.n_layers, max_seq_len=args.max_seq_len, use_moe=args.use_moe)\n",
|
||
"model, tokenizer = init_model(lm_config)\n",
|
||
"apply_lora(model)\n",
|
||
"\n",
|
||
"# 由于 MiniMind 用于 LoRA 微调的数据集和 SFT 数据集格式相同，可以用 SFT 数据集进行加载\n",
|
||
"train_ds = SFTDataset(args.data_path, tokenizer, max_length=lm_config.max_seq_len)\n",
|
||
"\n",
|
||
"train_loader = DataLoader(\n",
|
||
" train_ds,\n",
|
||
" batch_size=args.batch_size,\n",
|
||
" pin_memory=True,\n",
|
||
" drop_last=False,\n",
|
||
" shuffle=False,\n",
|
||
" num_workers=args.num_workers,\n",
|
||
")\n",
|
||
"\n",
|
||
"print(f'模型位于设备：{model.device}, 词表长度：{tokenizer.vocab_size}, DataLoader：{train_loader}')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c35bb3ae-0b31-42ff-88d1-04d7e945bd31",
|
||
"metadata": {},
|
||
"source": [
|
||
"可以看到，LoRA 模块挂接在 Attention Block 的 Query 与 Output 线性层上，下面查看 LoRA 微调下可学习参数的占比："
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "766ef2d5-1f24-4efb-9eb9-f969a57a85b0",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"LLM 总参数量: 8980992\n",
|
||
"LoRA 参数量: 65536\n",
|
||
"LoRA 参数占比: 0.73%\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"total_params = sum(p.numel() for p in model.parameters()) # 总参数数量\n",
|
||
"lora_params_count = sum(p.numel() for name, p in model.named_parameters() if 'lora' in name) # LoRA 参数数量\n",
|
||
"print(f\"LLM 总参数量: {total_params}\")\n",
|
||
"print(f\"LoRA 参数量: {lora_params_count}\")\n",
|
||
"print(f\"LoRA 参数占比: {lora_params_count / total_params * 100:.2f}%\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8a487807-65da-4e19-ac2f-fce7077494c0",
|
||
"metadata": {},
|
||
"source": [
|
||
"接下来，冻结 MiniMindLM 主干网络的参数，做好 LoRA 微调准备."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "db99fd52-4f07-4278-939f-47b91aeeedbe",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"for name, param in model.named_parameters():\n",
|
||
" if 'lora' not in name:\n",
|
||
" param.requires_grad = False\n",
|
||
"lora_params = [] # 收集 LoRA 模块的可学习参数, 提供给优化器\n",
|
||
"for name, param in model.named_parameters():\n",
|
||
" if 'lora' in name:\n",
|
||
" lora_params.append(param)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6f865b22-35a7-4c2c-85a1-a13d1ada12a7",
|
||
"metadata": {},
|
||
"source": [
|
||
"### 启动训练\n",
|
||
"\n",
|
||
"接下来，我们定义 MiniMind LoRA 微调所使用的优化器，损失函数和学习率调度，并进行一轮简单的训练."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "2b57beaf-c04a-4b88-9617-7142054f2927",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# 学习率调度方面 采用余弦退火学习率\n",
|
||
"def get_lr(current_step, total_steps, lr):\n",
|
||
" return lr / 10 + 0.5 * lr * (1 + math.cos(math.pi * current_step / total_steps))\n",
|
||
"\n",
|
||
"# 优化器方面 选择 AdamW 优化器 并在混精度场景下创建 scaler 进行梯度缩放避免数值下溢\n",
|
||
"scaler = torch.cuda.amp.GradScaler(enabled=(args.dtype in ['float16', 'bfloat16']))\n",
|
||
"# 只训练 LoRA 模块参数\n",
|
||
"optimizer = optim.AdamW(lora_params, lr=args.learning_rate)\n",
|
||
"\n",
|
||
"device_type = \"cuda\" if \"cuda\" in args.device else \"cpu\"\n",
|
||
"ctx = nullcontext() if device_type == \"cpu\" else torch.cuda.amp.autocast() # 在 cuda 上启动混精度训练，否则空白上下文"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8037d2b2-1fa3-4e19-9ad5-9674ff866a36",
|
||
"metadata": {},
|
||
"source": [
|
||
"接下来，我们来看训练函数."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"id": "8f50d42d-da6d-4f8b-aa35-853b3900f8b5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# 代码和 SFT 几乎一致\n",
|
||
"def train_epoch(epoch):\n",
|
||
" loss_fct = nn.CrossEntropyLoss(reduction='none')\n",
|
||
" start_time = time.time()\n",
|
||
" for step, (X, Y, loss_mask) in enumerate(train_loader):\n",
|
||
" X = X.to(args.device)\n",
|
||
" Y = Y.to(args.device)\n",
|
||
" loss_mask = loss_mask.to(args.device)\n",
|
||
" lr = get_lr(epoch * iter_per_epoch + step, args.epochs * iter_per_epoch, args.learning_rate)\n",
|
||
" for param_group in optimizer.param_groups:\n",
|
||
" param_group['lr'] = lr\n",
|
||
"\n",
|
||
" with ctx:\n",
|
||
" res = model(X)\n",
|
||
" loss = loss_fct(\n",
|
||
" res.logits.view(-1, res.logits.size(-1)),\n",
|
||
" Y.view(-1)\n",
|
||
" ).view(Y.size())\n",
|
||
" loss = (loss * loss_mask).sum() / loss_mask.sum()\n",
|
||
" loss += res.aux_loss\n",
|
||
" loss = loss / args.accumulation_steps\n",
|
||
"\n",
|
||
" scaler.scale(loss).backward() # 梯度缩放\n",
|
||
"\n",
|
||
" if (step + 1) % args.accumulation_steps == 0:\n",
|
||
" scaler.unscale_(optimizer)\n",
|
||
" torch.nn.utils.clip_grad_norm_(lora_params, args.grad_clip) # 梯度剪裁\n",
|
||
"\n",
|
||
" scaler.step(optimizer)\n",
|
||
" scaler.update()\n",
|
||
"\n",
|
||
" optimizer.zero_grad(set_to_none=True)\n",
|
||
"\n",
|
||
" if step % args.log_interval == 0:\n",
|
||
" spend_time = time.time() - start_time\n",
|
||
" print(\n",
|
||
" 'Epoch:[{}/{}]({}/{}) loss:{:.3f} lr:{:.12f} epoch_Time:{}min:'.format(\n",
|
||
" epoch + 1,\n",
|
||
" args.epochs,\n",
|
||
" step,\n",
|
||
" iter_per_epoch,\n",
|
||
" loss.item(),\n",
|
||
" optimizer.param_groups[-1]['lr'],\n",
|
||
" spend_time / (step + 1) * iter_per_epoch // 60 - spend_time // 60))\n",
|
||
"\n",
|
||
" # 只保存 LoRA 模块的权重\n",
|
||
" # if (step + 1) % args.save_interval == 0 and (not ddp or dist.get_rank() == 0):\n",
|
||
" # model.eval()\n",
|
||
" # # 【区别1】只保存lora权重即可\n",
|
||
" # save_lora(model, f'{args.save_dir}/lora/{args.lora_name}_{lm_config.dim}.pth')\n",
|
||
" # model.train()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "618876fa-7f0f-43b3-9eed-384be6cd66be",
|
||
"metadata": {},
|
||
"source": [
|
||
"接下来，我们启动一个 Epoch 的训练进行观察."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "38584341-9ef6-4ebd-9024-0c44341de7c8",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch:[1/1](0/1) loss:8.894 lr:0.000550000000 epoch_Time:0.0min:\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"iter_per_epoch = len(train_loader)\n",
|
||
"for epoch in range(args.epochs):\n",
|
||
" train_epoch(epoch)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"id": "61150c72-39eb-4b90-b321-d4c7aa6c885a",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"del model"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dd5e9aa8-b5a5-4380-989d-e54948f33a1b",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 参考资料\n",
|
||
"\n",
|
||
"- [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)\n",
|
||
"- [HuggingFace LoRA](https://huggingface.co/docs/diffusers/en/training/lora)\n",
|
||
"- [LoRA 微调和低秩矩阵](https://www.cnblogs.com/ghj1976/p/18032882/lora-finetuning-he-di-zhi-ju-zhen)"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.10.14"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|