docs: add Mac Apple Silicon fine-tuning notes (#187) - #227
Open
Yu-0312 wants to merge 1 commit into
Open
Conversation
审查者指南（小型 PR 中折叠显示）审查者指南为 Mac Apple Silicon 用户添加平台特定的微调说明到英文和中文 README 中，涵盖 MPS 的使用、FlashAttention 的限制以及与内存相关的配置建议。 文件级更改
针对关联 Issue 的评估
提示和命令与 Sourcery 交互
自定义使用体验访问你的控制面板以：
获取帮助Original review guide in EnglishReviewer's guide (collapsed on small PRs)Reviewer's GuideAdds platform-specific fine-tuning notes for Mac Apple Silicon users to both the English and Chinese READMEs, covering MPS usage, FlashAttention limitations, and memory-related configuration guidance. File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
你好——我发现了 2 个问题
AI Agent 提示词
请处理此次代码审查中的评论：
## 个别评论
### 评论 1
<location path="README.md" line_range="152" />
<code_context>
+> [!NOTE]
+> **Mac Device (Apple Silicon) Fine-tuning Notes**:
+> - LlamaFactory supports training using Mac's MPS (Metal Performance Shaders). No additional configuration is needed, PyTorch will automatically recognize the MPS device.
+> - `flash-attn` **cannot be installed** on Mac (FlashAttention only supports NVIDIA GPUs), so make sure not to configure options dependent on FlashAttention in `settings.jsonc` (LlamaFactory will automatically fall back to standard attention or SDPA by default).
+> - Limited by Mac's unified memory, it is recommended to appropriately reduce `per_device_train_batch_size` and `cutoff_len` based on your machine's memory size.
+
</code_context>
<issue_to_address>
**问题（更广泛的影响）：** Mac 说明要求避免使用 FlashAttention，但文档中的设置步骤会复制一个模板，而该模板中的 `train_sft_args.flash_attn` 默认值为 `fa2`，训练命令也会将此值传递给 LlamaFactory。因此，用户如果按照设置步骤操作，却没有手动编辑生成的设置，就可能尝试使用不可用的软件包，而不是获得所承诺的标准注意力机制回退。
**触发条件：** Mac 用户复制 `examples/tg.template.jsonc` 或其他提供的模板，并直接运行文档中未修改的训练命令时。
**建议修复：** 告知 Mac 用户移除 `flash_attn`，或将其设置为受支持的非 FlashAttention 模式，并确保模板或设置说明体现这一要求；同时，将前面可选的安装命令明确限定为非 Mac 环境使用。
</issue_to_address>
### 评论 2
<location path="README_zh.md" line_range="150" />
<code_context>
+> **Mac 设备（Apple Silicon）微调注意事项**：
+> - LlamaFactory 支持使用 Mac 的 MPS（Metal Performance Shaders）进行训练，无需额外配置，PyTorch 会自动识别 MPS 设备。
+> - Mac 上**无法安装** `flash-attn`（FlashAttention 仅支持 NVIDIA GPU），因此请确保在 `settings.jsonc` 中不要配置依赖于 FlashAttention 的选项（LlamaFactory 默认会自动回退到普通注意力机制或 SDPA）。
+> - 受限于 Mac 的统一内存，建议根据机型显存大小适当调小 `per_device_train_batch_size` 和 `cutoff_len`。
+
### 多卡训练
</code_context>
<issue_to_address>
**问题：** 中文说明建议 Apple Silicon 用户根据 `显存大小`（VRAM）调整设置，但 Apple Silicon 使用的是统一系统内存，而不是独立的显存。这会让用户参考错误的资源指标，并可能导致他们选择超出可用统一内存的批次大小和序列长度。
**触发条件：** 中文 Mac 用户根据设备报告的 GPU/VRAM 数值应用内存调整建议时。
**建议修复：** 将 `显存大小` 替换为 `统一内存大小`，并明确说明相关限制是进程可用的系统总内存，包括其他应用占用的内存。
```suggestion
> - 受限于 Mac 的统一内存，建议根据机型统一内存大小（需考虑其他应用占用）适当调小 `per_device_train_batch_size` 和 `cutoff_len`。
```
</issue_to_address>帮助我变得更有用！请对每条评论点击 👍 或 👎，我会利用反馈来改进审查结果。
Original comment in English
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="README.md" line_range="152" />
<code_context>
+> [!NOTE]
+> **Mac Device (Apple Silicon) Fine-tuning Notes**:
+> - LlamaFactory supports training using Mac's MPS (Metal Performance Shaders). No additional configuration is needed, PyTorch will automatically recognize the MPS device.
+> - `flash-attn` **cannot be installed** on Mac (FlashAttention only supports NVIDIA GPUs), so make sure not to configure options dependent on FlashAttention in `settings.jsonc` (LlamaFactory will automatically fall back to standard attention or SDPA by default).
+> - Limited by Mac's unified memory, it is recommended to appropriately reduce `per_device_train_batch_size` and `cutoff_len` based on your machine's memory size.
+
</code_context>
<issue_to_address>
**issue (broader_impact):** The Mac instructions say to avoid FlashAttention, but the documented setup copies a template whose `train_sft_args.flash_attn` defaults to `fa2`, and the training command passes that value to LlamaFactory. A user who follows the setup without manually editing the generated settings can therefore attempt to use the unavailable package instead of receiving the promised standard-attention fallback.
**Triggers:** When a Mac user copies `examples/tg.template.jsonc` or another provided template and runs the documented training command unchanged.
**Suggested fix:** Tell Mac users to remove `flash_attn` or set it to the supported non-FlashAttention mode, and make the templates or setup instructions reflect that requirement; also qualify the earlier optional installation command as non-Mac-only.
</issue_to_address>
### Comment 2
<location path="README_zh.md" line_range="150" />
<code_context>
+> **Mac 设备（Apple Silicon）微调注意事项**：
+> - LlamaFactory 支持使用 Mac 的 MPS（Metal Performance Shaders）进行训练，无需额外配置，PyTorch 会自动识别 MPS 设备。
+> - Mac 上**无法安装** `flash-attn`（FlashAttention 仅支持 NVIDIA GPU），因此请确保在 `settings.jsonc` 中不要配置依赖于 FlashAttention 的选项（LlamaFactory 默认会自动回退到普通注意力机制或 SDPA）。
+> - 受限于 Mac 的统一内存，建议根据机型显存大小适当调小 `per_device_train_batch_size` 和 `cutoff_len`。
+
### 多卡训练
</code_context>
<issue_to_address>
**issue:** The Chinese note tells Apple Silicon users to size the settings according to `显存大小` (VRAM), but Apple Silicon exposes unified system memory rather than discrete VRAM. This gives users the wrong resource to measure and can lead them to choose batch and sequence lengths that exceed the available unified memory.
**Triggers:** When a Chinese-language Mac user applies the memory-tuning recommendation using the machine's reported GPU/VRAM value.
**Suggested fix:** Replace `显存大小` with `统一内存大小` and make clear that the total system memory available to the process, including other applications, is the relevant limit.
```suggestion
> - 受限于 Mac 的统一内存，建议根据机型统一内存大小（需考虑其他应用占用）适当调小 `per_device_train_batch_size` 和 `cutoff_len`。
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Yu-0312
force-pushed
the
docs/issue-187-mac-tuning
branch
from
August 22, 2026 07:32
202b3cd to
82cd27f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #187. Added notes for Mac (Apple Silicon) users regarding MPS support, flash-attention limitations, and memory tuning recommendations in both English and Chinese READMEs.
Sourcery 摘要
文档：
Original summary in English
Sourcery 摘要
文档：
Original summary in English
Summary by Sourcery
Documentation: