Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Finetune是否有针对训练集的衡量指标？ #917

Open
WalkerWorldPeace opened this issue Feb 21, 2025 · 1 comment
Open

Finetune是否有针对训练集的衡量指标？ #917

WalkerWorldPeace opened this issue Feb 21, 2025 · 1 comment

Comments

@WalkerWorldPeace
Copy link

您好，
请问我在SFT训练时，除了loss作为训练集的指标，能否有类似准确率之类的metric反应训练集的学习情况？因为测试benchmark往往和训练集联系的不是那么紧密，我该如何检查微调训练数据的准确率呢？
非常期待您的回答，将对我有所帮助，谢谢。
@lll2343
Copy link
Collaborator

lll2343 commented Feb 24, 2025

Hi,
可以选一些和你训练集相同分布的benchmark看看性能呢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lll2343 @WalkerWorldPeace