Skip to content

7B就行lora微调过程报错 #65

@lichengyang666

Description

@lichengyang666

训练7B的过程中,将原始的lora参数调整之后,训练到一半时报错
Exception: Current loss scale already at minimum - cannot decrease scale anymore. Exiting run. ,应该如何解决?

尝试将fp16修改为bf16就行尝试,然后又报错
File "/home/jupyter/.cache/huggingface/modules/transformers_modules/TeleChat-7B/modeling_telechat.py", line 185, in forward
assert all((i.dtype in [torch.float16, torch.bfloat16] for i in (q, k, v)))

期待您的回复~

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions