Skip to content

TeleChat1B模型在Huggingface的示例推理代码有错误 #82

@Betsyxing

Description

@Betsyxing

我下载Huggingface开源的TeleChat-1B模型和模型介绍中的推理示例代码,结果输出与问题无关,换个问题输出就乱码,麻烦看下啦,谢谢
代码:

import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
os.environ["CUDA_VISIBLE_DEVICES"] = '0'
tokenizer = AutoTokenizer.from_pretrained('*/models/TeleChat-1B', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('*/models/TeleChat-1B', trust_remote_code=True, device_map="auto", torch_dtype=torch.float16)
question="<_user>生抽与老抽的区别?<_bot>"
context_ids = tokenizer(question, return_tensors="pt")
output = model.generate(context_ids["input_ids"].to(0), do_sample=False, max_length=1024)
print(output)
answer = tokenizer.decode(output[0].tolist()).split('<_bot>')[-1]
print(answer)

结果如下图:

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions