Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问如果我想使用您提供的docker镜像运行别的模型,需要在哪修改template。 #75

Open
Halflifefa opened this issue Mar 25, 2024 · 0 comments

Comments

@Halflifefa
Copy link

因为不同模型的prompt template不一样。
按照目前的指令启动,不修改其他设置,会直接使用CodeShell的prompt template,也不会去读取
tokenizer_config.json下的chat_template,请问有办法修改吗?目前是参考的这个docker启动。

docker run --gpus 'all' --shm-size 1g -p 9090:80 -v $HOME/models:/data
--env LOG_LEVEL="info,text_generation_router=debug"
ghcr.nju.edu.cn/huggingface/text-generation-inference:1.0.3
--model-id /data/CodeShell-7B-Chat --num-shard 1
--max-total-tokens 5000 --max-input-length 4096
--max-stop-sequences 12 --trust-remote-code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant