We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
因为不同模型的prompt template不一样。 按照目前的指令启动,不修改其他设置,会直接使用CodeShell的prompt template,也不会去读取 tokenizer_config.json下的chat_template,请问有办法修改吗?目前是参考的这个docker启动。
docker run --gpus 'all' --shm-size 1g -p 9090:80 -v $HOME/models:/data --env LOG_LEVEL="info,text_generation_router=debug" ghcr.nju.edu.cn/huggingface/text-generation-inference:1.0.3 --model-id /data/CodeShell-7B-Chat --num-shard 1 --max-total-tokens 5000 --max-input-length 4096 --max-stop-sequences 12 --trust-remote-code
The text was updated successfully, but these errors were encountered:
No branches or pull requests
因为不同模型的prompt template不一样。
按照目前的指令启动,不修改其他设置,会直接使用CodeShell的prompt template,也不会去读取
tokenizer_config.json下的chat_template,请问有办法修改吗?目前是参考的这个docker启动。
docker run --gpus 'all' --shm-size 1g -p 9090:80 -v $HOME/models:/data
--env LOG_LEVEL="info,text_generation_router=debug"
ghcr.nju.edu.cn/huggingface/text-generation-inference:1.0.3
--model-id /data/CodeShell-7B-Chat --num-shard 1
--max-total-tokens 5000 --max-input-length 4096
--max-stop-sequences 12 --trust-remote-code
The text was updated successfully, but these errors were encountered: