Stop and Continue response like llmstudio #8969
Unanswered
electro199
asked this question in
Q&A
Replies: 2 comments
-
|
the same question. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
In llama-server, to stop a request, just close the connection. To continue it, just resent the whole prompt (so it doesn't work with chat endpoint, eh) with the partially generated message you saved |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In LLM studio there is option to stop generation edit the response and then continue so LLM use the provided text/token for completion. How to replicate that in the llama-server ?
Beta Was this translation helpful? Give feedback.
All reactions