Skip to content

Support concurrent translation #143

Description

@vitonsky

Currently we handle texts in one thread:

const free = await this.semafor.take();

That make sense, because free services have strong rate limits.

However it works very bad with LLM translators. They are error prone, they generate a lot of errors and wait for 3-5 retries.
We could run at least 10 threads in parallel and show the translations faster.

Let's add the optional method like getConcurrency(): number and in case it is larger than 1 we will run translation in multiple threads.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions