Syncing Qdrant with another Qdrant

[cemremengu]

I want to run 2 standalone qdrant instance that can sync each other (i.e. cross cluster replication). For example when I delete entity from qdrant instance1 that instance will delete from instance2 (update, insert, delete). Is there any way that I can do that ?

[timvisee]

Yes! Please see our documentation on this here: https://qdrant.tech/documentation/guides/distributed_deployment/

More specifically, see this section describing: 1. enabling cluster mode. 2. starting the first node. 3. bootstrapping other nodes.

Once you've set up a cluster you can send operations to any node. They'll be synced across the cluster.

Note that if you want to prevent downtime if one of the nodes is down, you should ideally have at least 3 nodes. That is because there should be a majority if nodes, which would not be possible with 1 alive + 1 dead node. In that case, also make sure you have at least a replication factor of 2.

[timvisee]

Thanks for elaborating. Since you were talking about instances, I was thinking about nodes in a cluster.

At this point we don't have any knobs for setting up multi-region deployments, pairs of clusters or availability zones.

Alternatively you could set up a separate second cluster and continuously sync data to it by using snapshots. It will not be real time though and will require significant manual effort. It is also possible to form a cluster with nodes across different zones though that'll likely introduce noticeable latency. Depending on your needs either of those may be sufficient (for now).

I would like to note that it is on our roadmap.

It's likely best to just rely on a single cluster now, and deploy multiple zones later once we support it.

[cemremengu]

Great! Thanks again for taking your time to reply and your work on Qdrant.

[Prakash2403]

It is also possible to form a cluster with nodes across different zones though that'll likely introduce noticeable latency.

Hi @timvisee, how do I redirect a request to a specific node in a cluster? If I am using docker, can I do IP-Address-of-Node:6333 (Assuming I have done relevant port mappings)?

My problem is that I have users worldwide but my server is in the USA. Due to this, there's a noticeable latency for users who are located far away from my servers (say India, Australia, etc). Hence I want to setup one server in each region, and redirect the incoming requests to the nearest server.

[timvisee]

If I am using docker, can I do IP-Address-of-Node:6333 (Assuming I have done relevant port mappings)?

Yes. You're responsible yourself for spreading requests to nodes. You can simply spread to all nodes, as all necessary internal routings are made internally.

Note that we do all this for you in Qdrant Cloud.

Hence I want to setup one server in each region, and redirect the incoming requests to the nearest server.

This currently is not possible. Unless you set up isolated clusters. That may very well be good enough for your current use case.

[Prakash2403]

Thanks for the prompt response! I might go ahead with isolated clusters. I am happy to switch to Qdrant cloud once cross-cluster replication is possible. Do keep us updated!

[redoh]

Hi @timvisee, thanks for the detailed answers in this thread — very helpful!

I have a follow-up question: What if the Qdrant instances are running in an air-gapped or isolated network environment with no external/public internet access? In that scenario, nodes can't directly discover or communicate with each other across different networks.

Is there a recommended approach for syncing data between two such isolated Qdrant instances? For example:

• Could the snapshot API be used in an offline/manual transfer workflow (export snapshot → move file to the other network → restore)?
• Is there any support for a push-based replication model where one instance pushes updates to another through a controlled channel (e.g., a bastion host or a unidirectional gateway)?
• Or would we need to build a custom sync layer on top of the Qdrant API (scroll + upsert) to handle this?

Any guidance on the best practice for keeping two Qdrant instances in sync when they are network-isolated from each other would be greatly appreciated. Thanks!

[timvisee]

I have a follow-up question: What if the Qdrant instances are running in an air-gapped or isolated network environment with no external/public internet access? In that scenario, nodes can't directly discover or communicate with each other across different networks.

You cannot deploy a Qdrant cluster with air gaps between machines. Note that I'm speaking about a single Qdrant cluster here. All peers that participate in the cluster must be able to reach all other participating peers at all times.

What you can do is deploy two (or more) separate clusters. You can have air gaps between them.

Could the snapshot API be used in an offline/manual transfer workflow (export snapshot → move file to the other network → restore)?

Yes, you can definitely do this. This is what I would suggest to do.

Is there any support for a push-based replication model where one instance pushes updates to another through a controlled channel (e.g., a bastion host or a unidirectional gateway)?

There is not. But you could set up some sort of a 3rd party queue that will propagate changes at a later time. Operations are idempotent and it should be fine to propagate them to the cluster at a later time. You could use Kafka for this or even build your own system for this. Note that we cannot offer real support for this scenario.

Or would we need to build a custom sync layer on top of the Qdrant API (scroll + upsert) to handle this?

That would also work. Snapshots are easier, though they include a full copy of the data.