[BUG] Potential Performance Degradation with Large Document Sets

[T-Ezzoury]

Pre-check

• I have searched the existing issues and none cover this bug.

Description

When working with large datasets, there could be noticeable delays or performance drops during document ingestion and context retrieval

Steps to Reproduce

1- Prepare a dataset containing a large number of documents.
2- Ingest the dataset using the tool’s document ingestion process.
3- Observe the performance during the ingestion and context retrieval phases.

Expected Behavior

The system should efficiently handle large document sets with minimal delays.

Actual Behavior

Noticeable delays and performance degradation are observed when processing a large number of documents.

Environment

Windows 11, NVIDIA® H100, H200

Additional Information

No response

Version

No response

Setup Checklist

• Confirm that you have followed the installation instructions in the project’s documentation.
• Check that you are using the latest version of the project.
• Verify disk space availability for model storage and data processing.
• Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

• Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
• Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
• Ensure proper permissions are set for accessing GPU resources.
• Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)