Seamless web search integration for LLM agents. Transforms natural language queries into structured web data, enabling AI models to access real-time information through a clean API.
-
Multi-Level Caching System
- Query-level caching with Redis using vector similarity search
- Page-level content caching with MongoDB for URL-based retrieval
-
Multiple Search Providers
- SearXNG - Privacy-focused metasearch engine
- DuckDuckGo - Privacy-respecting search engine
- Google Custom Search - Customizable web search
-
Flexible LLM Integration
- OpenAI-compatible API interface
- Configurable models for result summarization
- Token-aware content truncation
-
Config enviroment variables: Copy
.env.example
to.env
and configure the missing variables -
Build the Docker image:
docker build -t titan-sight:0.1.0 .
- Create required network:
docker network create titan_sight_net
- Start services:
docker-compose up -d
Search Endpoint
curl -X 'GET' \
'http://localhost:6969/v1/search?query=What%20is%20the%20weather%20like%20today%20in%20Hanoi%3F&provider=google&max_num_result=3&enable_cache=true' \
-H 'accept: application/json'
Response:
{
"query": "What is the weather like today in Hanoi?",
"results": [
{
"title": "Weather Forecast Hanoi",
"url": "https://weather.com/...",
"content": "Current conditions in Hanoi...",
"details": "Full page content...",
"answer": "Currently in Hanoi: 24°C, Clear skies..."
},
...
]
}
- FastAPI Backend: Handles API requests and orchestrates search operations
- Redis Cache: Stores query embeddings for fast similarity search
- MongoDB: Persists webpage content for frequently accessed URLs
- SearXNG: Self-hosted metasearch engine component
Contributions are welcome! Please feel free to submit a Pull Request.