Broker memory explosion with loss of elastic search.

[jackpmastrangelo]

Hi I'm not sure this is a bug with leek exactly, but I'd like some help understanding the behaviour of my leek/celery combo.

Describe the bug
Our ElasticSearch cluster died, and suddenly our brokers' memory usage skyrocketed since the celeryev.* keys were growing indefinitely. Is this expected? Or are we misconfiguring our events/leek somehow?

To Reproduce

1. Have celery running on redis broker and broadasting events to leek.
2. Have ES go down.

Expected behavior
Celery events are TTL'd somehow to prevent runaway growth.

Additional context
All running on AWS.

Thank you in advance, Hamza. Leek is a great tool!