If it won't be simple, it simply won't be. [Hire me, source code] by Miki Tebeka, CEO, 353Solutions

Tuesday, February 03, 2015

Logging from Celery to logstash and a structured log (JSON)

I love Celery, and we''re using it at one of my customers. One thing we wanted to have is centralized logging, since you can have multiple workers on multiple machines. We looked at several solutions and the winner came out to be logstash + kibana (AKA ELK stack).

Here''s some code to log to logstash with Celery current task information (if available) and also to a structured log (every line is a JSON object) for backup in case of network issues.

No comments:

Blog Archive