Celery: When should you choose Redis as a message broker over RabbitMQ?

PythonDjangoRedisRabbitmqCelery

Python Problem Overview


My rough understanding is that Redis is better if you need the in-memory key-value store feature, however I am not sure how that has anything to do with distributing tasks?

Does that mean we should use Redis as a message broker IF we are already using it for something else?

Python Solutions


Solution 1 - Python

I've used both recently (2017-2018), and they are both super stable with Celery 4. So your choice can be based on the details of your hosting setup.

  • If you must use Celery version 2 or version 3, go with RabbitMQ. Otherwise...
  • If you are using Redis for any other reason, go with Redis
  • If you are hosting at AWS, go with Redis so that you can use a managed Redis as service
  • If you hate complicated installs, go with Redis
  • If you already have RabbitMQ installed, stay with RabbitMQ

In the past, I would have recommended RabbitMQ because it was more stable and easier to setup with Celery than Redis, but I don't believe that's true any more.


Update 2019

AWS now has a managed service that is equivalent to RabbitMQ called Amazon MQ, which could reduce the headache of running this as a service in production. Please comment below if you have any experience with this and celery.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionPigView Question on Stackoverflow
Solution 1 - PythonMark ChackerianView Answer on Stackoverflow