Celery

• Use DatabaseScheduler for scheduled tasks on AWS Fargate.
• A ResultBackend is used to store tasks results (typically Redis).
• retry : Retry a task, for example, if there's an exception, retry.
• max_retries: By default, 3. Avoid broken tasks keep retrying forever.
• acks_late: Retry a task if the worker crashes, disabled by default. Workers send and ack when a task is complete, not just when it's received (default).
• Be aware that for retry and acks_late, ideally, tasks must be idempotent.
• Don't wait for a task inside another task (that makes it synchronous!)
• Retry when possible, make sure tasks are atomic and idempotent. Use exponential retries.
• Default concurrency for Celery workers is CPU number.
• If you don’t care about the results of a task, be sure to set the ignore_result option, as storing results wastes time and resources.
• Workers shutdown should be accomplished using the TERM signal. When shutdown is initiated the worker will finish all currently executing tasks before it actually terminates. If these tasks are important, you should wait for it to finish before doing anything drastic, like sending the KILL signal.
• worker_max_tasks_per_child: With this option you can configure the maximum number of tasks a worker can execute before it’s replaced by a new process. This is useful if you have memory leaks you have no control over for example from closed source C extensions.
• The simplest way to do routing is to use the task_create_missing_queues setting (on by default). With this setting on, a named queue that’s not already defined in task_queues will be created automatically. This makes it easy to perform simple routing tasks.
• Flower recommended for monitoring.
• -Ofair setting. Interesting reading.
• Don't pass Database/ORM objects to tasks: You shouldn't pass Database objects (for instance your User model) to a background task because the serialized object might contain stale data. What you want to do is feed the task the User id and have the task ask the database for a fresh User object.
• Set CELERY_IGNORE_RESULT to False if you don't need results.
• Set timeouts for tasks:

# None of our tasks should take more than 5min CELERYD_TASK_SOFT_TIME_LIMIT = 5 * 60 # If soft limit didn't help, kill the task right after CELERYD_TASK_TIME_LIMIT = CELERYD_TASK_SOFT_TIME_LIMIT + 10