/ zc.async alternatives?

Anyone using or considering alternatives to zc.async / for simple job-queue stuff? I'm contemplating doing something on this front (for media transcoding jobs), but I wanted to see what solutions folks have been using?

My goals:

  • Simple job queue, queue a context object and a callable that takes that context as an argument;

  • Don't care about scheduling, only avoiding race-conditions;

  • Want to be able to run something async as either long-running process or cron job (or maybe a runscript that will do either).

  • Light-weight, no ZODB writes except for logging job to queue, and committing final result.


Have you looked at how collective.documentviewer uses

He's looking for alternatives to! has a decent number of people using it. It's still new but more sane.

1 Like

although also based on, it's at least worth a look to see how Plume ( has implemented async transcoding of media files.

In ploneintranet we ran into quite some problems with and have started to work on a new architecture that avoids most of the drawbacks of current async stacks, by providing a full HTTP-request based async stack, so that callbacks are fully authenticated and whathaveyou.

Unfortunately this work is currently stalled on a feature branch but we hope to pick up on that after the holiday. Please let me know if you're interested in collaborating on this!

1 Like

I'am very happy with collective.zamqp and use it in several projects.

1 Like

Yes, FWIW, I want to do something with (or similar to), which I am using now with a fork of zc.async (release version of zc.async, by bringing in stale uuid package, breaks things [1]).


Cool, I will take a look.


we found collective.taskqueue as a good alternative, and we will try to use it in one project.

1 Like

Short comments on behalf of collective.zamqp and collective.taskqueue:

  • c.taskqueue is probably the only standalone async solution working without external services
  • c.taskqueue doesn't have any monitoring OOTB
  • c.taskqueue supports work distribution and quaranteed delivery with redis (yet, I've only used it without redis, because using c.zamqp for complex stuff)
  • c.zamqp has seen production a lot, and I'm not aware of ever losing a task
  • c.zamqp requires RabbitMQ, but allows direct control of its routing and message structure (opinionated routing and message were the issues, which removed celery from my options)
  • c.zamqp requires grok (it's that old) and there's no refactoring in progress, because "it's not broken"
  • c.zamqp works well with externalized services / workers, c.taskqueue expects everything be done and stored in Plone
  • both rely on asyncore and won't work on wsgi

2 posts were split to a new topic: Problem to make async part of collective.documentviews work