For better or worse, we have migrated our whole [LAMP](https://en.wikipedia.org/wiki/LAMP_(software_bundle)) web application from dedicated machines to the cloud (Amazon EC2 machines). It&#39;s going great so far but the way we do [crons](https://en.wikipedia.org/wiki/Cron) is sub-optimal. I have a Amazon-specific question about how to best manage cron jobs in the cloud using &quot;the Amazon way&quot;.

**The problem**: We have multiple webservers, and need to run crons for batch jobs such as creating RSS feeds, triggering emails, many different things actually. BUT the cron jobs **need to only run on one machine** because they often write to the database so would duplicate the results if run on multiple machines.

So far, we designated one of the webservers as the &quot;master-webserver&quot; and it has a few &quot;special&quot; tasks that the other webservers don&#39;t have. The trade-off for cloud computing is reliability - we don&#39;t want a &quot;master-webserver&quot; because it&#39;s a single point of failure. We want them to all be identical and to be able to upscale and downscale without remembering not to take the master-webserver out of the cluster.

How can we redesign our application to convert Linux cron jobs into transitory work items that don&#39;t have a single point of failure?

My ideas so far:

* Have a machine dedicated to only running crons. This would be a little more manageable but would still be a single-point-of-failure, and would waste some money having an extra instance.
* Some jobs could conceivably be moved from Linux crons to [MySQL Events][1] however I&#39;m not a big fan of this idea as I don&#39;t want to put application logic into the database layer.
* Perhaps we can run all crons on all machines but change our cron scripts so they all start with a bit of logic that implements a locking mechanism so only one server actually takes action and the others just skip. I&#39;m not a fan of this idea as it sounds potentially buggy and I would prefer to use a Amazon best-practice rather than rolling our own.
* I&#39;m imagining a situation where jobs are scheduled somewhere, added to a queue and then the webservers could each be a worker, that can say &quot;hey, I&#39;ll take this one&quot;. [Amazon Simple Workflow Service][2] sounds exactly this kind of thing but I don&#39;t currently know much about it so any specifics would be helpful. It seems kind of heavy-weight for something as simple as a cron? Is it the right service or is there a more suitable Amazon service?

----------

*Update:* Since asking the question I have watched the [Amazon Simple Workflow Service][3] webinar on YouTube and noticed at 34:40 ([http://www.youtube.com/watch?v=lBUQiek8Jqk#t=34m40s][4]) I caught a glimpse of a slide mentioning cron jobs as a sample application. In their documentation page, &quot;[AWS Flow Framework samples for Amazon SWF][5]&quot;, Amazon say they have sample code for crons:

&gt; ...
&gt; **&gt; Cron jobs** In this sample, a long running workflow periodically
&gt; executes an activity. The ability to continue executions as new
&gt; executions so that an execution can run for very extended periods of
&gt; time is demonstrated.
&gt; ...

I downloaded the AWS SDK for Java ([http://aws.amazon.com/sdkforjava/][6]) and sure enough buried within a ridiculous layers of folders there is some java code (`aws-java-sdk-1.3.6/samples/AwsFlowFramework/src/com/amazonaws/services/simpleworkflow/flow/examples/periodicworkflow`).

The problem is, if I&#39;m honest, this doesn&#39;t really help as it&#39;s not something I can easily digest with my skillset. The same sample is missing from the PHP SDK and there doesn&#39;t seem to be a tutorial that walks though the process. So basically, I&#39;m still hunting for advice or tips.


  [1]: http://dev.mysql.com/doc/refman/5.6/en/events.html
  [2]: http://aws.amazon.com/swf/
  [3]: http://dev.mysql.com/doc/refman/5.6/en/events.html
  [4]: http://www.youtube.com/watch?v=lBUQiek8Jqk#t=34m40s
  [5]: http://aws.amazon.com/code/3015904745387737
  [6]: http://aws.amazon.com/sdkforjava/

All Amazon Swf Solutions on Gang of Coders

Total of 1 Amazon Swf Solutions

How to convert Linux cron jobs to "the Amazon way"?