In Amazon SWF, I can abuse the solution task to actually get the job done

Question

I need Amazon SWF to distribute some work, make sure it is done asynchronously, make sure it is stored in a reliable way, and that it automatically restarts. However, the workflow logic that I need is extremely simple: just complete one task.

I implemented it now the way it should be done:

• Executing a workflow
• The solver finds out about it and plans an action
• Workers learn about an activity request, execute results, and return results
• Decider notifies the result and copies it at the end of the workflow

It seems to me that I can just get the receiver to do this work - as it were - and immediately complete the workflow. This will take care of a lot of code. (Activity can also be unsuccessful, timeout, etc. Everything I need to service now.)

So, back to my question: can I make a decision that does the work itself and immediately completes the "workflow"?

Answer 1

Yes. In fact, I think you created an interesting precedent: the use of a minimal workflow as a centralized mechanism for blocking one-time actions in a distributed system - for example, cron jobs executed from one host in a fleet of many (hosts have to go through elections first and depending on which win is blocked to perform an action). The same could be done with Amazon SWF and a minimal amount of code:

A small Python example using boto.swf (use 1. from this post to configure the domain):

To program a solution:

 #MyDecider.py import boto.swf.layer2 as swf class OneShotDecider(swf.Decider): domain = 'stackoverflow' task_list = 'default_tasks' version = '1.0' def run(self): history = self.poll() if 'events' in history: decisions = swf.Layer1Decisions() print 'got the decision task, doing the work' decisions.complete_workflow_execution() self.complete(decisions=decisions) return False return True 

To run the solution:

 $ ipython -i decider.py In [1]: while OneShotDecider().run(): print 'polling SWF for decision tasks' 

Finally, to start the workflow:

 $ ipython In [1]: wf_type = swf.WorkflowType(domain='stackoverflow', name='MyWorkflow', version='1.0', task_list='default_tasks') In [2]: wf_type.start() Out[2]: <WorkflowExecution 'MyWorkflow-1.0' at 0x32e2a10> 

In the solver window, you will see something like:

 polling SWF for decision tasks polling SWF for decision tasks got the decision task, doing the work 

If your workflow is likely to change your business logic or increase in the number of actions, it is probably best to stick to the standard way of making business logic and work decisions when solving problems.

Answer 2

So far yes, you can do this (as indicated in another answer), before you do this, there are some things:

• Why are you using SWF to complete this task? Why install it as a workflow and pay for "StartWorkflow" if you can get the same benefits just by contacting your code more directly? If you need to track the execution and execution refinement, you can simply use the SQS queue for this and get the same results for cheaper ones.
• Your workflows can be extremely simple right now, but they can often evolve to be more complex over time. Designing from the start can save you time in the long run. Do you want future developers to work on your code, thinking that they should just add more logic to the workflow? Will they know how to use actions or just follow the existing template that you started with? (Hint - they will most likely copy your template - the developers are lazy :))