Original question
I have a working version of my web application that I am trying to update at the moment, and I encountered the problem of completing a task that takes too long to complete in one HTTP request. The application receives the JSON list from the external JavaScript interface using the HTTP Post operation and returns a sorted / cut version of this list. As the input list grows, the sort operation takes a lot more time (obviously), so on suitable long input lists, I delete the 60-minute timeout of the HTTP request and the application fails.
I would like to start using the lazy library to do the sorting task, but I don’t understand how to store / retrieve data after this task is done. Here is my current code:
class getLineups(webapp2.RequestHandler):
def post(self):
jsonstring = self.request.body
inputData = json.loads(jsonstring)
playerList = inputData["pList"]
positions = ["QB","RB","WR","TE","DST"]
playersPos = sortByPos(playerList,positions)
rosters, playerUse = getNFLRosters(playersPos, positions)
try:
lineups = makeLineups(rosters,playerUse,50000)
self.response.headers["Content-Type"] = "application/json"
self.response.out.write(json.dumps(lineups))
except:
logging.error("60 second timeout reached on player list of length:", len(playerList))
self.response.headers["Content-Type"] = "text/plain"
self.response.set_status(504)
app = webapp2.WSGIApplication([
('/lineup',getLineups),
], debug = True)
Ideally, I would like to replace the entire try / except block with a call to the deferred task library:
deferred.defer(makeLineups,rosters,playerUse,50000)
But I do not understand how I will get the result from this operation. I think I will need to store it in the data store and then restore it, but how will my JavaScript interface know when the operation is complete? I read the documentation on Google, but I'm still unclear how to complete this task.
How i decided this
Using the main outline in the accepted answer, here is how I solved this problem:
def solveResult(result_key):
result = result_key.get()
playersPos = sortByPos(result.playerList, result.positions)
rosters, playerUse = getNFLRosters(playersPos,result.positions)
lineups = makeLineups(rosters,playerUse,50000)
storeResult(result_key,lineups)
@ndb.transactional
def storeResult(result_key,lineups):
result = result_key.get()
result.lineups = lineups
result.solveComplete = True
result.put()
class Result(ndb.Model):
playerList = ndb.JsonProperty()
positions = ndb.JsonProperty()
solveComplete = ndb.BooleanProperty()
class getLineups(webapp2.RequestHandler):
def post(self):
jsonstring = self.request.body
inputData = json.loads(jsonstring)
deferredResult = Result(
playerList = inputData["pList"],
positions = ["QB","RB","WR","TE","DST"],
solveComplete = False
)
deferredResult_key = deferredResult.put()
deferred.defer(solveResult,deferredResult_key)
self.response.headers["Content-Type"] = "text/plain"
self.response.out.write(deferredResult_key.urlsafe())
class queryResults(webapp2.RequestHandler):
def post(self):
safe_result_key = self.request.body
result_key = ndb.Key(urlsafe=safe_result_key)
result = result_key.get()
self.response.headers["Content-Type"] = "application/json"
if result.solveComplete:
self.response.out.write(json.dumps(result.lineups))
else:
self.response.out.write(json.dumps([]))
Javascript URL- queryLineups , , . , , . , , - , .