Can I set a timeout when an object is removed from the TensorFlow queue?

Question

I create an ensemble of a neural network where each network reads the input data from the input queue and writes its prediction to the output queue, and a separate client pushes a new input to all input queues, then pulls individual predictions from each output queue and aggregates them to create an ensemble prediction.

I would like the system to be resistant to a slow (or crashed) neural network client, so I need to set a timeout for output from each output queue. Ideally, the aggregator graph will behave beautifully and simply ignore this prediction.

The only solution I found has a dequeue timeout is to set the configuration parameter operation_timeout_in_mswhen creating the session, but this applies to all operations in the graph (for this session). Not very granular.

Any other option?

Answer 1

You can set a timeout for an individual call tf.Session.run(), which is most useful if you have a potentially blocking operation, for example dequeue(). To do this, pass an optional object tf.RunOptionsto call run()and set the timeout_in_msrequired timeout for the field in milliseconds

op = ...  # Assume this depends on dequeuing a tensor from a queue.
sess = tf.Session()

# Set a 10-second timeout.
run_options = tf.RunOptions(timeout_in_ms=10000)
try:
  sess.run(op, options=run_options)
except tf.errors.DeadlineExceededError:  # This will be raised if the timeout expires.
  # ...