I conducted a training task with tensor flow and got the following curve for the loss in the test set. After the 6000th iteration, the network begins to gain momentum. Therefore, I would like to get a model before re-equipment.
My workout code is as follows:
train_step = ......
summary = tf.scalar_summary(l1_loss.op.name, l1_loss)
summary_writer = tf.train.SummaryWriter("checkpoint", sess.graph)
saver = tf.train.Saver()
for i in xrange(20000):
batch = get_next_batch(batch_size)
sess.run(train_step, feed_dict = {x: batch.x, y:batch.y})
if (i+1) % 100 == 0:
saver.save(sess, "checkpoint/net", global_step = i+1)
summary_str = sess.run(summary, feed_dict=validation_feed_dict)
summary_writer.add_summary(summary_str, i+1)
summary_writer.flush()
After training, only five control points are saved (19600, 19700, 19800, 19900, 20,000). Is there a way that allows shadoworflow to maintain a breakpoint according to a validation error?
P.S. , tf.train.Saver max_to_keep, . ( ). , . ?