How does the step parameter relate to the size of the samples in tf.contrib.learn?

Question

Canned grades in 1.0 (LinearClassifier, DNNClassifier, etc.) use the Trainable interface , which defines:

fit(
    x=None,
    y=None,
    input_fn=None,
    steps=None,
    batch_size=None,
    monitors=None,
    max_steps=None
)

and describes the steps as

The number of steps to model training. If not, train forever. "steps" work gradually. If you call twice (steps = 10), then the training takes place in a total of 20 steps. If you do not want to have incremental behavior, set max_steps instead. If set, max_steps should be None.

I do not understand what it means.

m = LinearClassifier(
    feature_columns=[sparse_column_a, sparse_feature_a_x_sparse_feature_b],
    optimizer=tf.train.FtrlOptimizer(
      learning_rate=0.1,
      l1_regularization_strength=0.001
    )) 

m.fit(input_fn=train_input_fn, steps=???)

Using the LinearClassifier, how do we train in one run train_input_fn? Should the steps be the number of samples in train_input_fn?

, train_input_fn 3 ?

1 ( ?): " - , input_fn "

, Estimator _train_model

:

all_hooks = []
self._graph = ops.Graph()
with self._graph.as_default() as g, g.device(self._device_fn):
  random_seed.set_random_seed(self._config.tf_random_seed)
  global_step = contrib_framework.create_global_step(g)
  features, labels = input_fn()
  .......
  .......
  with monitored_session.MonitoredTrainingSession(
      ...
      hooks=all_hooks + model_fn_ops.training_hooks,
      chief_only_hooks=chief_hooks + model_fn_ops.training_chief_hooks,
      ...
  ) as mon_sess:
    loss = None
    while not mon_sess.should_stop():
      _, loss = mon_sess.run([model_fn_ops.train_op, model_fn_ops.loss])

input_fn , , mon_sess.run([model_fn_ops.train_op, model_fn_ops.loss])

, input_fn step. , , ,

def train_input_fn():
    current_log = TRAIN_FILES.pop()
    with open('./logs/' + str(random.random()) + "__" + str(uuid.uuid4()) + "__" +  str(time.time()) + ".run", "wb") as fh:
        fh.write(("Ran log file %s" % (current_log)).encode('utf-8'))

> 1 .

Answer 1

Trainable ( ) , , steps max_steps, , , :

fit(steps=100) 200 . , fit(max_steps=100) , , 100 .

, , steps, fit , -, , fit, . . :

def input_fn():
    """Get features and labels in batches."""
    new_features = get_new_features()
    new_labels = get_new_labels()
    return new_features, new_labels

for _ in range(num_iterations):
    m.fit(input_fn=input_fn, steps=num_steps)  # runs num_steps times on current data set

num_steps. , , , , , , , . , , max_steps. , .

def input_fn():
    """Get ALL features and labels."""
    all_features = get_all_features()
    all_labels = get_all_labels()
    return all_features, all_labels

for _ in range(num_epochs):
    m.fit(input_fn=input_fn, max_steps=num_max_steps)  # runs num_max_steps times on full dataset

, , (, batch_size fit ), .

input_fn fit, input_fn , fit .

Answer 2

LinearClassifier, train_input_fn? - train_input_fn?

input_fn : ( ) ( ). - .

input_fn :

def input_fn():
    value_a = tf.constant([0.0])
    features = {'a': value_a}

    label = tf.constant([1.0])
    return features, labels

m.fit(), _input_fn_, . m.fit() , , queue_runners .. session.run(...) , .

, output_fn, :

m.fit(input_fn=input_fn, steps=1)

-, input_fn, .

, train_input_fn 3 ?

:

• input_fn tf.constants, m.fit()
• (https://www.tensorflow.org/programmers_guide/reading_data), _num_epochs_, , .

1.1 (https://www.tensorflow.org/api_docs/python/tf/estimator/inputs) input_fn, numpy pandas . _num_epochs _.