TensorFlow Model Loses 0

Question

import tensorflow as tf
import numpy as np
def weight(shape):
return tf.Variable(tf.truncated_normal(shape, stddev=0.1))
def bias(shape):
return tf.Variable(tf.constant(0.1, shape=shape))
def output(input,w,b):
return tf.matmul(input,w)+b
x_columns = 33
y_columns = 1
layer1_num = 7
layer2_num = 7
epoch_num = 10
train_num = 1000
batch_size = 100
display_size = 1
x = tf.placeholder(tf.float32,[None,x_columns])
y = tf.placeholder(tf.float32,[None,y_columns])

layer1 = 
tf.nn.relu(output(x,weight([x_columns,layer1_num]),bias([layer1_num])))
layer2=tf.nn.relu
(output(layer1,weight([layer1_num,layer2_num]),bias([layer2_num])))
prediction = output(layer2,weight([layer2_num,y_columns]),bias([y_columns]))

loss=tf.reduce_mean
(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))
train_step = tf.train.AdamOptimizer().minimize(loss)

sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
for epoch in range(epoch_num):
   avg_loss = 0.
   for i in range(train_num):
      index = np.random.choice(len(x_train),batch_size)
      x_train_batch = x_train[index]
      y_train_batch = y_train[index]
      _,c = sess.run([train_step,loss],feed_dict=
{x:x_train_batch,y:y_train_batch})
      avg_loss += c/train_num
   if epoch % display_size == 0:
      print("Epoch:{0},Loss:{1}".format(epoch+1,avg_loss))
print("Training Finished")

My model Age: 2, loss: 0.0 Age: 3, loss: 0.0 Age: 4, loss: 0.0 Age: 5, loss: 0.0 Age: 6, loss: 0.0 Age: 7, loss: 0.0 Age: 8, loss: 0.0 Age: 9, loss: 0.0 Age: 10, loss: 0.0 Training completed

How can I solve this problem?

Answer 1

softmax_cross_entropy_with_logitsexpects tags in one hot form, i.e. with a form [batch_size, num_classes]. Here you have it y_columns = 1, which means only 1 class, which always is both predicted and the "truth of the earth" (from your point of view of the network), so your conclusion is always correct, no matter what the weight. Consequently, loss=0.

, , y_train . predictions [batch_size, num_classes], softmax_cross_entropy_with_logits tf.nn.sparse_softmax_cross_entropy_with_logits