Question about accuracy in coffee

Question

I have a network with 4 boolean outputs. This is not a classification problem, and each one makes sense. I expect to get zero or one for each of them. Right now I used the Euclidean loss function.

There are 1,000,000 samples. Each of them has 144 functions in the input file, so the input size is 1,000,000 * 144. I used a batch size of 50 because otherwise the processing time is too long. The output file has a size of 1,000,000 * 4, i.e. There are four exits for each entrance.

When I use the level of accuracy, he complains about the dimensionality of the output. He needs only one logical conclusion, not four. I think this is because it considers the problem as a classification problem. I have two questions. Firstly, given the accuracy level error, is the Euclidean loss function suitable for this task? And how can I get accuracy for my network? Secondly, I get the exact value of the predicted output for each of the four variables. I mean, I need accurate predicted values ​​for each test record. Now I have a loss value for each batch. Please help me solve these problems.

Thanks Afshin

Train network:

{ state { phase: TRAIN } layer { name: "abbas" type: "HDF5Data" top: "data" top: "label" hdf5_data_param { source: "/home/afo214/Research/hdf5/simulation/Train-1000-11- 1/Train-Sc-B-1000-11-1.txt" batch_size: 50 } } layer { name: "ip1" type: "InnerProduct" bottom: "data" top: "ip1" inner_product_param { num_output: 350 weight_filler { type: "xavier" } } } layer { name: "sig1" bottom: "ip1" top: "sig1" type: "Sigmoid" } layer { name: "ip2" type: "InnerProduct" bottom: "sig1" top: "ip2" inner_product_param { num_output: 150 weight_filler { type: "xavier" } } } 

The test network also:

  state { phase: TEST } layer { name: "abbas" type: "HDF5Data" top: "data" top: "label" hdf5_data_param { source: "/home/afo214/Research/hdf5/simulation/Train-1000-11- 1/Train-Sc-B-1000-11-1.txt" batch_size: 50 } } layer { name: "ip1" type: "InnerProduct" bottom: "data" top: "ip1" inner_product_param { num_output: 350 weight_filler { type: "xavier" } } } layer { name: "sig1" bottom: "ip1" top: "sig1" type: "Sigmoid" } layer { name: "ip2" type: "InnerProduct" bottom: "sig1" top: "ip2" inner_product_param { num_output: 150 weight_filler { type: "xavier" } } } layer { name: "sig2" bottom: "ip2" top: "sig2" type: "Sigmoid" } layer { name: "ip4" type: "InnerProduct" bottom: "sig2" top: "ip4" inner_product_param { num_output: 4 weight_filler { type: "xavier" } } } layer { name: "accuracy" type: "Accuracy" bottom: "ip4" bottom: "label" top: "accuracy" } layer { name: "loss" type: "EuclideanLoss" bottom: "ip4" bottom: "label" top: "loss" } 

And I get this error:

 accuracy_layer.cpp:34] Check failed: outer_num_ * inner_num_ == bottom[1]->count() (50 vs. 200) Number of labels must match number of predictions; eg, if label axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be N*H*W, with integer values in {0, 1, ..., C-1}. 

Without using an accuracy layer, the level gives me a loss value.

Answer 1

Should I use "EuclideanLoss" to predict binary outputs?

If you are trying to predict discrete binary labels, then "EuclideanLoss" not a good choice. This loss is better suited for regression tasks where you want to predict continuous values ​​(for example, evaluate coordinated bounding fields, etc.).
For predicting discrete labels, "SoftmaxWithLoss" or "InfogainLoss" best suited. Commonly used is "SoftmaxWithLoss" .
For predicting binary outputs, you can also consider "SigmoidCrossEntropyLoss" .

Why is there an error in the "Accuracy" layer?

In caffe , " Accuracy" layers assume two inputs ("lower"): one is the prediction vector, and the other is the expected discrete label. In your case, you need to provide for each binary output a vector of length 2 with predicted probabilities 0 and 1 and one binary label:

 layer { name: "acc01" type: "Accuracy" bottom: "predict01" bottom: "label01" top: "acc01" } 

In this example, you measure accuracy for a single binary output. The input "predict01" is two-vector for each example in the batch (for batch_size: 50 shape of this blob should be 50 by 2).

What can you do?

You are trying to predict 4 different outputs on the same network, so you need 4 different levels of loss and accuracy.
First, you need to divide ( "Slice" ) the labels of the basic truth into 4 scalars (instead of a single binary 4-vector):

 layer { name: "label_split" bottom: "label" # name of input 4-vector top: "label01" top: "label02" top: "label03" top: "label04" type: "Slice" slice_param { axis: 1 slice_point: 1 slice_point: 2 slice_point: 3 } } 

You should now have a level of prediction, loss, and accuracy for each of the binary labels.

 layer { name: "predict01" type: "InnerProduct" bottom: "sig2" top: "predict01" inner_product_param { num_outout: 2 # because you need to predict 2 probabilities one for False, one for True ... } layer { name: "loss01" type: "SoftmaxWithLoss" bottom: "predict01" bottom: "label01" top: "loss01" } layer { name: "acc01" type: "Accuracy" bottom: "predict01" bottom: "label01" top: "acc01" } 

Now you need to reproduce these three layers for each of the four binary labels that you want to predict.