How to define your own operators in TensorFlow

Question

In TensorFlow, we can use tf.nn.l2_loss()to perform L2 regularization. Let's say I want to define my own regularization operator for L1-regularization (let's call it tf.nn.l1_loss()). How can i do this? I find it difficult to find operator definitions in the TensorFlow source code.

Answer 1

As the comment says, there is a guide for adding op to TensorFlow . This guide describes adding a new op that is implemented in C ++. In general, you should do this in the following situations:

• The op operation cannot be implemented using existing TensorFlow operating tools (for example, it l1_losscan be implemented using the existing element-wise and reduction as a Python function).
• To improve performance (or memory consumption), a C ++ implementation is needed.
• The op operation can be implemented as part of the ops, but has a gradient that can be calculated more efficiently (or with better numerical stability) than the calculation of op-by-op gradients. (This is why it tf.nn.l2_lossis implemented as a compiled op in C ++.)