In your case, you can probably use top_k , which returns the highest k values. k may be a 1D vector defining the number of values up to 'top' for measurements. In your case, if you want the second axis set by k=[0, 5, 0] to be able to do this.
 tf.nn.top_k(matrix, k=[0,5,0], sorted=True) 
I did not run it. Hope this helps