I am using the Tensorflow Object Detection API (github.com/tensorflow/models/tree/master/object_detection) with one object detection task. Right now I have a problem servicing the discovery model that I trained with Tensorflow Serving (tensorflow.imtqy.com/serving/).
1. The first problem that I am facing is exporting the model to useable files. Detecting the api object kindly included exporting the script, so that I can convert the ckpt files to pb variable files. However, the output files will not contain content in the variable folder. I, although it was a mistake and reported this to Github, but it seems that they interned to convert variables to constants so that there are no variables. The detail can be found HERE .
The flags that I used when exporting the saved model are as follows:
CUDA_VISIBLE_DEVICES=0 python export_inference_graph.py \
--input_type image_tensor \
--pipeline_config_path configs/rfcn_resnet50_car_Jul_20.config \
--checkpoint_path resnet_ckpt/model.ckpt-17586 \
--inference_graph_path serving_model/1 \
--export_as_saved_model True
It works fine in python when I switch --export_as_saved_model to False.
But still, I have a problem servicing the model.
When I tried to run:
~/serving$ bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=gan --model_base_path=<my_model_path>
I got:
2017-07-27 16:11:53.222439: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:155] Restoring SavedModel bundle.
2017-07-27 16:11:53.222497: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:165] The specified SavedModel has no variables; no checkpoints were restored.
2017-07-27 16:11:53.222502: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:190] Running LegacyInitOp on SavedModel bundle.
2017-07-27 16:11:53.229463: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:284] Loading SavedModel: success. Took 281805 microseconds.
2017-07-27 16:11:53.229508: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: gan version: 1}
2017-07-27 16:11:53.244716: I tensorflow_serving/model_servers/main.cc:290] Running ModelServer at 0.0.0.0:9000 ...
, , " SavedModel , ".
, . .
2. .
:
from __future__ import print_function
from __future__ import absolute_import
from grpc.beta import implementations
import tensorflow as tf
import numpy as np
from PIL import Image
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
tf.app.flags.DEFINE_string('server', 'localhost:9000',
'PredictionService host:port')
tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format')
FLAGS = tf.app.flags.FLAGS
def load_image_into_numpy_array(image):
(im_width, im_height) = image.size
return np.array(image.getdata()).reshape(
(im_height, im_width, 3)).astype(np.uint8)
def main(_):
host, port = FLAGS.server.split(':')
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
image = Image.open(FLAGS.image)
image_np = load_image_into_numpy_array(image)
image_np_expanded = np.expand_dims(image_np, axis=0)
request.model_spec.name = 'gan'
request.model_spec.signature_name = 'predict_images'
request.inputs['inputs'].CopyFrom(
tf.contrib.util.make_tensor_proto(image_np_expanded))
result = stub.Predict(request, 60.0)
print(result)
if __name__ == '__main__':
tf.app.run()
request.model_spec.signature_name = 'predict_images', exporter.py script api (github.com/tensorflow/models/blob/master/object_detection/exporter.py), 289:
signature_def_map={
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
detection_signature,
},
To:
signature_def_map={
'predict_images': detection_signature,
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
detection_signature,
},
, .
:
bazel-bin/tensorflow_serving/example/client --server=localhost:9000 --image=<my_image_file>
:
Traceback (most recent call last):
File "/home/xinyao/serving/bazel-bin/tensorflow_serving/example/client.runfiles/tf_serving/tensorflow_serving/example/client.py", line 54, in <module>
tf.app.run()
File "/home/xinyao/serving/bazel-bin/tensorflow_serving/example/client.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/xinyao/serving/bazel-bin/tensorflow_serving/example/client.runfiles/tf_serving/tensorflow_serving/example/client.py", line 49, in main
result = stub.Predict(request, 60.0)
File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 324, in __call__
self._request_serializer, self._response_deserializer)
File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 210, in _blocking_unary_unary
raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.NOT_FOUND, details="FeedInputs: unable to find feed output ToFloat:0")
, .
, , , script , , , AbortionError github.com/tensorflow/tensorflow/blob/f488419cd6d9256b25ba25cbe736097dfeee79f9/tensorflow/core/graph/subgraph.cc. , . , .
, . , , , . , ? !