# TensorFlow Serving API Clients {#ovms_docs_clients_tfs}

## Python Client

@sphinxdirective
.. raw:: html

    <div id="switcher-python" class="switcher-anchor">Python</div>
@endsphinxdirective

When creating a Python-based client application, there are two packages on PyPi that can be used with OpenVINO Model Server:
- [tensorflow-serving-api](https://pypi.org/project/tensorflow-serving-api/)
- [ovmsclient](https://pypi.org/project/ovmsclient/)

### Install the Package
@sphinxdirective

.. tab:: ovmsclient  

    .. code-block:: sh

        pip3 install ovmsclient 

.. tab:: tensorflow-serving-api  

    .. code-block:: sh  

        pip3 install tensorflow-serving-api 

@endsphinxdirective

### Request Model Status

@sphinxdirective

.. tab:: ovmsclient

    .. code-block:: python

        from ovmsclient import make_grpc_client

        client = make_grpc_client("10.20.30.40:9000")
        model_status = client.get_model_status(model_name="my_model")


.. tab:: tensorflow-serving-api

    .. code-block:: python

        import grpc
        from tensorflow_serving.apis import model_service_pb2_grpc, get_model_status_pb2
        from tensorflow_serving.apis.get_model_status_pb2 import ModelVersionStatus

        channel = grpc.insecure_channel("10.20.30.40:9000")
        model_service_stub = model_service_pb2_grpc.ModelServiceStub(channel)

        status_request = get_model_status_pb2.GetModelStatusRequest()
        status_request.model_spec.name = "my_model"
        status_response = model_service_stub.GetModelStatus(status_request, 10.0)
                
        model_status = {}
        model_version_status = status_response.model_version_status
        for model_version in model_version_status:
            model_status[model_version.version] = dict([
                ('state', ModelVersionStatus.State.Name(model_version.state)),
                ('error_code', model_version.status.error_code),
                ('error_message', model_version.status.error_message),
            ])
        
    
@endsphinxdirective

### Request Model Metadata

@sphinxdirective

.. tab:: ovmsclient

    .. code-block:: python

        from ovmsclient import make_grpc_client

        client = make_grpc_client("10.20.30.40:9000")
        model_metadata = client.get_model_metadata(model_name="my_model")


.. tab:: tensorflow-serving-api

    .. code-block:: python

        import grpc
        from tensorflow_serving.apis import prediction_service_pb2_grpc, get_model_metadata_pb2
        from tensorflow.core.framework.types_pb2 import DataType

        channel = grpc.insecure_channel("10.20.30.40:9000")
        prediction_service_stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)

        metadata_request = get_model_metadata_pb2.GetModelMetadataRequest()
        metadata_request.model_spec.name = "my_model"
        metadata_response = prediction_service_stub.GetModelMetadata(metadata_request, 10.0)

        model_metadata = {}

        signature_def = metadata_response.metadata['signature_def']
        signature_map = get_model_metadata_pb2.SignatureDefMap()
        signature_map.ParseFromString(signature_def.value)
        model_signature = signature_map.ListFields()[0][1]['serving_default']

        inputs_metadata = {}
        for input_name, input_info in model_signature.inputs.items():
            input_shape = [d.size for d in input_info.tensor_shape.dim]
            inputs_metadata[input_name] = dict([
                ("shape", input_shape),
                ("dtype", DataType.Name(input_info.dtype))
            ])

        outputs_metadata = {}
        for output_name, output_info in model_signature.outputs.items():
            output_shape = [d.size for d in output_info.tensor_shape.dim]
            outputs_metadata[output_name] = dict([
                ("shape", output_shape),
                ("dtype", DataType.Name(output_info.dtype))
            ])

        version = metadata_response.model_spec.version.value
        model_metadata = dict([
            ("model_version", version),
            ("inputs", inputs_metadata),
            ("outputs", outputs_metadata)
        ])

@endsphinxdirective

### Request Prediction on a Binary Input

@sphinxdirective

.. tab:: ovmsclient

    .. code-block:: python

        from ovmsclient import make_grpc_client

        client = make_grpc_client("10.20.30.40:9000")
        with open("img.jpeg", "rb") as f:
            data = f.read()
        inputs = {"input_name": data}    
        results = client.predict(inputs=inputs, model_name="my_model")


.. tab:: tensorflow-serving-api

    .. code-block:: python

        import grpc
        from tensorflow_serving.apis import prediction_service_pb2_grpc, predict_pb2
        from tensorflow import make_tensor_proto, make_ndarray

        channel = grpc.insecure_channel("10.20.30.40:9000")
        prediction_service_stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)

        with open("img.jpeg", "rb") as f:
            data = f.read()
        predict_request = predict_pb2.PredictRequest()
        predict_request.model_spec.name = "my_model"
        predict_request.inputs["input_name"].CopyFrom(make_tensor_proto(data))
        predict_response = prediction_service_stub.Predict(predict_request, 10.0)
        results = make_ndarray(predict_response.outputs["output_name"])

@endsphinxdirective

### Request Prediction on a Numpy Array

@sphinxdirective

.. tab:: ovmsclient

    .. code-block:: python

        import numpy as np
        from ovmsclient import make_grpc_client

        client = make_grpc_client("10.20.30.40:9000")
        data = np.array([1.0, 2.0, ..., 1000.0])
        inputs = {"input_name": data}
        results = client.predict(inputs=inputs, model_name="my_model")


.. tab:: tensorflow-serving-api  

    .. code-block:: python

        import grpc
        from tensorflow_serving.apis import prediction_service_pb2_grpc, predict_pb2
        from tensorflow import make_tensor_proto

        channel = grpc.insecure_channel("10.20.30.40:9000")
        prediction_service_stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)

        data = np.array([1.0, 2.0, ..., 1000.0])
        predict_request = predict_pb2.PredictRequest()
        predict_request.model_spec.name = "my_model"
        predict_request.inputs["input_name"].CopyFrom(make_tensor_proto(data))
        predict_response = prediction_service_stub.Predict(predict_request, 10.0)
        results = make_ndarray(predict_response.outputs["output_name"])

@endsphinxdirective

For complete usage examples see [ovmsclient samples](https://github.com/openvinotoolkit/model_server/tree/releases/2022/1/client/python/ovmsclient/samples).

## C++ Client

@sphinxdirective
.. raw:: html

    <div id="switcher-cpp" class="switcher-anchor">C++</div>
@endsphinxdirective

Creating a client application in C++ follows the same principles as Python, but using C++ adds some complexity. There is no package or library available with convenient functions for interacting with OpenVINO Model Server.

To successfully set up communication with the model server you need to implement the logic to communicate with endpoints specified in the [API](api_reference_guide.md). For gRPC, download and compile protos, then link and use them in your application according to the [gRPC API specification](model_server_grpc_api_tfs.md). For REST, prepare your data and pack it into the appropriate JSON structure according to the [REST API specification](model_server_rest_api_tfs.md).

See our [C++ demo](../demos/image_classification/cpp/README.md) to learn how to build a sample C++ client application in a Docker container and get predictions via the gRPC API. 

## Go Client

@sphinxdirective
.. raw:: html

    <div id="switcher-go" class="switcher-anchor">Go</div>
@endsphinxdirective

Creating a client application in [Go ](https://go.dev/) follows the same principals as Python, but Go adds complexity. There's no Go package or library available with convenient functions for interacting with OpenVINO Model Server.

To successfully set up communication with the model server, you need to implement the logic to communicate with endpoints specified in the [API](api_reference_guide.md). For gRPC, download and compile protos, then link and use them in your application according to the [gRPC API specification](model_server_grpc_api_tfs.md). For REST, prepare your data and pack it into the appropriate JSON structure according to the [REST API specification](model_server_rest_api_tfs.md).

See our [Go demo](../demos/image_classification/go/README.md) to learn how to build a sample Go-based client application in a Docker container and get predictions via the gRPC API.
