Binary encoding (base64) for input to inference endpoint

[aganders3]

Hello! Thanks for reading and any help is appreciated.

I have created a couple endpoints for model serving in databricks MLFlow, but I am having trouble sending requests. The inputs are somewhat large tensors and the signature/schema seems to be correct.

However, at this size they are too big when serialized using json.dumps(data.tolist()). The databricks docs mention using Tensorflow Serving format, which includes a section on encoding binary values. So I've tried using this format using something like this (python):

{     
   "instances": [
        {"b64": base64.b64encode(data.tobytes()).decode("utf-8")}
    ]
}

but I can't seem to get this to work. I get one of two errors, depending on whether I'm using the "instances" or the "inputs" format:

Request failed with status 400, {"error_code": "BAD_REQUEST", "message": "Invalid input. 
Failed to parse input data. This model contains an un-named tensor-based model signature 
which expects a single n-dimensional array as input, however, an input of type 
<class 'dict'> was found."}

Request failed with status 400, {"error_code": "BAD_REQUEST", "message": "Failed to 
enforce schema of data '{'b64': array([['rMwQvzbv6b12Pr2+kJLZP+XpXj8c/6Q/g3P...
lzrrvm5XMD8QjMM+vK9kvizybr4AMp4/cLsxP4hdOT5IbWQ/SuClPjIUGj9w5xo/ynVFPxkjzD+mPrK9P
lh7vd4Pnz19cLA+12o8PSw5C0AuS+S9Wm2gvlwMk74bOJu+3d1avyVBFL1GjIE+LQ4Vvkw6u77CuoG/hP
kSv9ZNRr+Zmvu+BVMMwE6BGz/AW6w/fJOWvyCwJD8YbIY+']],\n      dtype='<U5308416')}' with 
schema '[Tensor('float32', (-1, 3, 576, 576))]'. Error: This model contains a tensor-based model
signature with no input names, which suggests a numpy array input or a pandas dataframe input
with proper column values, but an input of type <class 'dict'> was found."}

Is there a way to send a binary-encoded tensor to such an input?