How to fix this while integrating Machine Learning UDF function to Azure Stream Analytics

Pradyumn Joshi 40

Hi,

I am trying to integrate my ML endpoint with Azure Stream Analytics, but failing to do so as I cannot save the below details:

User's image

Here is my scoring script for the endpoint:

import os
import logging
import json
import joblib
from azureml.core.model import Model


from inference_schema.schema_decorators import input_schema, output_schema
from inference_schema.parameter_types.standard_py_parameter_type import StandardPythonParameterType

def init():
    global model
    global vectorizer
    logging.info(os.getenv("AZUREML_MODEL_DIR"))
    model_path = os.path.join(os.getenv("AZUREML_MODEL_DIR"), "trainingModel/sentimentPredictionModel.pkl")
    vectorizer_path = os.path.join(os.getenv("AZUREML_MODEL_DIR"), "trainingModel/sentimentVectorizer.pkl")
    
    # Load the models from the registered location
    model = joblib.load(model_path)
    vectorizer = joblib.load(vectorizer_path)
    logging.info("Init complete")
    
# Define the input schema
input_sample = "I like this"
input_schema_type = StandardPythonParameterType({'data':input_sample})

# Define the output schema
output_sample = {"prediction": [1]}
output_schema_type = StandardPythonParameterType(output_sample)

@input_schema('data', input_schema_type)
@output_schema(output_schema_type)

def run(data):
    print(data)
    logging.info(data)
    #value = json.loads(data)['data']
    #transformed_text = vectorizer.transform([value])
    transformed_text = vectorizer.transform([data])
    prediction = model.predict(transformed_text)
    return json.dumps({"prediction": prediction.tolist()})

According to my script, the schema it requires is {"data":"I like this"} as input and {"prediction":[1]} as output.

How can I correct this and save the function to invoke that in the stream analytics job.

1 answer

Sander van de Velde | MVP 30,711 Reputation points MVP

2024-06-04T18:03:34.5233333+00:00
Hello @Pradyumn Joshi,

welcome to this moderated Azure community forum.

The first possible issue is that no decent JSON is outputted by:

json.dumps({"prediction": prediction.tolist()})

It's not clear to me how 'prediction' looks like or what 'prediction.tolist()' produces.

Perhaps the problem is just lack of quotation:

{"data": I like this}

This is not correct JSON.

Please add some debug lines and check the output of the 'json.dumps()'

Then I noticed you say '{"data":"I like this"}' but the code outputs '{"prediction": prediction.tolist()}'.

Finally, Stream Analytics is complaining about yet another format containing 'inputName' or 'Inputs'.

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.
Please sign in to rate this answer.
Pradyumn Joshi 40 Reputation points

2024-06-05T05:51:40.7133333+00:00

Hi @Sander van de Velde

I have edited the question and added the expected output as well, I am not clear on what you said. Can you please elaborate a bit.

Sander van de Velde | MVP 30,711 Reputation points MVP

2024-06-05T07:10:36.26+00:00

Hello @Pradyumn Joshi,

Can you show the current output of your model endpoint?

Which tutorial are you following?

Please share the steps you execute within the Azure Stream Analytics job

Pradyumn Joshi 40 Reputation points

2024-06-05T07:17:30.3166667+00:00

The model fails to give the output as the input provided is not correct.

It gives the following error:

2024-06-05 07:14:09,328 E [72] azmlinfsrv - Encountered Exception: Traceback (most recent call last): File "/azureml-envs/azureml_f5a49a515211250fc6d6f3a98630820e/lib/python3.8/site-packages/azureml_inference_server_http/server/user_script.py", line 132, in invoke_run run_output = self._wrapped_user_run(**run_parameters, request_headers=dict(request.headers)) File "/azureml-envs/azureml_f5a49a515211250fc6d6f3a98630820e/lib/python3.8/site-packages/azureml_inference_server_http/server/user_script.py", line 156, in <lambda> self._wrapped_user_run = lambda request_headers, **kwargs: self._user_run(**kwargs) File "/azureml-envs/azureml_f5a49a515211250fc6d6f3a98630820e/lib/python3.8/site-packages/inference_schema/schema_decorators.py", line 64, in decorator_input kwargs[param_name] = _deserialize_input_argument(kwargs[param_name], param_type, param_name) File "/azureml-envs/azureml_f5a49a515211250fc6d6f3a98630820e/lib/python3.8/site-packages/inference_schema/schema_decorators.py", line 330, in _deserialize_input_argument raise ValueError("Invalid input data type to parse. Expected: {0} but got {1}".format( ValueError: Invalid input data type to parse. Expected: <class 'dict'> but got <class 'str'> The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/azureml-envs/azureml_f5a49a515211250fc6d6f3a98630820e/lib/python3.8/site-packages/azureml_inference_server_http/server/routes.py", line 222, in handle_score timed_result = main_blueprint.user_script.invoke_run(request, timeout_ms=config.scoring_timeout) File "/azureml-envs/azureml_f5a49a515211250fc6d6f3a98630820e/lib/python3.8/site-packages/azureml_inference_server_http/server/user_script.py", line 139, in invoke_run raise UserScriptException(ex) from ex azureml_inference_server_http.server.user_script.UserScriptException: Caught an unhandled exception from the user script 2024-06-05 07:14:09,328 I [72] azmlinfsrv - POST /score?verbose=true 500 0.819ms 92

when I pass the same input defined in the score script:

{"data":"I like this"}

I am following the Microsoft articles for scoring https://video2.skills-academy.com/en-us/azure/stream-analytics/machine-learning-udf

Pradyumn Joshi 40 Reputation points

2024-06-05T09:41:04.94+00:00

Also, i did change my score script and now it accepts this:

import os import logging import json import joblib from azureml.core.model import Model from inference_schema.schema_decorators import input_schema, output_schema from inference_schema.parameter_types.standard_py_parameter_type import StandardPythonParameterType def init(): global model global vectorizer logging.info(os.getenv("AZUREML_MODEL_DIR")) model_path = os.path.join(os.getenv("AZUREML_MODEL_DIR"), "trainingModel/sentimentPredictionModel.pkl") vectorizer_path = os.path.join(os.getenv("AZUREML_MODEL_DIR"), "trainingModel/sentimentVectorizer.pkl") model = joblib.load(model_path) vectorizer = joblib.load(vectorizer_path) logging.info("Init complete") # Define the input schema input_sample = "I like this" input_schema_type = StandardPythonParameterType(input_sample) # Define the output schema output_sample = {"prediction": [1]} output_schema_type = StandardPythonParameterType(output_sample) @input_schema('data', input_schema_type) @output_schema(output_schema_type) def run(data): print(f"Received input: {data}") logging.info(f"Received input: {data}") transformed_text = vectorizer.transform([data]) prediction = model.predict(transformed_text) result = {"prediction": prediction.tolist()} logging.info(f"Prediction result: {result}") return json.dumps(result)

Input:

{ "data": "I like this" }

Output:

{ "prediction": [ -1 ] }

While testing the endpoint, but still the stream is not letting me save the function

.

Sander van de Velde | MVP 30,711 Reputation points MVP

2024-06-05T12:43:19.3933333+00:00

Hello

Check out the video in this documentation, because it provides a walkthough.

I checked the video and there I see this function signature:

It seems this { "Inputs": ..... } is required.

Are you able to change the JSON input/output to this format?

Pradyumn Joshi 40 Reputation points

2024-06-10T07:55:36.5+00:00

I can see function signature on the ML studio but not in the ASA

Sander van de Velde | MVP 30,711 Reputation points MVP

2024-06-10T11:33:27.8233333+00:00

Does the function signature follow the same format?

I recommend trying some sample ML Studio project first to expose and endpoint. With that endpoint test the ML integration is Azure Stream Analytics.

If this does not work, contact Microsoft using a support request.

If it works, compare the output step/format from that example project with your output format.

Antonio Gonzalez 5 Reputation points

2024-06-10T15:54:47.7766667+00:00

{"data": I like this}

Sander van de Velde | MVP 30,711 Reputation points MVP

2024-06-11T07:03:29.2833333+00:00

This is not well-formed JSON.

It should be:

{"data": "I like this"}

We already established this { "Inputs": ..... } is required.

Please add context to your short answers and give a full report about the output of your service and one ML Studio sample.
Sign in to comment

Share via

How to fix this while integrating Machine Learning UDF function to Azure Stream Analytics

1 answer