Structured Output breaking.

[RangerFrank]

!!! Exception during processing !!!
Traceback (most recent call last):
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\suggest.py", line 611, in keyword_extract
parsed_response = json.loads(response)
^^^^^^^^^^^^^^^^^^^^
File "json_init_.py", line 346, in loads
File "json\decoder.py", line 337, in decode
File "json\decoder.py", line 353, in raw_decode
json.decoder.JSONDecodeError: Unterminated string starting at: line 3 column 15 (char 17)

[MilitantHitchhiker]

!!! Exception during processing !!!
Traceback (most recent call last):
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\suggest.py", line 611, in keyword_extract
parsed_response = json.loads(response)
^^^^^^^^^^^^^^^^^^^^
File "json_init_.py", line 346, in loads
File "json\decoder.py", line 337, in decode
File "json\decoder.py", line 353, in raw_decode
json.decoder.JSONDecodeError: Unterminated string starting at: line 3 column 15 (char 17)

[MilitantHitchhiker]

Over and over and over. keeps breaking workflows and preventing continuous processing without babysitting the node.

[gokayfem]

which llm you are using?

[MilitantHitchhiker]

llava 1.6 7b.Q5 mistral

[gokayfem]

maybe you can use more json output oriented llm, like nous hermes type llms, you can find their gguf files in huggingface

[MilitantHitchhiker]

Unfortunately the hermes models kind of suck for this purpose, the llava base models give significantly better outputs, any chance of adding some basic formatting error handling into the node?