Constrain Llama3.2-vision output to a list of options

chef_carol

I have several images of animals in the same directory as the script. How can I modify the following script to process an image but force the output to only be a single selection from a list: from pathlib import Path import base64 import requests def encode_image_to_base64(image_path): """Convert an image file to base64 string.""" return base64.b64encode(image_path.read_bytes()).decode('utf-8') def extract_text_from_image(image_path): """Send...

Originally asked by coolhand on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Llama3.2-vision supports structured outputs, so you can specify the schema of the response you want, and the model should follow it. Your request should look like this: payload = { "model": "llama3.2-vision", "stream": False, "messages": [ { "role": "user", "content": ( "Classify this image into one of these exact categories:\n" "- dog\n" "- cat\n" "- butterfly\n" ), "images": [base64_image] } ], "format": { "type": "object", "properties": { "animal": { "enum": [ "dog", "cat", "butterfly" ], "type": "string" } }, "required": [ "animal" ] } } And now the output should be a JSON with an attribute animal . For example: { "animal": "dog" }

Full answer by Sergio Peñafiel: https://stackoverflow.stackexchange.com/a/79542346 (CC BY-SA 4.0).

Need it done for your business? UphillTech.