Article
Multimodal Visual Triage: Passing Images to the Decisions API
The Decisions API accepts text and image context in the same call. Screenshot triage, visual moderation, and image verification become one constrained decision.
One call, two modalities
Context for the Decisions API is not limited to text. You can pass an image — a screenshot, a photo, a document scan — alongside instructions and a choice set. The same fast classification path applies to both modalities.
Where vision decisions help
- Moderation: route a flagged image to allow, review, or block without a separate vision pipeline
- Support triage: classify a screenshot of an error state into the right fix path
- Verification: check a submitted photo against declared status labels
Designing the choice set
The choices should be mutually exclusive and exhaustive for the business case. If a visual input can legitimately fall outside the declared labels, add an explicit catch-all choice such as needs_human_review — the API will pick it instead of inventing a label.