Deep dive · module 77 of 100
Multimodal workflows
Combine text, images and documents in a single workflow.
After this lesson
You design a simple workflow that uses multiple types of input.
In view · Working together towards one result
Multiple sources into one action list
Use a fictitious or permitted image as the first source.
Provide the corresponding text separately and recognisably.
Combine the information and compare actions against both sources.
Part 1 of 3 · Photo of a whiteboard
Your preference for less movement is on. Choose each part yourself.
What is multimodal?
The model can process text, images and documents simultaneously.
Practical example
A photo of a whiteboard + notes → structured action list.
Check
Check both the text and the interpretation of images.
New practice example
From explanation to a usable result.
A multimodal workflow combines sources, but must keep differences between text, image, and table visible.
Fictional practice situationA fictitious poster shows Friday 19 o'clock. The accompanying text says Friday 20 o'clock. The location is the same in both sources. No one has designated a definitive version.
View a possible solution
This is how it can be done
Create an overview with two start times and their source. Report the conflict and ask which version applies. You can mention the shared location in advance, but do not publish a self-chosen start time.
Why this works
Multiple sources do not only add information; they can also contradict each other. The model must not resolve such a conflict silently.
Make it slightly more difficult
Add a blurry photo with an unreadable room number. How do you keep perception, uncertainty, and required confirmation separate?
Extra practice, at your own pace. You do not need to submit anything. This does not count towards an automatic assessment.
Your workplace · learning by doing
Make something. See what changes.
Checking the AI connection… You can already view the written example.
Present your assignment to Ende.
Read the answer, change one thing, and compare.
Previously held
New version
You remain in control
What do you want to improve in your next attempt?
On this device only. AI answers remain only in this page memory. This is self-review, not a grade.
Why this approach works
Multiple sources do not only add information; they can also contradict each other. The model must not resolve such a conflict silently.
A step further
Add a blurry photo with an unreadable room number. How do you keep perception, uncertainty, and required confirmation separate?
A prompt to get you started
Describe a workflow in which you combine an image and a short text to achieve a usable result.
Apply it
You are reading a fictitious invoice from a photo. One amount is blurry and the calculated total differs.
Which approach best fits this situation?
- Use the total because the model answers confidently.
- Fill in the blurry amount so that the sum is correct.
- Check the read values and recalculate; leave the unreadable amount blank.
With JavaScript, you can check your approach and receive immediate feedback.
Test your insight
What is an advantage of multimodal workflows?
- You can process different types of information in a single step.
- It is always more accurate.
- It only works with text.
Extra practice
Sketch a multimodal workflow for one of your tasks.
Photo of an invoice + question “what is the total amount?” → verified answer.
