Deep dive · module 85 of 100
Monitoring and logging
Keep track of how AI systems perform in practice.
After this lesson
You can describe a simple way to monitor AI use and errors.
In view · An improvement loop
From signal to improvement
Track errors, complaints, quality, and effort.
Use a simple log without unnecessary sensitive content.
Adjust prompt, workflow, or agreements and measure again.
Part 1 of 3 · Follow signals
Your preference for less movement is on. Choose each part yourself.
What to monitor?
Errors, complaints, time savings, and anomalous results.
Start simply
A shared log or spreadsheet is often enough to get started.
Take action
Use the insights to adjust prompts, workflows, or agreements.
New practice example
From explanation to a usable result.
Good logging makes an error investigable without unnecessarily storing all prompt content or personal data.
Fictional practice situationA fictitious application creates summaries. The team wants to find errors, but the source texts may contain names and internal comments.
View a possible solution
This is how it can be done
Log a task number, timestamp, prompt version used, result status, and error type. For a repeatable substantive test, use an approved fictitious example. Do not store the full customer text in every technical log by default.
Why this works
Investigability and data minimisation must be designed together. More stored text is not automatically better evidence.
Make it slightly more difficult
Determine what notification you need if the model does not provide an answer and which data you specifically do not need to collect for that.
Extra practice, at your own pace. You do not need to submit anything. This does not count towards an automatic assessment.
Your workplace · learning by doing
Make something. See what changes.
Checking the AI connection… You can already view the written example.
Present your assignment to Ende.
Read the answer, change one thing, and compare.
Previously held
New version
You remain in control
What do you want to improve in your next attempt?
On this device only. AI answers remain only in this page memory. This is self-review, not a grade.
Why this approach works
Investigability and data minimisation must be designed together. More stored text is not automatically better evidence.
A step further
Determine what notification you need if the model does not provide an answer and which data you specifically do not need to collect for that.
A prompt to get you started
Describe a simple monitoring system for a team using AI for customer emails. What would you track?
Apply it
You want to track recurring errors. One proposal logs every complete customer message and all personal data indefinitely for that purpose.
Which approach best fits this situation?
- Always store all content indefinitely.
- Log necessary error categories and measurements with appropriate retention periods and access.
- Stop measuring as soon as the first version works.
With JavaScript, you can check your approach and receive immediate feedback.
Test your insight
What is the purpose of monitoring?
- To gain insight into performance and to be able to improve in a targeted way.
- Only to count costs.
- It is not necessary.
Extra practice
Note three things you would want to track if you were using AI structurally.
Number of corrections needed, types of errors, and time to usable result.
