Estimated Time: 20 minutes
What you’ll learn
By the end of this lesson, you’ll be able to:
- Understand how to evaluate AI outputs and processes thoughtfully
- Develop critical thinking skills for your AI interactions
- Learn to identify and address quality concerns in your AI interactions
Video: A Closer Look at Discernment
(5 minutes)
This video explores Discernment, the AI Fluency competency focused on thoughtfully evaluating AI outputs, processes, and behaviors. We explain that Discernment is the flip side of Description. While Description helps you communicate your intentions clearly, Discernment helps you evaluate whether what you receive meets your needs. The video introduces three types of Discernment:
- Product Discernment: Evaluating the quality of AI outputs
- Process Discernment: Assessing how the AI approached the task
- Performance Discernment: Evaluating how the AI behaved during the interaction itself
Together, these skills help ensure that your AI collaboration remains guided by thoughtful human judgment.
Key takeaways
- Discernment is your ability to thoughtfully evaluate what AI produces, how it produces it, and how it behaves
- Product Discernment focuses on evaluating the quality of actual outputs (accuracy, appropriateness, coherence, relevance)
- Process Discernment involves assessing how the AI arrived at its output, looking for logical errors, attention gaps, or inappropriate reasoning
- Performance Discernment evaluates how the AI behaves within the collaboration process itself, considering whether its communication style is effective for your needs
- Discernment works hand-in-hand with Description in a continuous feedback loop
- Even the most advanced AI systems benefit from human judgment and oversight
Exercises
Expert Discernment: Evaluating AI Responses in Your Domain
Activity Goal
Practice Product, Process, and Performance Discernment by evaluating AI-generated content in a domain where you have expertise, recognizing how your knowledge enhances your ability to critically assess AI outputs.
Instructions
Step 1: Return to Your Area of Expertise
Recall the topic you discussed with Claude in the earlier exercise (Lesson 2, Exercise 2: "Explore something you love"). This was a topic where you had strong knowledge and passion.
Step 2: Ask for Multiple Explanations
Start a new conversation with Claude and ask it to generate three different explanations or analyses about a specific aspect of your expert topic. For example:
- If your topic was photography, you might ask for three different explanations of depth of field
- If your topic was cooking, you might ask for three different analyses of fermentation techniques
- If your topic was history, you might ask for three different perspectives on a specific historical event
Step 3: Apply Your Expert Discernment
Drawing on your expertise, carefully evaluate each explanation provided by Claude:
Product Discernment:
- Which explanation contains the most accurate information?
- Are there any factual errors or misconceptions?
- Is the level of detail appropriate for someone learning about this topic?
Process Discernment:
- Does Claude follow logical reasoning in each explanation?
- Are there gaps in its analysis or thinking process?
- Does Claude make appropriate connections between concepts?
Performance Discernment:
- Was Claude attentive to your specific question and responsive to feedback and direction?
- Is terminology used appropriately for the topic?
- How does the tone and style affect the clarity of the explanation?
Step 4: Provide Feedback and Refinement
Based on your evaluation:
- Identify the strongest explanation and specifically tell Claude specifically why it's effective
- Identify the weakest explanation and provide specific feedback on what makes it problematic
- Work with Claude to create an improved version that addresses the issues you identified
Step 5: Reflection
Discuss with Claude (in the same chat):
- What specific knowledge did you have that allowed you to identify strengths or weaknesses?
- How might someone without your expertise struggle to discern the quality of these explanations?
- What does this experience teach you about the relationship between domain knowledge and effective Discernment?
For a more playful Discernment workout, you might want to try some of the “Game Night” suggestions in the final lesson ("Additional Activities").
Reflection
Before moving on, take a moment to consider:
- Which type of Discernment (Product, Process, or Performance) do you find most challenging to apply, and why?
- How does Discernment complement Description? How do they work together?
- What signals or patterns might indicate that an AI output requires closer scrutiny?
Lesson resources
Discernment summary slide (8.5x11)
A concise summary of the Discernment competency that you can print as a quick reference poster.
Discernment summary slide (16x9)
A concise summary of the Discernment competency that you can use in presentations.
What’s next
In the next lesson, you'll have the opportunity to apply both Description and Discernment skills to your overarching course project. You'll put into practice what you've learned about effectively communicating with AI and critically evaluating its outputs to produce results that leverage the best of both human and AI capabilities.
Feedback on this course
As you progress through the course, we'd love to hear from you about how you are using concepts from the course in your life, work, or classes and any feedback you may have. Share your feedback here.
Acknowledgments and license
Copyright 2025 Rick Dakan, Joseph Feller, and Anthropic. Released under the CC BY-NC-SA 4.0 license.
This course is based on The AI Fluency Framework by Dakan and Feller.
Supported in part by the Higher Education Authority, Ireland, through the National Forum for the Enhancement of Teaching and Learning.