How to Write Effective Briefs for AutoQA

Briefs help Quack AI understand how to interpret your evaluation questions.
They act as internal guidance for the AI model, improving scoring accuracy and consistency.

What Is a Brief

A Brief is a short instruction visible only to the AI.
It explains what a correct or complete answer should look like for a specific evaluation question.

Briefs are not shown to agents or human reviewers; they’re strictly for AI guidance.

Why Briefs Matter

  • Clarify intent: Define exactly what each question is evaluating

  • Improve accuracy: Reduce ambiguity and ensure consistent AI scoring

  • Enhance performance: Help the model handle diverse interaction types with greater reliability

💡 Pro Tip: Think of a brief as a quick scoring guide - it tells the AI exactly what to look for in a response and how to decide if it meets your standards.

How to Add a Brief

You can add a Brief when creating a new scorecard or editing an existing one.

  1. Go to Quality → Scorecards from your Quack AI dashboard.

  2. Select Create Scorecard, or open an existing one and click Edit.

  3. Navigate to Step 2: Evaluate.

  4. Click + Add Question.

  5. Enter your evaluation question.

  6. In the Brief field (optional), add a short instruction that tells the AI what a correct or complete answer looks like.

  7. Choose the question type (e.g., Rating or Achieved / Not Achieved).

  8. Click Add to save the question to your scorecard.

💡 Pro Tip: Add briefs to every question where “quality” could be interpreted in multiple ways, it helps the AI score more consistently.

Best Practices for Writing Briefs

  • Be specific: Describe what a correct response includes or avoids.

  • Keep it concise: One or two sentences is ideal.

  • Focus on outcomes: Define what makes a response complete, accurate, or aligned with your brand.

  • Use clear language: Avoid abstract terms like “good” or “professional”, specify what those mean.

💡 Pro Tip: Start each brief with an action verb (e.g., Evaluate whether, Give full credit if, Check that). This helps AutoQA interpret instructions consistently.

Example Briefs for Common QA Questions

1. Clarity

Question: Did the agent provide a clear and understandable response?
Brief: Evaluate whether the agent’s response is easy to follow and uses simple, customer-friendly language. Full credit if the explanation avoids jargon and directly answers the customer’s question.

2. Empathy

Question: Did the agent show understanding and empathy toward the customer?
Brief: Assess whether the agent acknowledges the customer’s feelings or frustration. Full credit if the tone is respectful, reassuring, and aligned with brand voice. Partial credit if empathy is implied but not explicitly expressed.

3. Accuracy

Question: Was the information provided accurate and aligned with internal policies?
Brief: Check that the agent’s answer is factually correct and consistent with company policies or documentation. Full credit if no misinformation or unsupported claims appear in the response.

💡 Pro Tip: Use a mix of behavioral (focused on tone or empathy) and technical (focused on accuracy or completeness) questions to give your AI well-rounded evaluation criteria.