> ## Documentation Index
> Fetch the complete documentation index at: https://koreai.mintlify.app/llms.txt
> Use this file to discover all available pages before exploring further.
# By Question Metric
The By Question metric evaluates how well agents answer specific questions during interactions. You can apply it to all conversations (static) or use it selectively in trigger-based scenarios (dynamic). It supports focused feedback, targeted coaching, and continuous improvement.
***
## What It Offers
* Standardized quality assessment framework
* Customizable evaluation criteria based on business needs
* Multi-language support for global operations
* Automated evaluation suggestions through AI
## When to Use This Metric
| Use Case | Description |
| ------------------------------- | ------------------------------------------------------------------------------ |
| **Quality Assurance** | Systematically evaluate agent adherence to protocols or expected responses |
| **Training Assessment** | Measure how well agents follow prescribed interaction patterns |
| **Compliance Monitoring** | Verify how agents deliver critical information (disclaimers, privacy policies) |
| **Performance Standardization** | Apply consistent evaluation criteria across agents and interactions |
***
## How It Works
The metric uses two adherence approaches:
### Static Adherence
* Evaluates agent responses across **all conversations** with no conditional requirements.
* Best for universal standards like greetings or mandatory procedures.
* No trigger conditions required.
### Dynamic Adherence
* Evaluates agent responses only when specific triggers occur.
* Uses customer or agent utterances as triggers.
* Best for conditional scenarios requiring specific responses in certain contexts.
***
## Configure By Question Metric
1. Navigate to **Quality AI** > **Configure** > **Evaluation Forms** > **Evaluation Metrics**.
2. Select **+ New Evaluation Metric**.
3. From the **Evaluation Metrics Measurement Type** dropdown, select **By Question**.
4. Enter a descriptive **Name** (for example, `Agent's Warm Greeting`).
5. Select a **Language**.
6. Enter an evaluation **Question**-a reference prompt for supervisors during audits and reviews.
7. Select the **Adherence Type** (Static or Dynamic).
For **Static**, configure at least one agent answer utterance. For **Dynamic**, configure at least one trigger and one agent answer utterance.
***
## Static Adherence Configuration
Applies to all calls without any condition or trigger. Use for mandatory, universal compliance items like greeting scripts or regulatory disclaimers.
* Define acceptable utterances for the queue.
* Set a similarity threshold to evaluate whether the agent's response matches the predefined utterance.
* Configure at least one agent utterance template.
***
## Dynamic Adherence Configuration
Evaluates adherence only when a configured trigger is detected. Use for context-sensitive checks.
* Define at least one trigger (customer or agent utterance) and one acceptable agent response.
* Set the similarity threshold based on criticality:
* **Lower threshold (\~60%, yellow)**-for casual interactions or greetings
* **Higher threshold (\~100%, green)**-for critical topics such as legal disclaimers or privacy policies
### Trigger Configuration
Choose the utterance source that initiates the trigger:
| Trigger Type | Description |
| ---------------------- | ------------------------------------------------------------------------------------------ |
| **Customer Utterance** | Triggered by something the customer says (for example, `I need a refund`) |
| **Agent Utterance** | Triggered by something the agent says (for example, `Can I transfer this call to support`) |
You can add multiple trigger utterances and define AND/OR conditions to control when the trigger activates.
### Trigger Detection Method
| Method | Description |
| ----------------- | ------------------------------------------------------------------------------------- |
| **GenAI-Based** | Uses LLMs to detect intent contextually. No sample utterances or thresholds required. |
| **Deterministic** | Relies on predefined sample utterances and semantic similarity matching. |
***
## Enable GenAI-Based Features (Prerequisites)
To activate GenAI-based features:
1. Navigate to **Manage** > **Generative AI** > **GenAI Features**.
2. Enable and publish the following features:
* **GenAI-based agent answer adherence**
* **GenAI-based customer trigger detection**
***
## Agent Answer Configuration
### GenAI-Based Adherence
Uses LLMs to detect meaning, context, and intent evaluates whether the agent's answer fulfills the intent, even if phrased differently.
1. Select **GenAI-Based Adherence** as the answer detection method.
2. Enter a **Description** explaining the metric's intent.
Before using GenAI-based adherence, ensure supported models and the GenAI features (GenAI-based agent answer adherence and customer trigger detection) are enabled. No sample utterances or thresholds are required — LLMs evaluate adherence using zero-shot prompts. For effective prompt writing, see the [Auto QA Prompting Guide](#autoqa-prompting-guide).
### Deterministic Adherence
Evaluates responses based on semantic similarity to predefined reference utterances.
1. Select **Deterministic Adherence**.
2. Define an **Answer**-acceptable utterances per queue. Use Generative AI to suggest similar variations.
3. Set the **Similarity** threshold:
* Lower threshold (\~60%) for soft skills like greetings and etiquette.
* Higher threshold (\~100%) for compliance-critical statements (Policy, Privacy, Disclaimers).
***
## Count Type Configuration
Choose how the metric evaluates adherence across the conversation:
### Entire Conversation
Evaluates adherence throughout the entire interaction at any point.
### Time Bound
Focuses on specific timeframes-the first or last X seconds or messages.
1. Select **Parameter**-First Part or Last Part of the conversation.
2. Set the evaluation window:
* **Voice**-enter number of seconds.
* **Chat**-enter number of messages.
3. Select **Create** to save and activate the metric.
***
## Edit or Delete By Question Metric
1. Select the metric from the By Question category.
2. Choose an option: **Edit** to modify, or **Delete** to remove.
3. Select **Update** to save changes.
### Language Dependency Warnings
* You can't remove a language if any evaluation form or attribute uses it.
* Remove the language from all associated forms and attributes before modifying.
### Delete Warnings
Before deleting a metric:
1. Remove it from all associated evaluation forms.
2. If any attributes link to the metric, assign them to a different metric first.
3. The system helps deletion only after all dependencies are resolved.
***