Evaluation Forms

Evaluation Forms let QA managers create standardized assessments for voice and chat interactions. Forms align scoring with operational goals and ensure consistent, compliant evaluations. Each queue supports one evaluation form per channel.

Manual Evaluation lets QA managers mark specific metrics for manual scoring and exclude them from agent attributes and scorecard calculations.

Key Features

Feature	Description
Multi-language support	Localized metrics for accurate global team assessments
Flexible scoring	Percentage-based (simpler forms) or points-based (complex evaluations)
Advanced scoring options	Negative scoring, fatal criteria, and pass-score thresholds
Channel-specific configuration	Separate settings for Voice and Chat
Queue and channel assignment	Assign forms to specific queues and channels
AutoQA and manual audits	Both automated and manual audit scoring

How It Works

Evaluation Forms and Evaluation Metrics are the two core components:

• Evaluation Forms — defines the overall scoring structure, scoring type, language, channel, pass threshold, and queue assignments.
• Evaluation Metrics — defines the individual quality parameters used to measure agent performance.

QA managers create forms with weighted metrics totaling 100% and assign them to queues and channels for AutoQA scoring and manual audits.

Access Evaluation Forms

Navigate to Quality AI > CONFIGURE > Evaluation Forms.

The list shows the following columns:

Column	Description
Name	Evaluation form name
Description	Short description
Queues	Assigned and unassigned queues
Channel	Voice or Chat
Created By	Form creator
Pass Score	Minimum score for the agent to pass
Status	Enable or disable scoring for this form
Search	Quick search by name

Enable Auto QA in Quality AI Settings before creating evaluation forms.

---

Create a New Evaluation Form

1. Select the Evaluation Forms tab.
2. Select + New Evaluation Forms in the upper right.

The creation process has three sections: General Settings, Assignments, and Evaluation Metrics.

General Settings

1. Enter a Name and optional Description.
2. Select the required Language.
3. Select a Channel:
   • Chat — shows only Chat-relevant metrics; excludes speech-based and Voice-specific Playbook metrics.
   • Voice — shows all Voice-related metrics, including speech and Playbook metrics.
4. Select a Scoring Type (Percentage or Points).
5. Set the minimum Pass Score.
6. Select Next.

Assignments

This section assigns queues and defines the conversation source for evaluation.

1. Use the search bar to find available queues.
2. Select the required queues.
3. Select Add Queues.
4. Add or remove queue assignments as needed.
5. Select Next.

Conversation source options:

Source	Description
Quality AI Express	Enables Quality AI Express-based interaction processing
CCAI Integration	Ingests conversation data from CCAI
Agent AI Integration	Processes interactions received from Agent AI

If you assign CCAI or Agent AI queues together with Quality AI Express queues in the same form, By Playbook and By Dialog metrics become unavailable.

Queue assignment rules:

• Each queue can be associated with only one form per channel.
• The search list shows only queues assigned to the current user.

Metric configuration rules:

• The system validates metric availability based on conversation source, language, and channel.
• You can reorder metrics to control display order in the AI-Assisted Manual Audit screen.
• The metrics list shows only those configured for all selected form languages and the chosen channel.

Evaluation Metrics

1. Use the search bar to find evaluation metrics.
2. Add the selected metrics to the form.
3. Select Edit to configure each metric.
4. Choose the correct Response (what constitutes a match for this metric).
5. Assign a Weightage based on the scoring type:

   • Percentage — enter a percentage value.

     

   • Points — enter a points value.

     

Outcome Configuration

For each metric, define outcomes (for example, Yes or No) and assign positive, zero, or negative weight based on the expected response.

• Optionally enable Fatal Error for compliance-critical metrics.
• Select Create to finalize the form.

---

Scoring Types

Percentage-Based

• Assign metric weights as percentages.
• Total weight must equal 100%.
• Best for smaller forms (fewer than 20 metrics).

Points-Based

• Assign metric weights as point values — no upper limit on total positive points.
• Total negative points cannot exceed total positive points.
• Recommended for complex forms (20+ metrics, ideal for 40+).
• Supports Manual Evaluation metrics.

Points-Based Formula:

Kore Evaluation Score = [∑(Myi × Wyi) − ∑(Mni × Wni) / ∑(Wyi)] × 100

Where:

• Myi, Wyi = adhered metrics and positive points
• Mni, Wni = non-adhered metrics and negative points

Scoring Logic

• Pass — score ≥ Pass Score threshold.
• Fail — score < Pass Score threshold.
• Fatal error — sets the score to 0 and marks the interaction as failed, regardless of other metric scores.

Comparison

Feature	Percentage-Based	Points-Based
Best for	Smaller forms (under ~20 metrics)	Larger forms (20+ metrics)
Total weight	Must equal 100%	No fixed maximum
Scalability	Limited by 100% cap	High flexibility
Weight per metric	Decreases as metrics increase	Assign any point value
Negative scoring	Managed within 100%	Allowed, can’t exceed total positive
Final score	Direct percentage (0–100)	Normalized to percentage (0–100)

---

Language Selection

• Evaluation forms support multi-language selection.
• The system shows only By-Question metrics configured for all selected languages (AND condition).
• Example: selecting English and Dutch shows only metrics available in both.

---

Edit or Delete an Evaluation Form

1. Select the three-dot menu (⋮) next to the form.
2. Select Edit to update, or Delete to remove.
3. Select Update to save changes.

Switching Scoring Types

Changing the scoring type clears all existing metric weights and requires reconfiguration. Ensure the total meets validation rules for the new type.

---

Warnings and Error Handling

Language Configuration Warnings

Changes to language settings can affect speech recognition accuracy and metric results.

Metric Deletion Warnings

• The system shows a warning if any form uses the metric.
• Remove the metric from all associated forms before deleting.
• Reassign any linked attributes to a different metric.
• Deletion is only possible after all dependencies are resolved.

Unsupported Language Error (Form Level)

This error occurs when you add a language to a form, but some metrics do not support it.

To resolve:

1. Review language settings for each metric in the form.
2. Update each metric to support the new language.
3. Add the language to the form after updating all metrics.

Metric-Level Language Limitation

This warning appears when a metric doesn’t support a language configured at the form level.

To resolve:

1. Configure the required language in the metric.
2. Or replace the metric with one that supports all form languages.

Channel Mode Change Warning

• Switching between Voice and Chat deletes speech-based metrics automatically.
• Update remaining metrics and adjust weights accordingly.
• Select Update to save.

---