> ## Documentation Index
> Fetch the complete documentation index at: https://koreai.mintlify.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Utterance Testing - NLP

<Badge icon="arrow-left" color="gray">[Back to NLP Topics](/ai-for-service/automation/natural-language/nlp-topics)</Badge>

Utterance Testing lets you enter user inputs and see how the NLP engine processes and matches them. Use it to validate intent identification and train the app directly from test results.

***

## How to Test

1. Open the app and go to **Testing > Utterance Testing**.
2. (Optional) For a multiple-intent model, select the **Intent Model** to scope ML detection.
3. In the **Type a user utterance** field, enter an utterance. Example: Book a flight.
4. Review the result: single match, multiple matches, or no match.

***

## Match Types

| Match Type           | Description                           | Behavior in Production                                                |
| -------------------- | ------------------------------------- | --------------------------------------------------------------------- |
| **Definitive Match** | High-confidence exact match.          | Single: task executes directly. Multiple: user is prompted to choose. |
| **Probable Match**   | Reasonably scored but not definitive. | Sent as "Did you mean?" suggestions to the user.                      |

**Possible outcomes:**

* **Single match**: Correct — proceed or train further. Incorrect — mark as incorrect and select the right intent.
* **Multiple matches**: Select the correct radio button and train.
* **Unidentified intent**: Select an intent from the dropdown and add training.

***

## Entity Match

Matched entities are displayed after intent detection. Processing order: NER and pattern entities first, then remaining entities.

Entity match details (v8.0+):

| Field                     | Description                                                                      |
| ------------------------- | -------------------------------------------------------------------------------- |
| **Identification Engine** | ML, FM, or KG                                                                    |
| **Training Type**         | NER, pattern, entity name, system concept, etc. Click for pattern match details. |
| **Confidence Score**      | ML NER score (only when Conditional Random Field is the NER model).              |

***

## NLP Analysis

Each test result includes an **NLP Analysis** box showing shortlisted intents, which model found them, scores, and the winning intent.

<Note>
  NLP scores are absolute values. They can only be compared against other tasks for the same input, not across different utterances.
</Note>

### ML Model

Matches the input against task labels and training utterances. Multi-sentence inputs are tested sentence by sentence. More training utterances increase discovery chances.

### FM Model

Scores every task using a custom NLP algorithm based on task names, synonyms, and patterns. Click **Processed Utterance** to see how the input was analyzed.

**FM scoring factors:**

| Factor                 | Description                                                                      |
| ---------------------- | -------------------------------------------------------------------------------- |
| **Words Matched**      | Score for the number of words matching the task name or utterances.              |
| **Word Coverage**      | Ratio of matched words to total task words (name, fields, utterances, synonyms). |
| **Exact Words**        | Words matched without synonyms.                                                  |
| **Sentence Structure** | Bonus for sentence structure match.                                              |
| **Word Position**      | Score for words near the start of the sentence.                                  |
| **Order Bonus**        | Bonus for words in the same order as the task label.                             |
| **Role Bonus**         | Bonus for matched primary/secondary roles (subject/verb/object).                 |
| **Spread Bonus**       | Bonus based on distance between first and last matched words.                    |
| **Penalty**            | Penalty for phrases before the task name or conjunctions in the task label.      |

FM scoring varies by language:

* **German/French**: Uses word roles, Universal Parts of Speech, and dependency relations.
* **Other languages**: Uses original word, role in sentence, and processed word (spell-corrected).

### Knowledge Graph

Extracts terms from the utterance and maps them to the KG to fetch relevant paths. Paths covering more than a threshold number of terms are shortlisted. A path with 100% term coverage and a matching FAQ is a perfect match. See [Knowledge Graph Training](/ai-for-service/automation/knowledge-ai/knowledge-graph-training#knowledge-graph-training).

### Ranking and Resolver

Determines the final winning intent across all engines.

* If ML or KG finds a **perfect match**, R\&R presents it without rescoring (even multiple perfect matches are shown as options).
* **All other good/unsure matches** are rescored using the FM model. If the rescored intent exceeds the threshold, it is also considered a match.

**R\&R V1 intent elimination reasons:**

* Intents matched only by entity values (date, number) from ML are eliminated.
* Possible matches are eliminated if a definitive match exists.
* A definitive match is eliminated if another definitive match appeared earlier in the utterance (multi-intent case).
* Intent patterns following a definitive match are eliminated.
* Intents below the minimum threshold are eliminated.
* Definitive matches against a negative pattern are eliminated.
* Intents with unmet pre-conditions are eliminated.
* KG "Search In Answer" definitive matches are eliminated if another match exists.

**R\&R V2** ranks only definitive matches from ML and KG (not FM) without rescoring. The R\&R window shows the winning intent, ranking score, eliminated intents and their reasons, and the final result. See [Ranking and Resolver](/ai-for-service/automation/natural-language/training/ranking-and-resolver#ranking-and-resolver-engine).

***

## Training from Utterance Testing

### Utterance-Based Training

1. After testing, open training options:
   * **Unmatched intent**: Select an intent from the **Select an Intent** dropdown.
   * **Multiple matches**: Select the radio button for the correct intent.
   * **Single match**: Click the matched intent name.
2. The utterance appears in the **ML Utterances** field. Click **Save** to add it to the intent.
3. Under **Intent Synonyms**, add synonyms for each word in the task name.
4. Under **Intent Patterns**, add task patterns.
5. Click **Re-Run Utterance** to verify improved scores.

### Matched Training Data

The **Ranking and Resolver NLP Analysis** window shows the training data (utterances, synonyms, questions, patterns, traits) that led to intent qualification.

| NLP Engine          | Supported Models   | Matched Training Data Shown                      |
| ------------------- | ------------------ | ------------------------------------------------ |
| Machine Learning    | Standard, Few-shot | Sample utterance for every qualified intent.     |
| Knowledge Graph     | Ontology, Few-shot | Questions/alternate questions of qualified FAQs. |
| Fundamental Meaning | All models         | Patterns for qualified intents.                  |

For qualified and eliminated intents, the system also shows the elimination reason alongside the matched utterance, processed utterance, and winning intent.

### Train with FAQ

To match utterances to FAQs:

* Set terms, term configuration, or classes from the FAQ page, train the KG, and retest.
* Add the utterance as an alternate question to the FAQ from the Knowledge Graph page, train, and retest.

See [Knowledge Graph Training](/ai-for-service/automation/knowledge-ai/knowledge-graph-training).

### Mark an Incorrect Match

When a user input matches the wrong task, mark it as incorrect and select the right intent to retrain.