> ## Documentation Index
> Fetch the complete documentation index at: https://koreai.mintlify.app/llms.txt
> Use this file to discover all available pages before exploring further.
# Health and Monitoring - NLP
[Back to NLP Topics](/ai-for-service/automation/natural-language/nlp-topics)
The Health and Monitoring dashboard provides goal-driven insights into your app's NLP model performance. It analyzes training data, test coverage, and test results to surface actionable recommendations.
**Go to:** Testing > Regression Testing > **Health & Monitoring**
***
## Dashboard Sections
| Section | Description |
| -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **NLP** | Aggregates batch test results. Shows performance metrics and test coverage for Dialog intents, FAQs, Small Talks, Traits, and Entities. |
| **Flow** | Summarizes conversation flow coverage. Shows transition coverage and intent summary from conversation test suites. |
***
## NLP Metrics
Displayed as aggregate values in the **Bot Health** summary and as individual scores in the intent-type panels (Dialog Intents, FAQs, Small Talk, Traits).
| Metric | Description |
| ------------------------- | ------------------------------------------------------------------------------------ |
| **Accuracy** | Whether the identified intent is correct. |
| **F1 Score** | Weighted average of Precision and Recall. |
| **Precision Score** | Ratio of true positives to all predicted positives (TP / (TP + FP)). |
| **Recall Score** | Ratio of true positives to all actual positives (TP / (TP + FN)). |
| **Total Test Coverage %** | Average test coverage across Dialog Intents, FAQs, Small Talk, Traits, and Entities. |
***
## Test Cases Detailed Analysis
Click **View Test Cases** in the NLP section to open the detailed analysis window. It shows results for **Intents**, **Entities**, and **Traits**.
### Intents Tab
| Column | Description |
| ------------------- | ------------------------------------------------- |
| **Test Cases** | Test case name. |
| **Intent Type** | Dialog intent, FAQ, or Small Talk. |
| **Expected Intent** | Intent expected from the utterance. |
| **Matched Intent** | Intent actually matched. |
| **Result Type** | True Positive, False Positive, or False Negative. |
| **Tags** | Follow-up labels assigned by the analyst. |
### Entities Tab
| Column | Description |
| ------------------ | -------------------------------------- |
| **Utterances** | The user utterance in the test case. |
| **Entity Name** | Entity name mapped to the test case. |
| **Expected Value** | Entity value expected. |
| **Matched Value** | Entity value actually matched. |
| **Entity Result** | True (matched) or False (not matched). |
| **Tags** | Follow-up labels. |
### Traits Tab
| Column | Description |
| ------------------ | ------------------------------------------------- |
| **Test Cases** | Trait test case name. |
| **Intent Type** | Displays "Trait". |
| **Trait Name** | Name of the trait analyzed. |
| **Expected Trait** | Trait expected from the utterance. |
| **Matched Trait** | Trait actually matched. |
| **Trait Result** | True Positive, False Positive, or False Negative. |
| **Tags** | Follow-up labels. |
### Tags
Tags are labels for intent, entity, and trait test results that indicate follow-up actions.
| Tag | Meaning |
| -------------------------- | ------------------------------------------------ |
| **Add Negative Pattern** | A negative pattern should be added. |
| **NeedNLPHelp** | Requires explicit NLP support. |
| **Needs Negative Pattern** | Needs a negative pattern to work as expected. |
| **Needs Training** | App needs training for this intent/entity/trait. |
| **New Intent** | A new intent was detected during execution. |
***
## NLP Analysis (per Test Case)
Click the **NLP Analysis** tab in the test case detail view to see the historic NLP analysis captured at the time of execution. Shows qualified (definitive and probable) and disqualified intents for:
* Traits (if applicable)
* ML engine
* FM engine
* KG engine
* Trait Rule (if applicable)
* Ranking and Resolver
This differs from Utterance Testing, which shows current analysis based on the latest training data. NLP Analysis here is the snapshot from when the test ran.
***
## NLP Performance Metrics (per Intent Type)
| Result Type | Description |
| ----------------------- | ------------------------------------------------------ |
| **True Positive (TP)** | Utterances that correctly matched the expected intent. |
| **False Positive (FP)** | Utterances that matched an unexpected intent. |
| **False Negative (FN)** | Utterances that did not match the expected intent. |
***
## Performance Metrics Table
The details window provides a drill-down view for intents, entities, and traits:
| Metric | Intent | Entity | Trait |
| ---------------------------------------- | ------ | ------ | ----- |
| Expected Intent/Value | Yes | Yes | Yes |
| Matched Intent/Value | Yes | Yes | Yes |
| Parent Intent | Yes | No | Yes |
| Task State (Configured/Published) | Yes | No | Yes |
| Result Type | Yes | No | Yes |
| ML / FM / RR Scores | Yes | No | Yes |
| Entity Name | No | Yes | No |
| Result (True/False) | No | Yes | No |
| Identified by (NLU engine) | No | Yes | No |
| Identified using (entity type reference) | No | Yes | No |
| Confidence Score | No | Yes | No |
***
## Dialog Intent Summary
### Test Coverage
Shows the count and percentage of covered vs. uncovered intents. An intent is covered if it has at least one test case in the selected suite(s). Use **View details** to find uncovered intents and add test cases.
### Recommendations
Training recommendations appear when errors or warnings are triggered during execution. Click **View Recommendations** to see the summary and corrective actions.
***
## Intent Details Window
Click **View Details** in any summary panel (Dialog Intents, FAQs, Small Talks) to open a drill-down of performance metrics and recommendations.
**Training Data Summary columns:**
| Metric | Dialog Intent | FAQ | Small Talk |
| ---------------------------------- | ------------------------------- | ---------------------------- | ---------------------- |
| Intent | Intent name | FAQ intent name | Small Talk intent name |
| Utterances | Count of training utterances | N/A | — |
| Test Cases | Count in selected suite(s) | — | — |
| True Positive (TP) | Count of TP results | — | — |
| False Negative (FN) | Count of FN results | — | — |
| False Positive (FP) | Count of FP results | — | — |
| Covered In | Test suite names | — | — |
| F1 / Accuracy / Precision / Recall | Recommendation scores | — | — |
| Recommendations | Count + corrective actions link | N/A | N/A |
| Group | N/A | N/A | Small Talk group |
| Path | N/A | KG node path | N/A |
| Alt Question | N/A | Count of alternate questions | N/A |
**View Intents Not Covered**: Click the three-dot menu on the panel to list intents not covered in batch testing. Add them to your training data to improve coverage.
***
## FAQ Summary
Shows recommendation scores for FAQs from the latest batch test. Click **View Recommendations** to review the last report. Click **Knowledge Graph** to navigate to [KG Analysis](/ai-for-service/automation/knowledge-ai/knowledge-graph-analysis#knowledge-graph-analysis).
## Small Talk Summary
Shows recommendation scores for Small Talk interactions from the latest batch test. Click **Small Talk** to view group names, user utterances, and bot responses.
## Trait and Entity Summary
Shows recommendation scores for traits and entities from the latest batch test. Use **Test Coverage** and **Test Results Analysis** in each panel for details.
***
## Utterance Testing from Health and Monitoring
Click the **go to utterance testing** (magic wand) icon on the Test Cases Detailed Analysis page to open the Utterance Testing window. From there you can retrain the app based on test failures. See [Utterance Testing](/ai-for-service/automation/testing/testing-your-bot-with-nlp#utterance-testing).