General Configuration Principles
- Maintain diversity and realism in metric configurations.
- Ensure configurations reflect plausible agent responses.
- Review and update configurations regularly to reflect current use cases.
Sentence Length and Complexity
- Limit configurations to concise utterances — typically 1–3 sentences.
- Avoid configuring very long sentences or extensive text passages.
Handling Specific Identifiers
Company and Brand Names
- Include when necessary, but be aware they can impact semantic adherence.
- Use a moderate threshold (for example, 50–60% match) to accommodate variations.
Agent Names
- Minimize the use of specific agent names in configurations.
- Use
[Agent Name]as a placeholder where the agent references themselves or other agents.
Product Names and Model IDs
- Avoid including specific model identifiers unless necessary.
- If included, expect precise usage in agent responses.
- Use a moderate threshold (for example, 50–60% match) for these configurations.
Threshold Recommendations
| Configuration Type | Recommended Match Threshold |
|---|---|
| General configurations | 60–70% |
| Configurations with company or brand names (policy) | 70–80% |
| Configurations with product specifics | 50–60% |
Adjust these thresholds based on your specific use case and requirements.
Testing and Validation
- Validate configurations against real agent responses.
- Monitor and analyze adherence rates to identify areas for reconfiguration.