Project Settings

Project Settings allow Project Admins to manage metadata, security, users, baselines, and model resources associated with a specific Project. Navigation: In the the left navigation of your project, go to Settings.

Settings Reference

Setting	Description
General Settings	Manage project metadata and archive the project.
Baseline Metrics	Configure financial, operational, and quality baselines.
Users & Roles	Add and manage user access for this project.
API Key Management	Generate, rotate, revoke, and view SDK integration keys.
Notification Preferences	Manage personal notification settings.
Models Registry	Register, test, and configure AI models used by the project.
Custom Attributes	Define custom attributes used by the project.

General Settings

Use this page to view and update your Project’s core information, including metadata, tags, and activity details. You can also archive Projects that are no longer active.

Action	Description
Review Project Metadata	View Project ID, created date, last activity, status, and counts for policies, datasets, team members, and models.
Edit Project Details	Update the project name, description, or tags to reflect changes in scope, purpose, or categorization.
Archive the Project	Deactivate the project using the option in the Danger Zone. Archived projects become read-only—no new data, policies, or configurations can be added, and associated API keys are disabled.

Before you archive a project, consider the following:

Projects cannot be deleted. Archiving is the only way to deactivate a project.
Archiving is permanent and cannot be undone.
Archived projects are removed from active views, but their data is preserved for reporting and compliance purposes.

Baseline Metrics

Define pre-automation metrics for your current manual processes to establish financial, operational, and quality baselines. These values are used to calculate ROI, measure efficiency improvements, and perform trend analysis after automation is applied.

This section is optional. If you added baseline metrics during project creation, you can update them here in Project Settings.

Financial Baselines

Used to understand the cost and revenue impact of manual workflows.

Field	Input Type
Avg. Manual Cost per Task	Currency + numeric input
Avg. Revenue per Task	Currency + numeric input

Operational Baselines

Used to measure time and throughput performance.

Field	Input Type
Avg. Manual Handling Time (AHT)	Numeric input + unit selector (minutes or hours)
Monthly Capacity per FTE	Numeric input

Quality Baselines

Used to evaluate customer experience and error reduction.

Field	Input Type
Current Baseline CSAT Score	Scale input (1–5 or 1–100)
Manual Process Error Rate (%)	Numeric input (percentage)

Tips & Best Practices

Use the Audit History tab to track all changes made to baseline metrics for this project.
Populate baselines for all relevant categories to get accurate ROI calculations.
Use optional fields strategically—incomplete baselines may affect trend analysis.
Review and update baselines periodically as processes change.

Users and Roles

Admins can control who can access and manage a project, ensuring team members have the correct permissions. The Users & Permissions page lists all project users and lets you manage their project-level roles.

Project Roles

Users retain their account-level role and gain an additional project-level role when added to a project.

Role	Access Level
Project Admin	Full access to all project features.
Project Viewer	Read-only access.

Adding Users to a Project

Project Admins can add workspace users to a project and assign project-specific roles. When adding users, you can select one or more workspace users and assign the appropriate role.

Revoking User Access

Project Admins can remove users from a project when access is no longer required. Once removed, the user can no longer view or modify project data. If access is needed again, the user must be re-added to the project.

Tips for Project Admins

Assign only the permissions necessary for the user’s role to reduce security risks.
Review permissions regularly to ensure users have only what they need.
Use bulk selection to onboard teams efficiently.

API Key Management

Manage SDK integration keys for secure, project-level connections. API keys identify the project when agents send telemetry to the platform, and serve as project identifiers for integrating external or third-party agents. This page lists all project API keys, including the first key created during project setup, with both active and revoked keys. Each key shows its name, masked value, status, creation date, creator, expiry date, and available actions (Revoke, Rotate, Delete).

Only Project Admins can generate, revoke, or rotate API keys.

Key Actions

Action	Description
Generate	Create a named API key with an optional expiration date. The key secret is displayed once and must be copied securely.
Revoke	Deactivate a key to prevent further authentication. Revocation is logged in Audit History.
Rotate	Refresh a key without losing metadata or history. The old key is deactivated, and the new key must be used for all integrations.

Tips & Best Practices

The key secret is shown only once—copy it securely immediately after generation or rotation.
Use descriptive key names to indicate environment, project, or purpose.
Rotate keys regularly to follow security best practices.
Monitor usage through Audit History for security and compliance.

Notification Preferences

Use Notification Preferences to control how and when you are alerted about policy evaluations, governance issues, or performance trends.

Notification Types

Type	Trigger
Policy Violations	Triggered when agent responses fail active policies.
Metrics Trend Deterioration	Triggered when aggregated performance metrics decline over time.

Notification Channels

Channel	Description
In-App	Receive notifications directly in the platform.
Email	Receive notifications at your registered email address.

Best practices for notifications

Use multiple channels (In-App + Email) for high-severity notifications such as policy violations.
Use a single channel for lower-severity notifications such as trend alerts.
Assign policy violation alerts to all admins, and trend notifications only to relevant owners.

Models Registry

The Models Registry lets you configure LLM providers for policy evaluations. By adding your LLM credentials, you enable the LLM-as-a-Judge framework to automatically detect policy violations.

Models configured here are used only for policy evaluation. They do not impact your agent’s production workloads.

Supported Providers

Provider	Notes
OpenAI	GPT-4, GPT-3.5
Anthropic Claude	Direct API
Google Vertex AI	—
Azure OpenAI Service	—
AWS Bedrock	Claude, Titan
Custom Endpoints	OpenAI-compatible APIs

How to Add a Model

Go to Project Settings → Models Registry.
Select Add Model.
Enter a Connection Name for the model configuration.
Select the Model Provider.
Enter the provider-specific credentials:
Field Description
API Key or Access Token Authentication credential for the provider.
Endpoint URL Required for Azure or custom deployments.
Model Name For example, gpt-4 or claude-3-sonnet.
Select Test Connection to verify credentials.
Save the configuration.

Field	Description
API Key or Access Token	Authentication credential for the provider.
Endpoint URL	Required for Azure or custom deployments.
Model Name	For example, `gpt-4` or `claude-3-sonnet`.

Managing Models

Action	Description
Edit	Update credentials, change model versions, or switch endpoints.
Test Connection	Re-verify credentials after updates.
Delete	Models linked to active policies cannot be deleted. Deactivate the associated policies first.

Using Models in Policies

When creating an LLM-as-a-Judge policy, select the model to use for evaluation. Models are invoked automatically during policy execution. You can assign different models to different policies based on their strengths—for example, GPT-4 for complex reasoning and Claude for safety checks.

Model parameters such as temperature, max tokens, top-p, and frequency or penalty settings are configured within each LLM-as-a-Judge policy, not in the Models Registry. Adjust these values when creating or editing a policy.

Custom Attributes

Custom Attributes let you extend the platform’s telemetry schema with fields specific to your domain. Define attributes once in the project settings, pass them through the SDK during agent execution, and use them as dimensions or measures in dashboards and policy definitions. The platform ships with a set of standard attributes derived from the SDK’s default telemetry schema. Custom attributes supplement these with organization-specific metadata, such as a risk score, customer tier, or deployment region, that the standard schema does not capture.

Overview

Section	Purpose
Standard Attributes	View the built-in telemetry fields provided by the SDK
Create a Custom Attribute	Define a new attribute schema for your project
Attribute Lifecycle	Manage attributes through Active, Inactive, and Archived states
Filtering and Search	Locate attributes by level or span type
Using Custom Attributes	Use attributes in dashboards, widgets, and policy definitions

Standard Attributes

The Standard tab displays the built-in telemetry fields defined by the SDK. These attributes are available in every project by default and represent the core data model for traces, spans, and sessions. Navigation: Settings → Custom Attributes → Standard

Column	Description
Attribute Name	The user-friendly display name shown in the UI
Technical Name	The key used in the SDK to pass this attribute (for example, `duration_ms`, `session.id`)
Level	The telemetry level at which the attribute is captured: Trace, Span, or Session
Span Type	The span category this attribute applies to, such as Performance or Runs. A dash (–) indicates the attribute is not scoped to a specific span type
Data Type	The attribute’s data type, such as `duration_ms`, `uuid`, or `Enum`
Description	A brief explanation of what the attribute represents
Dashboard	A toggle that controls whether this attribute is available as a dimension or measure in dashboards

You can toggle the Dashboard column to control whether a standard attribute appears in the widget builder. By default, commonly used attributes are enabled.

You can copy a standard attribute into a custom attribute if your system uses a different naming convention for the same data point.

Create a Custom Attribute

Define custom attributes to capture domain-specific metadata that the standard schema does not include. Once created, the attribute becomes available in the SDK integration, dashboards, and policy definitions. Navigation: Settings → Custom Attributes → Active → + Create Attribute

Click + Create Attribute.
In the Define new attribute panel, configure the following fields:
- Display name: A user-friendly name shown in dashboards, policies, and the UI. Example: Risk score.
- Technical name: The key your SDK integration uses to pass this attribute. Must be unique within the project. Example: risk_score.
- Description: A free-text explanation of what the attribute represents.
- Level: The telemetry level at which this attribute is captured: Trace, Span, or Session.
- Span Type: The span category this attribute applies to, such as Agent, LLM, or RAG. This field is available only when Level is set to Span.
- Data Type: The attribute’s value type. See Supported Data Types.
- Dashboard: Enabled by default. Controls whether this attribute is available in the widget builder.
Review the SDK Integration preview at the bottom of the panel. The platform generates a code snippet showing how to pass this attribute in your SDK calls based on the selected level.
Click Create attribute.

The new attribute appears in the Active tab. Supported Data Types

Data Type	Description	Additional Configuration
Boolean	True or false values	None
Number	Numeric values	Select Integer (whole numbers) or Float (decimal values)
Enum	A predefined list of allowed values	Choose from built-in templates (Environments, Severity, Tiers) or define a custom list
String	Free-text values	Optionally set a maximum length and a format constraint (for example, Email or UUID)
Timestamp	Date and time values	None

When a format constraint is set on a String attribute (for example, UUID), the platform validates incoming SDK data against that format. Values that do not match are rejected with an attribute validation error.

SDK Integration Preview When you create a custom attribute, the platform generates a context-aware SDK snippet based on the attribute’s level.

The snippet updates dynamically as you change the level or technical name.
For a span-level attribute, the snippet shows the attribute nested under the span’s attributes object.
For a trace-level attribute, the snippet shows the attribute at the trace level.

For example, a span-level attribute named risk_score produces:

bm.log(attributes={'risk_score': 95})

The preview ensures developers always have an accurate integration reference without leaving the configuration screen.

Attribute Lifecycle

Custom attributes follow a three-state lifecycle: Active, Inactive, and Archived. Each state is represented as a tab on the Custom Attributes page.

State	Description	Available Actions
Active	The attribute is live. Incoming SDK data is captured, and the attribute is available in dashboards and policies	Edit, Deactivate, Archive
Inactive	The attribute is paused. The platform stops capturing new data for this attribute, but existing data is retained	Reactivate (returns to Active), Archive
Archived	The attribute is permanently retired. It cannot be reactivated	None

To change an attribute’s state, hover over the attribute row to reveal the actions menu (⋮) and select the appropriate action.

Archiving is a permanent action. Once archived, an attribute cannot be returned to Active or Inactive status. Deactivate first if you are unsure whether the attribute is still needed.

Filtering and Search

Use the filter controls above the attributes table to locate specific attributes.

Control	Description
Level	Filter by telemetry level: Trace, Span, or Session
Span Type	Filter by span category: API, Agent, LLM, Orchestration, or RAG
Search	Free-text search across attribute names

Filters apply to whichever tab is currently active (Standard, Active, Inactive, or Archived).

Using Custom Attributes

Once a custom attribute is created and active, it is available across the platform. In Dashboards Custom attributes appear in the widget builder alongside standard attributes. The platform classifies them automatically based on data type:

Classification	Data Types	Widget Role
Custom Dimensions	String, Enum, Boolean	Used on the X-axis or as a color/series grouping
Custom Measures	Number, Timestamp	Used as the metric series (Y-axis value)

You can combine custom dimensions and measures with standard dimensions and measures in any widget. For example, use a custom region dimension on the X-axis and a standard Error Count measure on the Y-axis to visualize error distribution by region. In Policy Definitions Custom attributes are available as placeholders in natural-language policy prompts. When defining a policy rule, you can reference any active custom attribute to create governance conditions based on your domain-specific data.

Policy integration for custom attributes is being finalized and will be available in an upcoming release.

Best Practices

Practice	Rationale
Use consistent naming conventions	Simplifies SDK integration across teams and prevents duplicate technical names
Deactivate before archiving	Archiving is irreversible; deactivation lets you pause and evaluate before committing
Enable Dashboard selectively	Keep the widget builder focused by only exposing attributes your team actively uses for analytics
Prefer Enum with built-in templates	Standardizes values across teams and prevents free-text inconsistencies
Review SDK snippets before implementation	Ensures developers use the correct attribute key and nesting level

Managing Agents

Administration

References

Settings Reference

General Settings