Admin Readiness Review

10 topics · Cloud · Hybrid · Self-Hosted

Work through each topic to verify your LangSmith organization is configured for production. Check items off as you complete them — progress is saved in your browser.

0 of 10 topics complete0%
Foundation
Org Structure & Workspaces0/7

Your LangSmith organization is the top-level container for workspaces, users, and billing. A well-planned workspace structure — one per team with clear ownership — is foundational and difficult to restructure later.

Identity & Access
SSO & SCIM0/10

SAML SSO centralizes authentication through your Identity Provider. SCIM automates user lifecycle — adding and removing users in LangSmith when their IdP status changes. Both are required for enterprise-grade access management.

User Roles & RBAC0/6

LangSmith has two role levels: organization roles (Admin, User, Viewer) and workspace roles (Admin, Editor, Viewer, plus custom). A deliberate role strategy minimizes over-permissioning and reduces security risk.

ABAC Policy0/4

Attribute-Based Access Control (ABAC) adds row-level data scoping on top of RBAC — restricting which traces a user can see based on resource tags. Available for all LangSmith form factors: Cloud, Hybrid, and Self-Hosted.

Keys & Secrets
API Key Strategy0/6

LangSmith has two key types: Personal Access Tokens (lsv2_pt_) for individuals and Service Keys (lsv2_sk_) for applications and integrations. Using the right type for each use case is critical for security and operational continuity.

Workspace Secrets0/4

Workspace secrets store LLM provider API keys (OpenAI, Anthropic, etc.) within LangSmith — available to evals, prompts, and Fleet agents without hardcoding credentials in application code. Required for Fleet to function.

Data Management
Data Retention0/5

LangSmith retains trace data for a configurable period. Base retention is 14 days. Extended retention (up to 400 days for Enterprise) is available at the org or workspace level.

Usage Limits0/4

Usage limits cap trace volume at the org or workspace level to prevent unexpected cost spikes from misconfigured tracing, runaway agent loops, or traffic bursts.

Resource Tags0/4

Resource tags are key-value pairs that organize projects, datasets, and prompts within a workspace. The "Environment" tag enables dev/staging/prod separation within a single workspace — simpler than maintaining one workspace per environment.

Agents & Integrations
Fleet & MCP0/5

Fleet is LangSmith's no-code agent platform. Before your team can run agents, admins must configure workspace secrets with an LLM API key, set spend controls, and connect MCP servers. These are admin-only actions.