Amplitude Experiment Management

Live

BEARER TOKEN

FEATURE FLAGS

Analytics

Feature flags, experiments, variants, holdouts, and deployments are managed through Amplitude Experiment, reached with a bearer token Scalekit vaults per tenant.

  • One vaulted bearer token: flag and experiment writes resolve the tenant's own management token at request time, never a token in a prompt.
  • Credentials stay vaulted: AES-256, resolved at request time, never in LLM context.
  • Scoped before every call: User permissions enforced. 90-day audit trail.
Amplitude Experiment Management
agent · Acme Q3
Run
Which experiments are still running past their planned end date?
S
amplitudeexperimentmanagement_list_flags
121ms
Experimentation agent
2 experiments are overdue: checkout-copy-v3 (ended Nov 12, still 50/50) and search-rank-beta (ended Nov 20, 10% holdout live).
Sources: 14 experiments, 2 deployments
amplitudeexperimentmanagement
14 experiments
18:29
Message Claude...

Tools your experimentation agent reaches for on Amplitude Experiment, scoped per tenant.

CALL ANY TOOL
Feature flags and experiments in one toolkit: flags, variants, cohort and user targeting, mutex groups, holdouts, and deployments. The bearer token stays vaulted.
amplitudeexperimentmanagement_add_experiment_variant_cohorts
Add experiment variant cohorts
Add specific cohorts to this experiment variant's targeting inclusions. This adds to the variant's existing cohort inclusions; it does not replace them. CONFIRMED from Amplitude's docs: POST /api/1/experiments/{id}/variants/{variantKey}/cohorts with body {"inclusions": [...]}, an array of cohort ID strings. CONFIRMED no inclusion-count limit is documented for cohorts on either the Flags or Experiments API docs pages — the 2,000-inclusion cap that exists for user/device inclusions is stated by Amplitude as applying specifically to users on the Flags side, and cohorts aren't mentioned there at all. A successful call returns 200 OK with the literal text "OK", not a JSON body. Rate limit: 100 requests/second and 100,000/day, shared across the whole Experiment Management API (not per-endpoint). CONFIRMED (inferred from the identical flags-side finding, not independently confirmed): a well-formed request with a syntactically valid cohort_id that does not correspond to a REAL, existing Amplitude cohort returns a generic HTTP 400 "Internal server error" from Amplitude, not a clean validation error — unlike user_ids, which accept arbitrary/free-form identifiers without requiring them to pre-exist. Use a real cohort ID from this Amplitude project's Cohorts, or expect this error.
Parameters
Name
Type
Required
Description
cohort_ids
string
Required
Cohort IDs to add to this variant's targeting, as a JSON-encoded array of strings.
id
string
Required
The experiment's ID.
variant_key
string
Required
The variant's key.
amplitudeexperimentmanagement_add_flag_variant_cohorts
Add flag variant cohorts
amplitudeexperimentmanagement_bulk_delete_experiment_variant_cohorts
Bulk delete experiment variant cohorts
amplitudeexperimentmanagement_bulk_delete_flag_variant_cohorts
Bulk delete flag variant cohorts
amplitudeexperimentmanagement_create_deployment
Create deployment
amplitudeexperimentmanagement_create_experiment
Create experiment
amplitudeexperimentmanagement_create_experiment_variant
Create experiment variant
amplitudeexperimentmanagement_create_flag_deployment
Create flag deployment
amplitudeexperimentmanagement_create_holdout_group
Create holdout group
amplitudeexperimentmanagement_delete_experiment_deployment
Delete experiment deployment
amplitudeexperimentmanagement_delete_flag_deployment
Delete flag deployment
amplitudeexperimentmanagement_get_experiment
Get experiment
amplitudeexperimentmanagement_get_experiment_variant_cohorts
Get experiment variant cohorts
amplitudeexperimentmanagement_get_experiment_version
Get experiment version
amplitudeexperimentmanagement_get_flag_variant
Get flag variant
amplitudeexperimentmanagement_get_flag_variant_users
Get flag variant users
amplitudeexperimentmanagement_get_holdout_group
Get holdout group
amplitudeexperimentmanagement_list_all_versions
List all versions
amplitudeexperimentmanagement_list_deployments
List deployments
amplitudeexperimentmanagement_list_experiment_variants
List experiment variants
amplitudeexperimentmanagement_list_experiments
List experiments
amplitudeexperimentmanagement_list_flag_deployments
List flag deployments
amplitudeexperimentmanagement_list_flag_versions
List flag versions
amplitudeexperimentmanagement_list_holdout_groups
List holdout groups
amplitudeexperimentmanagement_remove_all_experiment_variant_users
Remove all experiment variant users
amplitudeexperimentmanagement_remove_experiment_variant_user
Remove experiment variant user
amplitudeexperimentmanagement_update_deployment
Update deployment
amplitudeexperimentmanagement_update_experiment_variant
Update experiment variant
amplitudeexperimentmanagement_update_flag_variant
Update flag variant
amplitudeexperimentmanagement_update_mutex_group
Update mutex group

For more tools, view docs.

Build your Agent
Drop the toolkit in, point it at the tenant, and your agent can audit flags and roll variants from the first run.
Python · LlamaIndex
import { ScalekitClient } from "@scalekit-sdk/node";
import { DynamicStructuredTool } from "@langchain/core/tools";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
import { z } from "zod";

const sk = new ScalekitClient(envUrl, clientId, clientSecret);

const { tools } = await sk.tools.listScopedTools("user_123", {
filter: { connectionNames: ["amplitudeexperimentmanagement"], toolNames: ["amplitudeexperimentmanagement_list_flags", "amplitudeexperimentmanagement_get_experiment", "amplitudeexperimentmanagement_update_flag"] },
pageSize: 100,
});

const lcTools = tools.map((t) => new DynamicStructuredTool({
name: t.tool.definition.name,
description: t.tool.definition.description,
schema: z.object({}).passthrough(),
func: async (args) => {
const { data } = await sk.tools.executeTool({
toolName: t.tool.definition.name,
identifier: "user_123",
params: args,
});
return JSON.stringify(data);
},
}));

const agent = createReactAgent({ llm, tools: lcTools });
import { ScalekitClient } from "@scalekit-sdk/node";
import OpenAI from "openai";

const sk = new ScalekitClient(envUrl, clientId, clientSecret);
const openai = new OpenAI();

const { tools } = await sk.tools.listScopedTools("user_123", {
filter: { connectionNames: ["amplitudeexperimentmanagement"], toolNames: ["amplitudeexperimentmanagement_list_flags", "amplitudeexperimentmanagement_get_experiment", "amplitudeexperimentmanagement_update_flag"] },
pageSize: 100,
});

const llmTools = tools.map((t) => ({
type: "function",
function: {
name: t.tool.definition.name,
description: t.tool.definition.description,
parameters: t.tool.definition.input_schema,
},
}));

const resp = await openai.responses.create({
model: "gpt-4o", input: prompt, tools: llmTools,
});
import { ScalekitClient } from "@scalekit-sdk/node";
import Anthropic from "@anthropic-ai/sdk";

const sk = new ScalekitClient(envUrl, clientId, clientSecret);
const anthropic = new Anthropic();

const { tools } = await sk.tools.listScopedTools("user_123", {
filter: { connectionNames: ["amplitudeexperimentmanagement"], toolNames: ["amplitudeexperimentmanagement_list_flags", "amplitudeexperimentmanagement_get_experiment", "amplitudeexperimentmanagement_update_flag"] },
pageSize: 100,
});

const llmTools = tools.map((t) => ({
name: t.tool.definition.name,
description: t.tool.definition.description,
input_schema: t.tool.definition.input_schema,
}));

const msg = await anthropic.messages.create({
model: "claude-sonnet-4-6", max_tokens: 1024,
tools: llmTools,
messages: [{ role: "user", content: prompt }],
});
import { Agent } from "@google/adk/agents";
import {
MCPToolset, StreamableHTTPConnectionParams,
} from "@google/adk/tools/mcp";

const toolset = new MCPToolset({
connectionParams: new StreamableHTTPConnectionParams({
url: "https://mcp.scalekit.com/amplitudeexperimentmanagement",
headers: { Authorization: `Bearer ${userScopedToken}` },
}),
});

const agent = new Agent({
name: "agent", model: "gemini-2.0-flash",
tools: await toolset.getTools(),
});
Try these prompts
Paste any prompt into your agent to start managing Amplitude experiments from your workflows.
Flag hygiene
Copy the prompt
Copied
List flags that have been at 100% rollout for more than 30 days.
Copy the prompt
Copied
Which flags have no owner or description set?
Copy the prompt
Copied
Show every flag active in the production deployment.
Experiment operations
Copy the prompt
Copied
Which experiments are running past their planned end date?
Copy the prompt
Copied
Add cohort [cohort_id] to the treatment variant of [experiment_key].
Copy the prompt
Copied
Roll [flag_key] to 25% and confirm the new allocation.
Targeting
Copy the prompt
Copied
List users individually targeted into variants on [flag_key].
Copy the prompt
Copied
Show mutex groups and which experiments share each one.
Copy the prompt
Copied
Remove the holdout on [experiment_key] and report the new state.
SEE HOW AUTH WORKS
Your users connect once. Their Amplitude Experiment Management credentials stay vaulted, every call is scope-checked, and every action is logged.
1
Authorize
Your user connects
Amplitude Experiment Management
once. We tie it to their identity and the meetings they approved — no shared bot account, no org-wide access
Who:
user ‘A’
when:
Once per user
access:
Limited to user
2
Store
Their
Amplitude Experiment Management
token lives in a vault scoped to them. User A's meetings are never reachable by an agent acting for user B, even on the same connection
vault:
encrypted
scope:
per-user
tokens:
auto-refreshed
3
Resolve
When your agent calls a
Amplitude Experiment Management
tool, we fetch the right token server-side. It never touches your agent, never appears in the LLM context, never shows up in your logs
speed:
~40ms
check:
before every call
seen by:
nobody
4
Audit
Every
Amplitude Experiment Management
tool call is logged — who triggered it, which meeting was fetched, what came back. 90 days of history, tied to the user who authorized it
history:
90 days
export:
SIEM-ready
logged:
every call
Test other agents
See the same per-user auth pattern across other connectors.
GTM and RevOps Teams
Revenue forecast commentary
Pulls open pipeline from Salesforce and HubSpot, calculates coverage against quota, flags at-risk stages, posts commentary to Slack, and logs every snapshot to Google Sheets.
GTM and RevOps Teams
Deal intelligence agent
Pulls recent Gong calls, scores deal risk with an LLM, cross-references the record in Attio, and DMs each owner their at-risk deals in Slack. Every read is scoped to that rep's own access.
Engineering Teams
Engineering standup agent
Pulls commits from GitHub and GitLab, tracks issue movement in Jira, and posts a per-engineer standup brief to Slack. Each engineer's activity is read on their own delegated OAuth.
GTM and RevOps Teams
Competitive intelligence briefing agent
Scans Gong calls for competitor mentions, matches each one to its Notion battlecard, and DMs every affected rep a single Slack digest per cycle. Every call runs as the PMM who owns the briefing, never a shared bot.
Test other agents
See the same per-user auth pattern across other connectors.
SALES
Deal intelligence agent
Score Gong call risk with an LLM, cross-reference the Attio record, and DM each owner their at-risk deals in Slack.
ENGINEERING
Engineering standup agent
Pull commits from GitHub and GitLab, track Jira issue movement, and post a per-engineer standup brief to Slack.
GTM
Revenue forecast agent
Score pipeline coverage against quota across Salesforce and HubSpot, post forecast commentary to Slack, log snapshots to Sheets.
GTM
Competitive intelligence briefing agent
Scan Gong calls for competitor mentions, match each one to its Notion battlecard, and DM every affected rep a single Slack digest.
Why Scalekit
Secure your agent's access. Connectors ship in minutes
01.
Shared tokens break per-user analytics
A shared token looks fine in a demo. In production every call looks like a service account. Scalekit resolves the real user credential so attribution, audit, and scope stay accurate.
// shared token
audit → bot_service_account
user_filter → broken

// scalekit
audit → user_abc
scope → enforced ✓
02.
Authentication is not authorization
03.
Multi-tenancy is architectural
04.
Amplitude Experiment Management today. Others tomorrow.
“Our agents act across Salesforce, Gong, Google Drive, and more, on behalf of every customer. Scalekit behind the scenes meant we can keep adding tools without ever rebuilding how credentials or tool calling work.”
Venu Madhav Kattagoni
Head of Engineering / Von
FAQs
Frequently Asked Questions
Does the agent access Amplitude Experiment Management as the user or as a shared key?
As the user. Each workspace member authorizes once and Scalekit resolves their credential at request time. Audit logs attribute every action to that user, not a shared service account.
Where is the Amplitude Experiment Management bearer token stored?
In Scalekit's managed AES-256 token vault, namespaced per tenant. Refresh is automatic. Revocation is a single dashboard action. Tokens never appear in prompts, logs, or LLM context.
Can I limit what the agent is allowed to do in Amplitude Experiment Management?
Yes. Pass a tool name filter to listScopedTools so the analytics agent only sees the subset you authorize. Pre-API-call scope checks block out-of-policy actions before the request reaches Amplitude Experiment Management.
What happens when a user revokes Amplitude Experiment Management access?
The connection is invalidated on the next tool call. Subsequent requests for that user fail closed with a clear error. Other users in the tenant remain unaffected. The event is logged for audit.
Can the agent change a live rollout without review?
Only if the write tools are in scope. Flag and variant updates are separate tools from the list and get tools, so an audit agent stays read only while a release agent keeps write access. Each write is scope-checked before the Amplitude call and logged with the acting user.
Start in your coding agent
Up and running in one command
Install the Scalekit skill in your editor of choice. Connector, auth, tools, prompt, all wired up
Claude Code REPL
/plugin marketplace add scalekit-inc/claude-code-authstack
/plugin install agentkit@scalekit-auth-stack
Cursor Code REPL
# ~/.cursor/mcp.json
{
""mcpServers"": {
""amplitudeexperimentmanagement"": {
""url"": ""https://mcp.scalekit.com/amplitudeexperimentmanagement"",
""headers"": { ""Authorization"": ""Bearer $SCALEKIT_TOKEN"" }
}
}
}
Codex Code REPL
# ~/.codex/config.toml
[mcp_servers.amplitudeexperimentmanagement]
url = ""https://mcp.scalekit.com/amplitudeexperimentmanagement""
auth_env = ""SCALEKIT_TOKEN""
Copilot Code REPL
# .vscode/mcp.json
{
""servers"": {
""amplitudeexperimentmanagement"": {
""url"": ""https://mcp.scalekit.com/amplitudeexperimentmanagement"",
""type"": ""http""
}
}
}