Google Gemini connector for AI agents

Tools your LLM ops agent reaches for on Google Gemini, scoped per user.

CALL ANY TOOL

26 tools for multimodal AI in Google Gemini: generate content, images, and video, create embeddings, count tokens, run and cancel batch jobs, manage context caches and files, and run interactions such as Deep Research.

gemini_batch_embed_contents

Batch embed contents

Create embeddings for several texts in one synchronous call, one vector per text. Returns embeddings, an array of {values} in the same order as the input texts. Use for up to about 100 short texts. Use gemini_embed_content for one input, and gemini_create_embeddings_batch for large asynchronous jobs at lower cost.

Parameters

Name

Type

Required

Description

model

string

Required

Model id without the models/ prefix, for example gemini-embedding-001. Use an embedding model.

texts

array

Required

Texts to embed, one embedding each. Example: ["What is AI?", "How do magnets work?"].

output_dimensionality

integer

Optional

Truncate the embedding to this many dimensions, for example 768, 1536 or 3072. Leave empty for the model's full size.

task_type

string

Optional

What the embedding is for, which tunes it: RETRIEVAL_QUERY, RETRIEVAL_DOCUMENT, SEMANTIC_SIMILARITY, CLASSIFICATION, CLUSTERING, QUESTION_ANSWERING, FACT_VERIFICATION or CODE_RETRIEVAL_QUERY. Honoured by gemini-embedding-001; for text with gemini-embedding-2 Google recommends a task prefix in the text instead. One of: `RETRIEVAL_QUERY`, `RETRIEVAL_DOCUMENT`, `SEMANTIC_SIMILARITY`, `CLASSIFICATION`, `CLUSTERING`, `QUESTION_ANSWERING`, `FACT_VERIFICATION`, `CODE_RETRIEVAL_QUERY`.

gemini_cancel_batch

Batch cancel

gemini_cancel_interaction

Cancel interaction

gemini_count_tokens

Count tokens

Count the input tokens a prompt uses on a Gemini model, including any system instruction and tool declarations you pass. Returns totalTokens, cachedContentTokenCount and per-modality promptTokensDetails. Use before gemini_generate_content to check a prompt fits the model's inputTokenLimit or to estimate cost. Counting is free.

Parameters

Name

Type

Required

Description

model

string

Required

Model id without the models/ prefix, for example gemini-3.8-flash. Token counts differ between models.

contents

array

Optional

Conversation turns, oldest first. Each turn has a role (user or model) and a parts array. A part is one of: text ({"text": "..."}); inline media as base64 ({"inlineData": {"mimeType": "image/png", "data": "<base64>"}}), for images, audio, video or PDF up to the request size limit; or a file reference ({"fileData": {"mimeType": "video/mp4", "fileUri": "https://generativelanguage.googleapis.com/v1beta/files/abc-123"}}) for a file already in the Files API or a public YouTube URL. Function results go back as {"functionResponse": {"name": "...", "response": {...}}}. Example: [{"role": "user", "parts": [{"text": "Summarize this in one line."}]}]. Use prompt instead for a single text turn.

prompt

string

Optional

Single user text turn. Shortcut for contents when you only need to send text; ignored when contents is set.

system_instruction

string

Optional

System instruction that sets the model's role, tone or rules. Text only.

tools

array

Optional

Tools the model may use. Each entry is one tool object: {"googleSearch": {}} grounds the answer in Google Search; {"urlContext": {}} lets the model read URLs in the prompt; {"codeExecution": {}} lets it run Python; {"fileSearch": {"fileSearchStoreNames": ["fileSearchStores/my-store"]}} searches a File Search store; {"functionDeclarations": [{"name": "get_weather", "description": "Get the weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}]} declares functions the caller runs. Example: [{"googleSearch": {}}].

gemini_create_batch

Batch create

gemini_create_cached_content

Create cached content

Create an explicit context cache holding large, reused input (documents, media file URIs, a system instruction or tools) for one model, so later requests send only the new part. Returns the cache: name (cachedContents/{id}), model, displayName, createTime, expireTime and usageMetadata.totalTokenCount. Use when many requests share a long prefix; pass the returned name as cached_content to gemini_generate_content. Gemini 2.5 and newer also cache implicitly with no setup. Explicit caching is not available for Gemini 4 Argon and later models, and the cached input must meet the model's minimum token count. Charges apply for cached tokens and storage time to the API key's Google Cloud project.

Parameters

Name

Type

Required

Description

model

string

Required

Model id without the models/ prefix, for example gemini-2.5-flash. The cache can only be used with this model.

contents

array

Optional

Content to cache, in the same format as gemini_generate_content contents: turns with a role and parts (text, inlineData base64 or fileData file URIs). Example: [{"role": "user", "parts": [{"fileData": {"mimeType": "application/pdf", "fileUri": "https://generativelanguage.googleapis.com/v1beta/files/abc-123"}}]}].

display_name

string

Optional

Readable name for the cache, up to 128 characters.

expire_time

string

Optional

Absolute expiry time in UTC, RFC 3339, for example 2026-10-21T18:00:00Z. Set ttl or expire_time, not both.

system_instruction

string

Optional

System instruction that sets the model's role, tone or rules. Text only.

tool_config

object

Optional

Tool configuration. Use {"functionCallingConfig": {"mode": "ANY", "allowedFunctionNames": ["get_weather"]}} to force a function call, mode AUTO to let the model decide, or NONE to forbid calls.

tools

array

Optional

ttl

string

Optional

Time to live as seconds with an s suffix, for example 3600s for one hour. Set ttl or expire_time, not both.

gemini_create_embeddings_batch

Batch create embeddings

gemini_create_interaction

Create interaction

Create an interaction on the Gemini Interactions API (beta): run a model or a managed agent such as Deep Research on text or multimodal input, with optional server-side conversation state, tools, structured output and background execution. Non-streaming. Returns the interaction: id, status (completed, in_progress, requires_action, failed and so on), model or agent, steps (thoughts, tool calls and the model_output content) and usage. Use for agents, background or long-running jobs, and multi-turn chats continued with previous_interaction_id. Use gemini_generate_content for a single stateless request, and gemini_get_interaction to poll a background run. Charges apply to the API key's Google Cloud project; interactions are stored (55 days on paid tier, 1 day on free tier) unless store is false.

Parameters

Name

Type

Required

Description

input

string

Required

The input: a plain string, or an array of content blocks such as {"type": "text", "text": "..."}, {"type": "image", "mime_type": "image/png", "data": "<base64>"}, or {"type": "image", "uri": ""}. To answer a function call, send a step list such as [{"type": "function_result", "call_id": "<id from the function_call step>", "name": "get_weather", "result": {"temp_c": 21}}] with previous_interaction_id. Example: "Explain how AI works in a few words".

agent

string

Optional

Managed agent to run instead of a model, for example deep-research-preview-04-2026 or deep-research-max-preview-04-2026. Agents need background set to true.

agent_config

object

Optional

Agent-specific settings for the chosen agent, as documented for that agent.

background

boolean

Optional

Run in the background and return immediately; poll with gemini_get_interaction. Required for agents. Not allowed with store false.

generation_config

object

Optional

Generation settings: temperature, top_p, max_output_tokens, stop_sequences, seed, thinking_level (minimal, low, medium, high), thinking_summaries (auto, none), tool_choice, image_config.

model

string

Optional

Model id, for example gemini-3.8-flash. Set exactly one of model or agent.

previous_interaction_id

string

Optional

Id of the previous interaction to continue the conversation from, so the server supplies the history.

response_format

object

Optional

Output format. For JSON output use {"type": "text", "mime_type": "application/json", "schema": {...JSON Schema...}}; for images {"type": "image", "aspect_ratio": "16:9"}.

response_modalities

array

Optional

Output modalities to request, from TEXT, IMAGE and AUDIO. Example: ["TEXT", "IMAGE"].

service_tier

string

Optional

Service tier: standard, flex (cheaper, best effort), priority, or deferred. One of: `standard`, `flex`, `priority`, `deferred`.

store

boolean

Optional

Whether to store the interaction (default true). Set false to keep nothing; this disables previous_interaction_id and background.

system_instruction

string

Optional

System instruction for this interaction. It is not carried over by previous_interaction_id.

tools

array

Optional

Tools for this interaction, each with a type, for example [{"type": "google_search"}], [{"type": "url_context"}], [{"type": "code_execution"}], or [{"type": "function", "name": "get_weather", "description": "Get the weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}].

gemini_delete_batch

Batch delete

gemini_delete_cached_content

Delete cached content

gemini_delete_file

Delete file

gemini_delete_interaction

Delete interaction

gemini_embed_content

Embed content

gemini_generate_content

Generate content

Send a prompt or conversation to a Gemini model and get the next response (non-streaming), with optional system instruction, multimodal inputs (inline base64 or file URIs), tools such as Google Search grounding, URL context, code execution and function calling, structured JSON output, thinking settings and a context cache. Returns candidates (content.parts with text, functionCall, inlineData or executableCode, finishReason, safetyRatings, groundingMetadata with sources), promptFeedback and usageMetadata token counts. Use for text, chat, analysis of images, audio, video and documents, and structured output. Use gemini_generate_image for a simple image request, gemini_create_interaction for server-side conversation state or agents, and gemini_create_batch for many requests at half price. Charges apply to the API key's Google Cloud project for tokens used.

Parameters

Name

Type

Required

Description

model

string

Required

Model id without the models/ prefix, for example gemini-3.8-flash. Use gemini_list_models to find models that support generateContent.

cached_content

string

Optional

Name of a context cache to reuse as the prompt prefix, in the form cachedContents/{id}. The cache must have been created for the same model.

contents

array

Optional

generation_config

object

Optional

Generation settings. Common keys: temperature, topP, topK, maxOutputTokens, stopSequences, candidateCount, seed; thinkingConfig ({"thinkingLevel": "LOW"} or {"thinkingBudget": 1024}, {"includeThoughts": true}); structured output with responseMimeType "application/json" plus responseJsonSchema (a JSON Schema) or responseSchema (OpenAPI subset); responseModalities such as ["TEXT", "IMAGE"] and imageConfig ({"aspectRatio": "16:9"}) for image models. Example: {"responseMimeType": "application/json", "responseJsonSchema": {"type": "object", "properties": {"title": {"type": "string"}}, "required": ["title"]}}.

prompt

string

Optional

Single user text turn. Shortcut for contents when you only need to send text; ignored when contents is set.

safety_settings

array

Optional

Safety thresholds, at most one per category. Each entry has a category (HARM_CATEGORY_HARASSMENT, HARM_CATEGORY_HATE_SPEECH, HARM_CATEGORY_SEXUALLY_EXPLICIT, HARM_CATEGORY_DANGEROUS_CONTENT or HARM_CATEGORY_CIVIC_INTEGRITY) and a threshold (BLOCK_LOW_AND_ABOVE, BLOCK_MEDIUM_AND_ABOVE, BLOCK_ONLY_HIGH, BLOCK_NONE or OFF). Example: [{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_ONLY_HIGH"}].

service_tier

string

Optional

Service tier: standard (default), flex for cheaper best-effort capacity, or priority for higher-priority capacity where your project has it. One of: `standard`, `flex`, `priority`.

system_instruction

string

Optional

System instruction that sets the model's role, tone or rules. Text only.

tool_config

object

Optional

Tool configuration. Use {"functionCallingConfig": {"mode": "ANY", "allowedFunctionNames": ["get_weather"]}} to force a function call, mode AUTO to let the model decide, or NONE to forbid calls.

tools

array

Optional

gemini_generate_image

Generate image

gemini_generate_video

Generate video

Generate or edit a short video with audio from a text prompt using Gemini Omni Flash on the Interactions API, optionally guided by base64 images (start frame, or first and last frame) or an existing video file to edit. Returns the interaction: id, status and steps; the model_output step holds the video as base64 data (delivery inline) or as a Files API uri to fetch later (delivery uri). Use for video generation. Set background to true for long renders and poll with gemini_get_interaction. Pass previous_interaction_id to refine a video you generated earlier. Charges apply per second of generated video to the API key's Google Cloud project.

Parameters

Name

Type

Required

Description

prompt

string

Required

Description of the video, including subject, action, camera movement and sound. For edits, describe the change.

aspect_ratio

string

Optional

16:9 landscape (default) or 9:16 portrait. One of: `16:9`, `9:16`.

background

boolean

Optional

Run in the background and return immediately with status in_progress; poll with gemini_get_interaction.

delivery

string

Optional

How the video is returned: inline puts base64 MP4 data in the response (the API default); uri returns a Files API uri instead and is needed for videos over 4 MB. The uri is downloaded with the API key and is not fetched by this tool. One of: `inline`, `uri`.

images

array

Optional

Images to animate, as base64. One image is used as a guide or start frame; two images are treated as the first and last frame. Each item is {"mime_type": "image/jpeg", "data": "<base64>"}.

model

string

Optional

Video model id. Defaults to gemini-omni-1.1-flash. Default: `gemini-omni-1.1-flash`.

previous_interaction_id

string

Optional

Id of an earlier video interaction to refine or extend in a multi-turn edit.

resolution

string

Optional

Output resolution: 360p, 720p (default), 1080p or 4k. Higher resolutions take longer. One of: `360p`, `720p`, `1080p`, `4k`.

video_uri

string

Optional

URI of a video to edit or extend, from the Files API (10 seconds or shorter for uploaded videos). Example: https://generativelanguage.googleapis.com/v1beta/files/abc-123.

gemini_get_batch

Batch get

gemini_get_cached_content

Get cached content

gemini_get_file

Get file

gemini_get_interaction

Get interaction

gemini_get_model

Get model

gemini_list_batches

List batches

gemini_list_cached_contents

List cached contents

gemini_list_files

List files

gemini_list_models

List models

gemini_update_cached_content

Update cached content

Build your Agent

Same auth pattern across LangChain, OpenAI, Anthropic, and Google ADK.

import { ScalekitClient } from "@scalekit-sdk/node";
import { createReactAgent } from "@langchain/langgraph/prebuilt";

const sk = new ScalekitClient(env.SCALEKIT_ENV_URL, env.SCALEKIT_CLIENT_ID, env.SCALEKIT_CLIENT_SECRET);

// Google Gemini tools scoped to this user
const { tools } = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["gemini"], toolNames: [
    "gemini_count_tokens",
    "gemini_generate_content",
    "gemini_embed_content"] },
  pageSize: 100,
});

const agent = createReactAgent({ llm, tools });
await agent.invoke({ messages: [{ role: "user", content: "Count the tokens in this prompt, then summarize the attached report" }] });

import OpenAI from "openai";
import { ScalekitClient } from "@scalekit-sdk/node";

const sk = new ScalekitClient(env.SCALEKIT_ENV_URL, env.SCALEKIT_CLIENT_ID, env.SCALEKIT_CLIENT_SECRET);
const openai = new OpenAI();

const { tools } = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["gemini"] }, pageSize: 100,
});

const res = await openai.chat.completions.create({
  model: "gpt-5",
  messages: [{ role: "user", content: "Count the tokens in this prompt, then summarize the attached report" }],
  tools,
});

// Execute the tool call with the user's vaulted Google Gemini API key
await sk.tools.executeTool(res.choices[0].message.tool_calls[0], "user_123");

import Anthropic from "@anthropic-ai/sdk";
import { ScalekitClient } from "@scalekit-sdk/node";

const sk = new ScalekitClient(env.SCALEKIT_ENV_URL, env.SCALEKIT_CLIENT_ID, env.SCALEKIT_CLIENT_SECRET);
const anthropic = new Anthropic();

const { tools } = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["gemini"] }, pageSize: 100,
});

const msg = await anthropic.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Count the tokens in this prompt, then summarize the attached report" }],
  tools,
});

// Tool call runs with the user's vaulted Google Gemini API key
await sk.tools.executeTool(msg.content, "user_123");

import { Agent } from "@google/adk/agents";
import { ScalekitClient } from "@scalekit-sdk/node";

const sk = new ScalekitClient(env.SCALEKIT_ENV_URL, env.SCALEKIT_CLIENT_ID, env.SCALEKIT_CLIENT_SECRET);

const { tools } = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["gemini"] }, pageSize: 100,
});

const agent = new Agent({
  name: "llm_ops_agent",
  model: "gemini-2.5-pro",
  instruction: "Google Gemini tools scoped to this user",
  tools,
});

await agent.run("Count the tokens in this prompt, then summarize the attached report");

Try these prompts

Copy any prompt into your agent. Each maps directly to a Google Gemini tool. Click to copy, paste into your agent, done.

Generate

Copy the prompt

Copied

Summarize this uploaded PDF in 5 bullet points.

Copy the prompt

Copied

Generate a product image of a blue backpack on a mountain trail.

Copy the prompt

Copied

Generate a short video of a paper plane flying over a city.

Embeddings and tokens

Copy the prompt

Copied

Embed these 50 FAQ answers for our search index.

Copy the prompt

Copied

Count the tokens in this prompt before I send it.

Copy the prompt

Copied

List the Gemini models my API key can use.

Batches and caches

Copy the prompt

Copied

Create a batch job for these 1,000 classification prompts.

Copy the prompt

Copied

Cache this 300-page manual so later questions reuse it.

Copy the prompt

Copied

Run a Deep Research interaction on the EV charging market.

SEE HOW AUTH WORKS

Each user connects their own Google Gemini API key once, and Scalekit sends it with every call. Keys stay vaulted, every call is scope checked, and every action is logged.

Authorize

Your user connects

Google Gemini

once. We tie it to their identity and the meetings they approved — no shared bot account, no org-wide access

Who:

user ‘A’

when:

Once per user

access:

Limited to user

Store

Their

Google Gemini

token lives in a vault scoped to them. User A's meetings are never reachable by an agent acting for user B, even on the same connection

vault:

encrypted

scope:

per-user

tokens:

auto-refreshed

Resolve

When your agent calls a

Google Gemini

tool, we fetch the right token server-side. It never touches your agent, never appears in the LLM context, never shows up in your logs

speed:

~40ms

check:

before every call

seen by:

nobody

Audit

Every

Google Gemini

tool call is logged — who triggered it, which meeting was fetched, what came back. 90 days of history, tied to the user who authorized it

history:

90 days

export:

SIEM-ready

logged:

every call

Test other agents

See the same per-user auth pattern across other AI and media connectors.

GTM and RevOps Teams

Competitive intelligence briefing agent

Scans Gong calls for competitor mentions, matches each one to its Notion battlecard, and DMs every affected rep a single Slack digest per cycle. Every call runs as the PMM who owns the briefing, never a shared bot.

Support and Ops Teams

Meeting prep

Pulls agenda, participant context, and open action items before every meeting.

Why Scalekit

Secure your agent's access. Connectors ship in minutes

01.

Shared tokens break per-user analytics

A shared Google Gemini API key looks fine in a demo. In production every generation request looks like one service account, and you cannot tell which user triggered it. Scalekit resolves the credential of the actual user who triggered the agent, never a shared bot.

// shared key
audit → bot_service_account

// scalekit
audit → user_abc ✓

02.

Authentication is not authorization

03.

Multi-tenancy is architectural

04.

Google Gemini today. Ten connectors tomorrow.

“Our agents act across Salesforce, Gong, Google Drive, and more, on behalf of every customer. Scalekit behind the scenes meant we can keep adding tools without ever rebuilding how credentials or tool calling work.”

Venu Madhav Kattagoni

Head of Engineering / Von

FAQs

Frequently Asked Questions

Does the agent access Google Gemini as the user or through a shared API key?

As the user. Each user connects their own Google Gemini API key once, and Scalekit sends it with every call. Audit logs attribute every generation and batch to that user, not a shared service account.

Where is the Google Gemini API key stored?

In Scalekit's managed AES-256 token vault, namespaced per tenant. To rotate it, the user reconnects with a new API key. Revocation is a single dashboard action. Credentials never appear in prompts, logs, or LLM context.

Can I limit what the agent does in Google Gemini?

Yes. Filter by tool name in listScopedTools to expose only what you want, for example content generation and token counting without gemini_delete_file, gemini_delete_cached_content, or gemini_delete_batch. Scalekit also enforces scope checks before every API call.

What happens when a user revokes Google Gemini access?

The connection is invalidated on the next tool call. Subsequent requests for that user fail closed with a clear error. Other users in the tenant remain unaffected. The event is logged for audit.

How long do uploaded files last?

Files in the Gemini Files API are kept for 48 hours. gemini_list_files and gemini_get_file show what is stored for the API key's project, and gemini_delete_file removes a file before it expires.