Google Cloud Vision Connector for AI Agents

Tools your vision agent reaches for on Google Cloud Vision, scoped per user.

CALL ANY TOOL

Annotate images, run OCR, and manage Product Search catalogs in Google Cloud Vision with the signed-in user's OAuth 2.0 token. <a href="https://docs.scalekit.com/agentkit/connectors/googlecloudvision/">View Docs</a>

googlecloudvision_files_annotate

Files annotate

Run Vision API feature detectors: primarily OCR/document text detection: against a single multi-page file (PDF, TIFF, or GIF) in one synchronous call, optionally scoped to up to 5 of its pages. Returns a responses array with exactly one entry (the API's BatchAnnotateFilesRequest currently accepts only one AnnotateFileRequest per call, despite the plural name); that entry holds per-page detection results (a nested responses array, one entry per processed page) or an error object if the file failed. Use files_annotate for one file that needs an inline, synchronous result. Use files_async_batch_annotate instead for multiple files, large files, or long-running jobs whose input/output live in Cloud Storage. Use images_annotate instead for standalone raster images (JPEG, PNG, etc.) rather than multi-page documents.

Parameters

Name

Type

Required

Description

requests

array

Required

Array holding exactly one per-file annotation request (BatchAnnotateFilesRequest currently supports only one AnnotateFileRequest per synchronous call, even though this field is an array). The object must include an "inputConfig" field shaped as either {"content": "<base64-encoded file bytes>", "mimeType": "application/pdf"} or {"gcsSource": {"uri": "gs://bucket/file.pdf"}, "mimeType": "application/pdf"}, and a "features" array such as [{"type": "DOCUMENT_TEXT_DETECTION"}]. May optionally include a "pages" array of 1-indexed page numbers (at most 5 pages, or the last 5 pages using negative numbers) to limit which pages are processed; omit to process the file's first 5 pages by default. Example: [{"inputConfig": {"gcsSource": {"uri": "gs://my-bucket/doc.pdf"}, "mimeType": "application/pdf"}, "features": [{"type": "DOCUMENT_TEXT_DETECTION"}], "pages": [1, 2, 3]}]

labels

object

Optional

Optional user-defined metadata labels attached to this request, as a flat object of string key-value pairs. Keys/values are limited to 63 characters, lowercase letters, numbers, underscores, and dashes; keys must start with a letter.

parent

string

Optional

Optional target project and location to route this call to, in the form "projects/{project-id}/locations/{location-id}". If omitted, Vision chooses a region automatically. Supported location IDs: "us", "asia", "eu".

googlecloudvision_files_async_batch_annotate

Files async batch annotate

Start an asynchronous, Cloud-Storage-backed batch file annotation job (typically OCR on multi-page PDFs or TIFFs) for files that already live in Cloud Storage. The job runs in the background and writes its results as JSON files to a Cloud Storage destination. Returns immediately with a longrunning Operation object: {"name": "operations/...", "done": false, ...}: not the annotation results themselves. Use files_async_batch_annotate for GCS-hosted files needing async OCR at scale. Use files_annotate instead for a handful of files sent inline that need an immediate, synchronous response. Unlike images_async_batch_annotate, outputConfig here is set per request inside each requests[] entry rather than once at the call level, and inputConfig only accepts gcsSource: inline content bytes are not supported on this async file path. Unlike files_annotate, page scoping is not supported here: every page of the file is always processed; do not send a "pages" field. Files must already reside in Cloud Storage before calling this tool. After starting the job, poll the returned operation name with operations_get until done is true, or list in-flight jobs with operations_list.

Parameters

Name

Type

Required

Description

requests

array

Required

Array of per-file async annotation requests. Each object must include an "inputConfig" shaped as {"gcsSource": {"uri": "gs://bucket/file.pdf"}, "mimeType": "application/pdf"}: inline base64 "content" is NOT supported here, only gcsSource: a "features" array such as [{"type": "DOCUMENT_TEXT_DETECTION"}], and: unlike images_async_batch_annotate: its own "outputConfig" field ({"gcsDestination": {"uri": "gs://bucket/prefix"}, "batchSize": 20}) nested inside this same object rather than set once for the whole call. Do NOT include a "pages" field here: unlike the synchronous files_annotate, this async request type does not support page scoping and Google rejects it with a 400 Unknown-field error; the whole file is always processed. Example: [{"inputConfig": {"gcsSource": {"uri": "gs://my-bucket/doc.pdf"}, "mimeType": "application/pdf"}, "features": [{"type": "DOCUMENT_TEXT_DETECTION"}], "outputConfig": {"gcsDestination": {"uri": "gs://my-bucket/vision-output/"}, "batchSize": 20}}]

labels

object

Optional

parent

string

Optional

googlecloudvision_images_annotate

Images annotate

googlecloudvision_images_async_batch_annotate

Images async batch annotate

Start an asynchronous, Cloud-Storage-backed batch image annotation job for image sets too large or slow for the synchronous images_annotate call. The job runs in the background and writes its results as JSON files to a Cloud Storage destination shared by the whole batch. Returns immediately with a longrunning Operation object: {"name": "operations/...", "done": false, ...}: not the annotation results themselves. Use images_async_batch_annotate for large batches or when output should land in Cloud Storage. Use images_annotate instead for a handful of images that need an inline, synchronous response. After starting the job, poll the returned operation name with operations_get until done is true, or list in-flight jobs with operations_list.

Parameters

Name

Type

Required

Description

output_config

object

Required

Call-level output destination applied to the entire batch (not set per-request). Must include gcsDestination.uri pointing to a Cloud Storage prefix, and may include batchSize (number of responses written per output JSON file; default 20 if omitted). Example: {"gcsDestination": {"uri": "gs://my-bucket/vision-output/"}, "batchSize": 20}

requests

array

Required

Array of per-image annotation requests for this batch. Each object must include an "image" field with either {"content": "<base64-encoded image bytes>"} or {"source": {"imageUri": "gs://bucket/file.jpg"}}, and a "features" array such as [{"type": "LABEL_DETECTION", "maxResults": 10}]. Do not include outputConfig inside these entries: it is a single call-level field set once via output_config, applying to the whole batch. Example: [{"image": {"source": {"imageUri": "gs://my-bucket/photo.jpg"}}, "features": [{"type": "LABEL_DETECTION"}]}]

labels

object

Optional

parent

string

Optional

googlecloudvision_operations_cancel

Cancel operations

googlecloudvision_operations_delete

Delete operations

googlecloudvision_operations_get

Get operations

googlecloudvision_operations_list

List operations

List Google Cloud Vision long-running operations, optionally filtered by an AIP-160 filter expression. Returns operations (an array of Operation objects with name, metadata, and done) and a nextPageToken for fetching further pages when more results exist. The name field must be exactly the literal string "operations": Vision API's ListOperations binding only accepts this one value (confirmed by the API's own published parameter pattern, `^operations$`); it does not support listing scoped by project or location, even though individual operation names returned by other calls may be project- and/or location-prefixed. To find one specific operation whose name you don't have, list here and match on the returned name/metadata; once you have the name, use operations_get to poll it directly (operations_get accepts the name in whatever form it was returned, prefixed or not). Use operations_list to browse in-flight or past operations, e.g. to find one you didn't keep the name of. Use operations_get instead once you already have a specific operation name to poll its status.

Parameters

Name

Type

Required

Description

filter

string

Optional

Optional AIP-160 filter expression to restrict which operations are returned (e.g. done=false to see only in-flight jobs). Leave blank to list all operations in the collection.

name

string

Optional

Must be exactly the literal string "operations": this is the only value Vision API's ListOperations endpoint accepts (enforced server-side; the API does not support listing scoped by project or location). Leave this at the default. Default: `operations`.

page_size

integer

Optional

Maximum number of operations to return per page. If unset, the server chooses a default page size.

page_token

string

Optional

Pagination token from a previous operations_list call's nextPageToken, used to fetch the next page of results.

return_partial_success

boolean

Optional

When true, reachable operations are returned normally and unreachable ones are reported in the response's unreachable field instead of causing an error. Only meaningful when listing across collections; not documented as supported for Vision API's single flat operations collection, so leave blank unless you have a specific reason to set it.

googlecloudvision_product_create

Create product

googlecloudvision_product_delete

Delete product

googlecloudvision_product_get

Get product

googlecloudvision_product_list

List product

googlecloudvision_product_purge

Product purge

googlecloudvision_product_set_add_product

Product set add product

googlecloudvision_product_set_create

Create product set

googlecloudvision_product_set_delete

Delete product set

googlecloudvision_product_set_get

Get product set

googlecloudvision_product_set_import

Import product set

googlecloudvision_product_set_list

List product set

googlecloudvision_product_set_products_list

List product set products

googlecloudvision_product_set_remove_product

Product set remove product

googlecloudvision_product_set_update

Update product set

googlecloudvision_product_update

Update product

googlecloudvision_reference_image_create

Create reference image

googlecloudvision_reference_image_delete

Delete reference image

googlecloudvision_reference_image_get

Get reference image

googlecloudvision_reference_image_list

List reference image

Build your Agent

Same OAuth 2.0 vault and scoped tool list across LangChain, OpenAI, Anthropic, and Google ADK.

import { ScalekitClient } from "@scalekit-sdk/node";
import { DynamicStructuredTool } from "@langchain/core/tools";
const sk = new ScalekitClient(
  process.env.SCALEKIT_ENVIRONMENT_URL,
  process.env.SCALEKIT_CLIENT_ID,
  process.env.SCALEKIT_CLIENT_SECRET,
);
const page = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["googlecloudvision"], toolNames: [
    "googlecloudvision_images_annotate",
    "googlecloudvision_files_annotate",
    "googlecloudvision_operations_list",
  ]}, pageSize: 100,
});

import OpenAI from "openai";
import { ScalekitClient } from "@scalekit-sdk/node";
const sk = new ScalekitClient(
  process.env.SCALEKIT_ENVIRONMENT_URL,
  process.env.SCALEKIT_CLIENT_ID,
  process.env.SCALEKIT_CLIENT_SECRET,
);
const page = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["googlecloudvision"], toolNames: [
    "googlecloudvision_images_annotate",
    "googlecloudvision_files_annotate",
    "googlecloudvision_operations_list",
  ]}, pageSize: 100,
});
const openai = new OpenAI();

import Anthropic from "@anthropic-ai/sdk";
import { ScalekitClient } from "@scalekit-sdk/node";
const sk = new ScalekitClient(
  process.env.SCALEKIT_ENVIRONMENT_URL,
  process.env.SCALEKIT_CLIENT_ID,
  process.env.SCALEKIT_CLIENT_SECRET,
);
const page = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["googlecloudvision"], toolNames: [
    "googlecloudvision_images_annotate",
    "googlecloudvision_files_annotate",
    "googlecloudvision_operations_list",
  ]}, pageSize: 100,
});
const anthropic = new Anthropic();

import { ScalekitClient } from "@scalekit-sdk/node";
const sk = new ScalekitClient(
  process.env.SCALEKIT_ENVIRONMENT_URL,
  process.env.SCALEKIT_CLIENT_ID,
  process.env.SCALEKIT_CLIENT_SECRET,
);
const page = await sk.tools.listScopedTools("user_123", {
  filter: { connectionNames: ["googlecloudvision"], toolNames: [
    "googlecloudvision_images_annotate",
    "googlecloudvision_files_annotate",
    "googlecloudvision_operations_list",
  ]}, pageSize: 100,
});

Try these prompts

Copy a prompt into your agent. Each one maps to Google Cloud Vision image insights for the signed-in user.

OCR and documents

Copy the prompt

Copied

Extract the invoice number and due date from gs://acme-docs/invoice.pdf pages 1 to 3.

Copy the prompt

Copied

What text is on the first page of this TIFF scan?

Copy the prompt

Copied

Summarize every heading in the uploaded PDF using document text detection.

Image insights

Copy the prompt

Copied

Label the objects in gs://acme-photos/shelf.jpg and list any logos.

Copy the prompt

Copied

Is this product photo safe to publish? Run safe search and label detection.

Copy the prompt

Copied

Read the text on this storefront photo and name the landmarks if any.

Product Search catalogs

Copy the prompt

Copied

List product sets in my us location and how many products each has.

Copy the prompt

Copied

Show reference images on the Acme sneaker product.

Copy the prompt

Copied

Add gs://acme-catalog/sneaker-side.jpg as a reference image on that product.

SEE HOW AUTH WORKS

Your users connect once. Their Google Cloud Vision OAuth 2.0 tokens stay vaulted, every call is checked, and every action is logged.

Authorize

Your user connects

Google Cloud Vision

once. We tie it to their identity and the meetings they approved — no shared bot account, no org-wide access

Who:

user ‘A’

when:

Once per user

access:

Limited to user

Store

Their

Google Cloud Vision

token lives in a vault scoped to them. User A's meetings are never reachable by an agent acting for user B, even on the same connection

vault:

encrypted

scope:

per-user

tokens:

auto-refreshed

Resolve

When your agent calls a

Google Cloud Vision

tool, we fetch the right token server-side. It never touches your agent, never appears in the LLM context, never shows up in your logs

speed:

~40ms

check:

before every call

seen by:

nobody

Audit

Every

Google Cloud Vision

tool call is logged — who triggered it, which meeting was fetched, what came back. 90 days of history, tied to the user who authorized it

history:

90 days

export:

SIEM-ready

logged:

every call

Test other agents

Reuse the same per-user OAuth 2.0 vault and audit chain on other Google and document connectors.

GTM and RevOps Teams

Competitive intelligence briefing agent

Scans Gong calls for competitor mentions, matches each one to its Notion battlecard, and DMs every affected rep a single Slack digest per cycle. Every call runs as the PMM who owns the briefing, never a shared bot.

GTM and RevOps Teams

Outbound prospecting agent

Searches Apollo for prospects matching your ICP, scores and ranks them, drafts personalized outreach in Gmail, and logs every send to Google Sheets. Mail goes out as the rep, not from a shared inbox.

GTM and RevOps Teams

Deal intelligence agent

Pulls recent Gong calls, scores deal risk with an LLM, cross-references the record in Attio, and DMs each owner their at-risk deals in Slack. Every read is scoped to that rep's own access.

Support and Ops Teams

Support triage agent

Fetches new Zendesk tickets, classifies them by type and urgency, searches the Notion knowledge base for an answer, and routes what it cannot resolve to Slack. Every call runs on the support agent's own delegated OAuth.

Why Scalekit

Secure your agent's access. Connectors ship in minutes

01.

Shared tokens break per-user analytics

A shared Google Cloud Vision service account makes every OCR and Product Search call look like a bot. Scalekit resolves the credential of the actual user who triggered the agent, never a shared bot.

// shared key
audit → bot_acct

// scalekit
audit → user_abc ✓

02.

Authentication is not authorization

03.

Multi-tenancy is architectural

04.

Google Cloud Vision today. Others tomorrow.

“Our agents act across Salesforce, Gong, Google Drive, and more, on behalf of every customer. Scalekit behind the scenes meant we can keep adding tools without ever rebuilding how credentials or tool calling work.”

Venu Madhav Kattagoni

Head of Engineering / Von

FAQs

Frequently Asked Questions

Does the agent access Google Cloud Vision as the user or a shared key?

Per-user resolution. Each tool call uses the signed-in user's Google Cloud Vision OAuth 2.0 token, so image insights and Product Search calls attribute to that user in the audit log.

Where is the Google Cloud Vision token stored?

In Scalekit's AES-256 vault, namespaced per tenant. Tokens refresh automatically. You revoke access from one dashboard without rotating a shared secret across every agent.

Can I limit what the agent does in Google Cloud Vision?

Yes. Filter tool names in listScopedTools (for example googlecloudvision_images_annotate only) and Scalekit checks that scope before the API call.

What happens when a user revokes access?

The next Google Cloud Vision tool call fails closed for that user. Other users and other connectors are unaffected. The revoke is logged.

Does the agent see every image in the Google Cloud project?

Only the images, product sets, and GCS objects the authorized user's token can already read. Scalekit does not broaden that access; it vaults and scopes the existing OAuth grant.