QueryGateway

Source: src/Cloudflare/AI/QueryGateway.ts

Binding service that turns a {@link Gateway} resource into a typed {@link QueryGatewayClient} for Worker runtime code. Wraps the Cloudflare.AI. Gateway runtime binding so each operation returns an Effect tagged with {@link GatewayError}, exposes the raw Workers AI handle for ai.run(...), and provides a model(options) factory that produces an effect/unstable/ai LanguageModel Layer.

Bind a {@link Gateway} to a Worker and obtain the Effect-native AI Gateway client (run, getUrl, model, …).

QueryGateway is a single identifier that is simultaneously the binding’s Context tag, its type, and the callable — yield* Cloudflare.AI.QueryGateway(gateway).

Calling AI Gateway

Bind the gateway during the Worker’s init phase, then use run or getUrl from request handlers.

const aiGateway = yield* Cloudflare.AI.QueryGateway(gateway);

return {
  fetch: Effect.gen(function* () {
    return yield* aiGateway.run({
      provider: "workers-ai",
      endpoint: "@cf/meta/llama-3.1-8b-instruct",
      headers: { "content-type": "application/json" },
      query: { prompt: "Write a concise status update" },
    });
  }),
};

Driving Effect AI through the gateway

model(options) produces a Layer<LanguageModel, never, RuntimeContext> that translates LanguageModel.generateText / streamText calls (including tool calls and structured outputs) into ai.run(...) against the bound Workers AI model, routed through the gateway.

const aiGateway = yield* Cloudflare.AI.QueryGateway(gateway);

const languageModel = aiGateway.model({
  model: "@cf/meta/llama-3.1-8b-instruct",
  parameters: { temperature: 0.7, maxTokens: 1024 },
});

const response = yield* LanguageModel.generateText({ prompt }).pipe(
  Effect.provide(languageModel),
);

Provide {@link QueryGatewayBinding} in the worker’s runtime layer to resolve the underlying Cloudflare.AI. binding at request time.