ChatBedrockConverse

Amazon Bedrock Converse is a fully managed service that makes Foundation Models (FMs) from leading AI startups and Amazon available via an API. You can choose from a wide range of FMs to find the model that is best suited for your use case. It provides a unified conversational interface for Bedrock models, but does not yet have feature parity for all functionality within the older Bedrock model service.

This will help you getting started with Amazon Bedrock Converse chat models. For detailed documentation of all ChatBedrockConverse features and configurations head to the API reference.

Overview

Integration details

Class	Package	Local	Serializable	PY support	Package downloads	Package latest
ChatBedrockConverse	@langchain/aws	❌	✅	✅

Model features

See the links in the table headers below for guides on how to use specific features.

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Token usage	Logprobs
✅	✅	❌	✅	❌	❌	✅	✅	❌

Setup

To access Bedrock models you’ll need to create an AWS account, set up the Bedrock API service, get an access key ID and secret key, and install the @langchain/community integration package.

Credentials

Head to the AWS docs to sign up for AWS and setup your credentials. You’ll also need to turn on model access for your account, which you can do by following these instructions.

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGCHAIN_TRACING_V2="true"
# export LANGCHAIN_API_KEY="your-api-key"

Installation

The LangChain ChatBedrockConverse integration lives in the @langchain/aws package:

tip

See this section for general instructions on installing integration packages.

npm

npm i @langchain/aws @langchain/core

yarn

yarn add @langchain/aws @langchain/core

pnpm

pnpm add @langchain/aws @langchain/core

Instantiation

Now we can instantiate our model object and generate chat completions.

There are a few different ways to authenticate with AWS - the below examples rely on an access key, secret access key and region set in your environment variables:

import { ChatBedrockConverse } from "@langchain/aws";

const llm = new ChatBedrockConverse({
  model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
  region: process.env.BEDROCK_AWS_REGION,
  credentials: {
    accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
    secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
  },
});

Invocation

const aiMsg = await llm.invoke([
  [
    "system",
    "You are a helpful assistant that translates English to French. Translate the user sentence.",
  ],
  ["human", "I love programming."],
]);
aiMsg;

AIMessage {
  "id": "f5dc5791-224e-4fe5-ba2e-4cc51d9e7795",
  "content": "J'adore la programmation.",
  "additional_kwargs": {},
  "response_metadata": {
    "$metadata": {
      "httpStatusCode": 200,
      "requestId": "f5dc5791-224e-4fe5-ba2e-4cc51d9e7795",
      "attempts": 1,
      "totalRetryDelay": 0
    },
    "metrics": {
      "latencyMs": 584
    },
    "stopReason": "end_turn",
    "usage": {
      "inputTokens": 29,
      "outputTokens": 11,
      "totalTokens": 40
    }
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 29,
    "output_tokens": 11,
    "total_tokens": 40
  }
}

console.log(aiMsg.content);

J'adore la programmation.

Chaining

We can chain our model with a prompt template like so:

import { ChatPromptTemplate } from "@langchain/core/prompts";

const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "You are a helpful assistant that translates {input_language} to {output_language}.",
  ],
  ["human", "{input}"],
]);

const chain = prompt.pipe(llm);
await chain.invoke({
  input_language: "English",
  output_language: "German",
  input: "I love programming.",
});

AIMessage {
  "id": "c6401e11-8f85-4a71-8e15-4856d55aef78",
  "content": "Here's the German translation:\n\nIch liebe Programmieren.",
  "additional_kwargs": {},
  "response_metadata": {
    "$metadata": {
      "httpStatusCode": 200,
      "requestId": "c6401e11-8f85-4a71-8e15-4856d55aef78",
      "attempts": 1,
      "totalRetryDelay": 0
    },
    "metrics": {
      "latencyMs": 760
    },
    "stopReason": "end_turn",
    "usage": {
      "inputTokens": 23,
      "outputTokens": 18,
      "totalTokens": 41
    }
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 23,
    "output_tokens": 18,
    "total_tokens": 41
  }
}

Tool calling

Tool calling with Bedrock models works in a similar way to other models, but note that not all Bedrock models support tool calling. Please refer to the AWS model documentation for more information.

API reference

For detailed documentation of all ChatBedrockConverse features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_aws.ChatBedrockConverse.html

Related

• Chat model conceptual guide
• Chat model how-to guides