Generative AI for Scholarship

Harvard Data Science Initiative (HDSI) & Faculty of Arts and Sciences (FAS)

Setting Up Jupyter AI with the Harvard HUIT Bedrock Endpoint (Mac)

This guide walks through configuring Jupyter AI (the JupyterLab AI chat assistant) to route queries through Harvard's HUIT Bedrock proxy. This gives you an interactive AI assistant directly inside JupyterLab on your local machine.

1 Create Your Working Directory

Open Terminal and create a directory for this work:

mkdir -p ~/Desktop/GAI/exercises
cd ~/Desktop/GAI/exercises

You will run all subsequent commands from this directory.

2 Verify Your Python Version

Ensure you are running Python 3.13 (or at least 3.10+). The system Python 3.9 from Xcode Command Line Tools is too old and causes package conflicts.

python3 --version
# Expected: Python 3.13.x

which python3
# Expected: /usr/local/bin/python3 (not /usr/bin/python3)

If you need to install Python 3.13, download it from python.org.

3 Create a Virtual Environment

Jupyter AI v3 is beta software that can pull in dependency versions incompatible with your existing Python packages. A virtual environment keeps everything isolated so nothing else on your system is affected.

python3 -m venv jupyter-ai-env

Activate it:

source jupyter-ai-env/bin/activate

Your terminal prompt should now show (jupyter-ai-env) at the beginning, confirming the virtual environment is active.

cd ~/Desktop/GAI/exercises
source jupyter-ai-env/bin/activate

4 Install JupyterLab

With the virtual environment active, install JupyterLab:

pip install notebook jupyterlab

Verify the installation and launch JupyterLab to confirm it works:

jupyter lab

Close it (Ctrl+C in the terminal) once you've confirmed it launches, then continue to the next step.

5 Install Jupyter AI (v3 Beta)

The HUIT proxy speaks the AWS Bedrock Converse API format. Jupyter AI v3 uses LiteLLM as its backend, which has native support for Bedrock endpoints with Bearer token authentication. This is why v3 (beta) is required instead of the stable v2 release.

pip install "jupyter-ai==3.0.0b9" boto3

Verify:

pip show jupyter-ai
# Expected: Version: 3.0.0b9

6 Create an Environment Setup Script

We'll create a small shell script that sets the required environment variables. This keeps everything in your working directory rather than modifying your system-wide shell configuration.

Open the file in nano:

nano set_env.sh

Paste the following contents (replace your-huit-api-key-here with your actual API key):

#!/bin/zsh
# ---- HUIT Bedrock Proxy Configuration ----
# Source this script before launching JupyterLab:
#   source ./set_env.sh

# API key for the HUIT Bedrock proxy
export ANTHROPIC_API_KEY="your-huit-api-key-here"

# Base URL for the HUIT Bedrock proxy
export ANTHROPIC_BEDROCK_BASE_URL=https://apis.huit.harvard.edu/ais-bedrock-llm/v2

# Required for Jupyter AI / LiteLLM
# LiteLLM uses this for Bearer token auth to Bedrock endpoints
# (use the same value as ANTHROPIC_API_KEY)
export AWS_BEARER_TOKEN_BEDROCK="$ANTHROPIC_API_KEY"

Save and exit nano (Ctrl+O, Enter, Ctrl+X). Then source the script:

source ./set_env.sh

Verify the key variables are set:

echo $AWS_BEARER_TOKEN_BEDROCK
echo $ANTHROPIC_API_KEY
# Both should print your API key

7 Launch JupyterLab

From your working directory, activate the virtual environment, source the environment script, and launch:

cd ~/Desktop/GAI/exercises
source jupyter-ai-env/bin/activate
source ./set_env.sh
jupyter lab

8 Configure Jupyternaut Settings

In the JupyterLab interface, open the Jupyternaut (AI assistant) settings panel.

8a. Find Settings

Open the Jupyternaut settings from the menu bar: Settings → Jupyternaut Settings.

Your settings should look like this when properly configured:

8b. Set the Chat Model ID

In the Chat model ID field, enter:

bedrock/converse/us.anthropic.claude-opus-4-5-20251101-v1:0

8c. Set Model Parameters

In the Model parameters section, add:

8d. Verify API Key

In the Secrets and API keys section, confirm that AWS_BEARER_TOKEN_BEDROCK appears as a static (read-only) secret imported from your environment. If it doesn't appear, add it using the "Add secret" button.

8e. Embedding Model

Leave the Embedding model blank. It is only needed for the /learn document indexing feature and is not required for chat.

8f. Save and Test

Click Update chat model, then send a test message in the Jupyternaut chat:

Hello, can you tell me what model you are?

Available Models

Change the model ID in Jupyternaut settings to use different Claude models:

Troubleshooting

"Unable to locate credentials"

LiteLLM is trying AWS SigV4 auth instead of Bearer token auth. Ensure AWS_BEARER_TOKEN_BEDROCK is set. Run source ./set_env.sh before launching JupyterLab.

"Invalid ApiKey" (oauth.v2.InvalidApiKey)

Verify that AWS_BEARER_TOKEN_BEDROCK is set to the correct HUIT API key (same value as ANTHROPIC_API_KEY).

"bedrock does not support parameters: ['tools']"

This should not occur with bedrock/converse/ (which supports tools). If it does, add to set_env.sh and re-source it:

export LITELLM_DROP_PARAMS=true

No response / silent failure

Check the terminal where you launched jupyter lab for error tracebacks. Common causes:

• Wrong model ID format (must start with bedrock/converse/)
• Missing api_base model parameter in Jupyternaut settings
• Stale environment variables (open a new terminal)

"No generation chunks were returned"

You may be using the wrong API format. The HUIT proxy uses Bedrock Converse API, not the native Anthropic Messages API. Do not use the anthropic-chat provider.

Summary: Packages Installed

Summary: Environment Variables