Generative AI for Scholarship

Harvard Data Science Initiative (HDSI) & Faculty of Arts and Sciences (FAS)

Setting Up Jupyter AI with the Harvard HUIT Bedrock Endpoint (Windows)

This guide walks through configuring Jupyter AI (the JupyterLab AI chat assistant) to route queries through Harvard's HUIT Bedrock proxy on Windows.

1 Create Your Working Directory

Open Terminal from the Start menu and create a directory for this work:

mkdir "$env:USERPROFILE\Desktop\GAI\exercises"
cd "$env:USERPROFILE\Desktop\GAI\exercises"

You will run all subsequent commands from this directory.

2 Verify Your Python Version

Ensure Python 3.10 or later is installed:

python --version
# Expected: Python 3.13.x (or 3.10+)

If Python is not installed or the version is too old, download it from python.org.

3 Create a Virtual Environment

Jupyter AI v3 is beta software that can pull in dependency versions incompatible with your existing Python packages. A virtual environment keeps everything isolated so nothing else on your system is affected.

python -m venv jupyter-ai-env

Activate it:

.\jupyter-ai-env\Scripts\Activate.ps1

Your terminal prompt should now show (jupyter-ai-env) at the beginning, confirming the virtual environment is active.

cd "$env:USERPROFILE\Desktop\GAI\exercises"
.\jupyter-ai-env\Scripts\Activate.ps1

4 Install JupyterLab

With the virtual environment active, install JupyterLab:

pip install notebook jupyterlab

Verify the installation and launch JupyterLab to confirm it works:

jupyter lab

Close it (Ctrl+C in the terminal) once you've confirmed it launches, then continue to the next step.

5 Install Jupyter AI (v3 Beta)

The HUIT proxy speaks the AWS Bedrock Converse API format. Jupyter AI v3 uses LiteLLM as its backend, which has native support for Bedrock endpoints with Bearer token authentication. This is why v3 (beta) is required instead of the stable v2 release.

pip install "jupyter-ai==3.0.0b9" boto3

Verify:

pip show jupyter-ai
# Expected: Version: 3.0.0b9

6 Configure Environment Variables

Set the following environment variables in your terminal. Replace your-huit-api-key-here with the API key from the link at the top of this page.

$env:ANTHROPIC_API_KEY = "your-huit-api-key-here"
$env:ANTHROPIC_BEDROCK_BASE_URL = "https://apis.huit.harvard.edu/ais-bedrock-llm/v2"
$env:AWS_BEARER_TOKEN_BEDROCK = "your-huit-api-key-here"

Verify they are set:

echo $env:AWS_BEARER_TOKEN_BEDROCK
echo $env:ANTHROPIC_API_KEY
# Both should print your API key

7 Launch JupyterLab

With the environment variables set and the virtual environment active, launch JupyterLab:

jupyter lab

If you opened a new terminal, you need to set the environment variables and activate the venv again first:

cd "$env:USERPROFILE\Desktop\GAI\exercises"
.\jupyter-ai-env\Scripts\Activate.ps1
$env:ANTHROPIC_API_KEY = "your-huit-api-key-here"
$env:ANTHROPIC_BEDROCK_BASE_URL = "https://apis.huit.harvard.edu/ais-bedrock-llm/v2"
$env:AWS_BEARER_TOKEN_BEDROCK = "your-huit-api-key-here"
jupyter lab

8 Configure Jupyternaut Settings

In the JupyterLab interface, open the Jupyternaut (AI assistant) settings panel.

8a. Find Settings

Look for the Jupyternaut settings. In v3, this may be accessible via:

• The menu bar: Settings → AI Settings
• The Jupyternaut icon in the left sidebar → settings gear

8b. Set the Chat Model ID

In the Chat model ID field, enter:

bedrock/converse/us.anthropic.claude-opus-4-5-20251101-v1:0

8c. Set Model Parameters

In the Model parameters section, add:

8d. Verify API Key

In the Secrets and API keys section, confirm that AWS_BEARER_TOKEN_BEDROCK appears as a static (read-only) secret imported from your environment. If it doesn't appear, add it using the "Add secret" button.

8e. Embedding Model

Leave the Embedding model blank. It is only needed for the /learn document indexing feature and is not required for chat.

8f. Save and Test

Click Update chat model, then send a test message in the Jupyternaut chat:

Hello, can you tell me what model you are?

Available Models

Change the model ID in Jupyternaut settings to use different models:

Troubleshooting

"Unable to locate credentials"

LiteLLM is trying AWS SigV4 auth instead of Bearer token auth. Ensure AWS_BEARER_TOKEN_BEDROCK is set in the same terminal session where you launched JupyterLab.

"Invalid ApiKey" (oauth.v2.InvalidApiKey)

Verify that AWS_BEARER_TOKEN_BEDROCK is set to the correct HUIT API key (same value as ANTHROPIC_API_KEY).

Script execution policy error when activating venv

Run this once to allow scripts for your user account:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Type Y and press Enter to confirm.

No response / silent failure

Check the terminal where you launched jupyter lab for error tracebacks. Common causes:

• Wrong model ID format (must start with bedrock/converse/)
• Missing api_base model parameter in Jupyternaut settings
• Environment variables not set in the current terminal session

Making Environment Variables Permanent (Optional)

To avoid re-entering environment variables every time you open a terminal, you can add them to your PowerShell profile:

# Create the profile file if it doesn't exist
if (!(Test-Path -Path $PROFILE)) { New-Item -Type File -Path $PROFILE -Force }

# Open it in Notepad
notepad $PROFILE

Add these lines to the profile file (with your actual API key), save, and close:

$env:ANTHROPIC_API_KEY = "your-huit-api-key-here"
$env:ANTHROPIC_BEDROCK_BASE_URL = "https://apis.huit.harvard.edu/ais-bedrock-llm/v2"
$env:AWS_BEARER_TOKEN_BEDROCK = "your-huit-api-key-here"

The variables will be set automatically every time you open a terminal.