usage() raises a TypeError with the AmazonBedrock client

[dipetkov]

I'm trying to use claudette with Claude 3.5 Sonnet via Bedrock and I run into this error:

chat("Which football team is known as the Lionesses?")
> TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Here's a reprex to reproduce the error.

import boto3
from anthropic import AnthropicBedrock
from claudette import Chat, Client

session = boto3.Session(profile_name="claudette")
credentials = session.get_credentials()

# Claude 3.5 Sonnet on Bedrock with cross-region inference
model = "us.anthropic.claude-3-5-sonnet-20241022-v2:0"

ab = AnthropicBedrock(
    aws_access_key=credentials.access_key,
    aws_secret_key=credentials.secret_key,
    aws_region="us-west-2",
)
cli = Client(model, cli=ab)
chat = Chat(model, cli=cli, sp="""You are a helpful and concise assistant.""")

chat("Which football team is known as the Lionesses?")

# > return usage(self.input_tokens+b.input_tokens, self.output_tokens+b.output_tokens, getattr(self,'cache_creation_input_tokens',0)+getattr(b,'cache_creation_input_tokens',0), getattr(self,'cache_read_input_tokens',0)+getattr(b,'cache_read_input_tokens',0))
# >
# > TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Here's the requirements.txt. I install these packages in a new environment.

fastcore==1.7.28
anthropic==0.42.0
toolslm==0.1.0
msglm==0.0.4
nbdev==2.3.34
boto3==1.35.92
claudette==0.1.1
ipykernel==6.29.5

(Unless I'm doing something wrong) the issue seems to be with getting the cache_creation_input_tokens and cache_read_input_tokens attributes: these are actually None rather than missing. So getattr(b,'cache_creation_input_tokens',0) returns None rather than 0.

Something like the following fixes the issue. (It's verbose though and there's probably a better solution; __repr__ and total need fixing as well.)

@patch
def __add__(self: Usage, b: Usage):
    "Add together each of `input_tokens` and `output_tokens`"
    cache_creation_input_tokens = getattr(self, "cache_creation_input_tokens", 0) or 0
    cache_creation_input_tokens += getattr(b, "cache_creation_input_tokens", 0) or 0
    cache_read_input_tokens = getattr(self, "cache_read_input_tokens", 0) or 0
    cache_read_input_tokens += getattr(b, "cache_read_input_tokens", 0) or 0
    return usage(
        self.input_tokens + b.input_tokens,
        self.output_tokens + b.output_tokens,
        cache_creation_input_tokens,
        cache_read_input_tokens,
    )

[rsalimans47]

Getting the same error for AmazonBedrock!