agentchat.conversable_agent.get_total_usage() total_cost not working for gpt-3.5-turbo-0125

[WebsheetPlugin]

I had been using get_total_usage and I noticed that it always shows only up to 5 digits after 0, for example: 0.00698.

But when I use gpt-3.5-turbo-0125 this is a problem, as I sometimes get a cost of 0

See here:

"gpt-3.5-turbo-0125": {
"cost": 0,
"prompt_tokens": 315,
"completion_tokens": 10,
"total_tokens": 325
}

So my question here, is how to disable this trimming and get full value?

[ekzhu]

cc @kevin666aa

It could be due to caching so you are not hitting the LLM. You can disable cache in your llm_config by setting "cache_seed": None.

Related to #2133

[WebsheetPlugin]

I have disabled the cache. I also always added a random string to each request to keep it unique.

Ok, now I have one cost with 6 digits. So I am not sure anymore they are trimmed to 5 digits.

Anyway, when using the older gpt-3.5-turbo-1106 instead of gpt-3.5-turbo or gpt-3.5-turbo-0125 the cost is not zero.

For you, it's working differently. Right?
Here more proof:

with old model:
{'total_cost': 0.000427, 'gpt-3.5-turbo-1106': {'cost': 0.000427, 'prompt_tokens': 397, 'completion_tokens': 15, 'total_tokens': 412}}

with new model:
({'total_cost': 0.02041, 'gpt-3.5-turbo-0125': {'cost': 0, 'prompt_tokens': 398, 'completion_tokens': 15, 'total_tokens': 413}, 'gpt-4-0125-preview': {'cost': 0.02041, 'prompt_tokens': 55, 'completion_tokens': 662, 'total_tokens': 717}}, {'total_cost': 0.02041, 'gpt-3.5-turbo-0125': {'cost': 0, 'prompt_tokens': 398, 'completion_tokens': 15, 'total_tokens': 413}, 'gpt-4-0125-preview': {'cost': 0.02041, 'prompt_tokens': 55, 'completion_tokens': 662, 'total_tokens': 717}})

I also have this print which shows it more clearly:

Usage summary excluding cached usage:
Total cost: 0

• Model 'gpt-3.5-turbo-0125': cost: 0, prompt_tokens: 398, completion_tokens: 15, total_tokens: 413

All completions are non-cached: the total cost with cached completions is the same as actual cost.

[yiranwu0]

Hello @WebsheetPlugin, can you update the package and see if cost for gpt-3.5-turbo-0125 is still zero? It should be fixed.

When printing, we are rounding to 5 digits (print_usage_summary). But if you access the numbers through dict, we don't trim the cost.

[ekzhu]

@WebsheetPlugin, please refer to @kevin666aa's answer. Mine was inaccurate.

[WebsheetPlugin]

Thanks guys!

I confirm newest version works.