Bug: Weave predefined scorers fail for AsyncOpenAI clients

[TeoZosa]

Excited for the addition of new scoring functions, thanks for the hard work. With it being hot off the presses, I did run into this issue.

AsyncOpenAI clients seem to be supported:

weave/weave/scorers/llm_utils.py

Lines 28 to 30 in 585752e

	_LLM_CLIENTS = Union[
	OpenAI, AsyncOpenAI, Anthropic, AsyncAnthropic, Mistral, GenerativeModel
	]

But the underlying calls for Weave scorers assume non-async clients:

weave/weave/scorers/llm_utils.py

Lines 85 to 96 in 585752e

	def embed(
	client: _LLM_CLIENTS, model_id: str, texts: str | list[str], **kwargs: Any
	) -> list[list[float]]:
	client_type = type(client).__name__.lower()
	if "openai" in client_type:
	response = client.embeddings.create(model=model_id, input=texts, **kwargs)
	return [embedding.embedding for embedding in response.data]
	elif "mistral" in client_type:
	response = client.embeddings.create(model=model_id, inputs=texts, **kwargs)
	return [embedding.embedding for embedding in response.data]
	else:
	raise ValueError(f"Unsupported client type: {type(client).__name__.lower()}")

[TeoZosa]

Thanks @tcapelle for jumping on this so fast! 🙏

[TeoZosa]

Hey @tcapelle, has this been deprioritized? We've been vendoring a fork of the code in our repos but this issue is blocking feature mainlining and broader release of some of our internal tools.

[tcapelle]

Not at all, but we have a significant scorer overhaul incoming, that's why this change has not been integrated yet.