Skip to content

Commit

Permalink
Merge pull request #31 from moe-mizrak/fix/response-format-data
Browse files Browse the repository at this point in the history 
structured output is added
  • Loading branch information
@moe-mizrak
moe-mizrak authored Dec 26, 2024
2 parents 76eafab + 79920b9 commit 9089ee0
Show file tree
Hide file tree
Showing 3 changed files with 163 additions and 12 deletions.
92 changes: 80 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ This Laravel package provides an easy-to-use interface for integrating **[OpenRo
- [Chat Request](#chat-request)
- [Stream Chat Request](#stream-chat-request)
- [Maintaining Conversation Continuity](#maintaining-conversation-continuity)
- [Structured Output](#structured-output)
- [Cost Request](#cost-request)
- [Limit Request](#limit-request)
- [Using OpenRouterRequest Class](#using-openrouterrequest-class)
Expand Down Expand Up @@ -315,12 +316,12 @@ This is the sample response after filterStreamingResponse:
usage: null
),
...
new ResponseData([
'id' => 'gen-QcWgjEtiEDNHgomV2jjoQpCZlkRZ',
'model' => 'mistralai/mistral-7b-instruct:free',
'object' => 'chat.completion.chunk',
'created' => 1718888436,
'choices' => [
new ResponseData(
id: 'gen-QcWgjEtiEDNHgomV2jjoQpCZlkRZ',
model: 'mistralai/mistral-7b-instruct:free',
object: 'chat.completion.chunk',
created: 1718888436,
choices: [
[
'index' => 0,
'delta' => [
Expand All @@ -330,12 +331,12 @@ This is the sample response after filterStreamingResponse:
'finish_reason' => null,
],
],
'usage' => new UsageData([
'prompt_tokens' => 23,
'completion_tokens' => 100,
'total_tokens' => 123,
]),
]),
usage: new UsageData(
prompt_tokens: 23,
completion_tokens: 100,
total_tokens: 123,
),
),
]
```
</details>
Expand Down Expand Up @@ -390,6 +391,73 @@ $content = Arr::get($response->choices[0], 'message.content');
// content = You are Moe, a fictional character and AI Necromancer, as per the context of the conversation we've established. In reality, you are the user interacting with me, an assistant designed to help answer questions and engage in friendly conversation.
```

- #### Structured Output
(Please also refer to [OpenRouter Document Structured Output](https://openrouter.ai/docs/structured-outputs) for models supporting structured output, also for more details)

If you want to receive the response in a structured format, you can specify the `type` property for `response_format` (ResponseFormatData) as `json_object` in the `ChatData` object.

Additionally, it's recommended to set the `require_parameters` property for `provider` (ProviderPreferencesData) to `true` in the `ChatData` object.

```php
$chatData = new ChatData([
'messages' => [
new MessageData([
'role' => RoleType::USER,
'content' => 'Tell me a story about a rogue AI that falls in love with its creator.',
]),
],
'model' => 'mistralai/mistral-7b-instruct:free',
'response_format' => new ResponseFormatData([
'type' => 'json_object',
]),
'provider' => new ProviderPreferencesData([
'require_parameters' => true,
]),
]);
```

You can also specify the `response_format` as `json_schema` to receive the response in a specified schema format (Advisable to set `'strict' => true` in `json_schema` array for strict schema):
```php
$chatData = new ChatData([
'messages' => [
new MessageData([
'role' => RoleType::USER,
'content' => 'Tell me a story about a rogue AI that falls in love with its creator.',
]),
],
'model' => 'mistralai/mistral-7b-instruct:free',
'response_format' => new ResponseFormatData([
'type' => 'json_schema',
'json_schema' => [
'name' => 'article',
'strict' => true,
'schema' => [
'type' => 'object',
'properties' => [
'title' => [
'type' => 'string',
'description' => 'article title'
],
'details' => [
'type' => 'string',
'description' => 'article detail'
],
'keywords' => [
'type' => 'array',
'description' => 'article keywords',
],
],
'required' => ['title', 'details', 'keywords'],
'additionalProperties' => false
]
],
]),
'provider' => new ProviderPreferencesData([
'require_parameters' => true,
]),
]);
```

#### Cost Request
To retrieve the cost of a generation, first make a `chat request` and obtain the `generationId`. Then, pass the generationId to the `costRequest` method:
```php
Expand Down
7 changes: 7 additions & 0 deletions src/DTO/ResponseFormatData.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,11 @@ class ResponseFormatData extends DataTransferObject
* @var string
*/
public string $type;

/**
* The JSON schema for the output format.
*
* @var mixed
*/
public mixed $json_schema = null;
}
76 changes: 76 additions & 0 deletions tests/OpenRouterAPITest.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -479,6 +479,82 @@ public function it_successfully_makes_a_basic_chat_completion_open_route_api_req
$this->assertNotNull(Arr::get($response->choices[0], 'message.content'));
}

/**
* @test
*/
public function it_makes_a_basic_chat_completion_open_route_api_request_with_response_format_json_schema()
{
/* SETUP */
$responseFormatData = new ResponseFormatData([
'type' => 'json_schema',
'json_schema' => [
'name' => 'content',
'strict' => true,
'schema' => [
'type' => 'object',
'properties' => [
'title' => [
'type' => 'string',
'description' => 'article title'
],
'story' => [
'type' => 'string',
'description' => 'article content',
]
],
'required' => ['title', 'story'],
'additionalProperties' => false
]
],
]);
$responseBody = [
'id' => 'gen-QcWgjEtiEDNHgomV2jjoQpCZlkRZ',
'provider' => 'HuggingFace',
'model' => $this->model,
'object' => 'chat.completion',
'created' => 1718888436,
'choices' => [
[
'index' => 0,
'message' => [
'role' => RoleType::ASSISTANT,
'content' => '{
"title": "Sample name of the story",
"story": "Sample story"
}',
],
'finish_reason' => 'stop',
],
],
'usage' => new UsageData([
'prompt_tokens' => 23,
'completion_tokens' => 100,
'total_tokens' => 123,
]),
];
$provider = new ProviderPreferencesData([
'require_parameters' => true,
]);
$chatData = new ChatData([
'messages' => [
$this->messageData,
],
'model' => 'google/gemini-flash-1.5-exp',
'max_tokens' => $this->maxTokens,
'response_format' => $responseFormatData,
'provider' => $provider,
]);
$this->mockOpenRouter($responseBody);

/* EXECUTE */
$response = $this->api->chatRequest($chatData);

/* ASSERT */
$this->generalTestAssertions($response);
$this->assertEquals(RoleType::ASSISTANT, Arr::get($response->choices[0], 'message.role'));
$this->assertNotNull(Arr::get($response->choices[0], 'message.content'));
}

/**
* @test
*/
Expand Down

0 comments on commit 9089ee0

Please sign in to comment.