generated from StanfordBDHG/SwiftPackageTemplate
-
-
Notifications
You must be signed in to change notification settings - Fork 19
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
# OpenAI Function Calling ## ♻️ Current situation & Problem As of now, the `SpeziLLMOpenAI` target doesn't provide an option to use the [OpenAI Function Calling mechanism](https://platform.openai.com/docs/guides/function-calling). This functionality is crucial to advanced LLM use cases, such as `HealthGPT` or `LLMonFHIR`. Related to #10 ## ⚙️ Release Notes - `SpeziLLMOpenAI` now supports the OpenAI function calling mechanism via a Domain Specific Language, enabling straightforward usage of function calling within Spezi-based applications. ## 📚 Documentation Inline DocC as well as DocC articles written, other documentation adjusted where necessary. ## ✅ Testing Unit tests to properly test injection logic of parameters into `LLMFunction`s as well as validation of schema synthization. No proper UI tests as we would test the OpenAI API and therefore need proper API tokens and OpenAI balance. However, extensive manual testing via the UI test application. ## 📝 Code of Conduct & Contributing Guidelines By submitting creating this pull request, you agree to follow our [Code of Conduct](https://github.com/StanfordSpezi/.github/blob/main/CODE_OF_CONDUCT.md) and [Contributing Guidelines](https://github.com/StanfordSpezi/.github/blob/main/CONTRIBUTING.md): - [x] I agree to follow the [Code of Conduct](https://github.com/StanfordSpezi/.github/blob/main/CODE_OF_CONDUCT.md) and [Contributing Guidelines](https://github.com/StanfordSpezi/.github/blob/main/CONTRIBUTING.md).
- Loading branch information
1 parent
c3b31a3
commit 24d6c19
Showing
52 changed files
with
2,811 additions
and
119 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| // | ||
| // This source file is part of the Stanford Spezi open source project | ||
| // | ||
| // SPDX-FileCopyrightText: 2022 Stanford University and the project authors (see CONTRIBUTORS.md) | ||
| // | ||
| // SPDX-License-Identifier: MIT | ||
| // | ||
|
|
||
| import Foundation | ||
|
|
||
|
|
||
| extension String { | ||
| /// Initializes a Swift `String` from a C++ `string`. | ||
| /// | ||
| /// - Parameters: | ||
| /// - cxxString: The given C++ `string` | ||
| /// | ||
| /// In the Release build mode, the Swift compiler is unable to choose the correct String initializer from the Swift stdlib. | ||
| /// Therefore, manual `String `extension by SpeziLLM that mirrors the C++ interop implementation within the Swift stdlib: https://github.com/apple/swift/blob/cf2a338afca54a787d59b83db6238b1568215b94/stdlib/public/Cxx/std/String.swift#L231-L239 | ||
| init(_ cxxString: std.string) { | ||
| let buffer = UnsafeBufferPointer<CChar>( | ||
| start: cxxString.__c_strUnsafe(), | ||
| count: cxxString.size() | ||
| ) | ||
| self = buffer.withMemoryRebound(to: UInt8.self) { | ||
| String(decoding: $0, as: UTF8.self) | ||
| } | ||
| withExtendedLifetime(cxxString) {} | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
56 changes: 56 additions & 0 deletions
56
Sources/SpeziLLMOpenAI/FunctionCalling/Helpers/LLMFunctionBuilder.swift
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| // | ||
| // This source file is part of the Stanford Spezi open-source project | ||
| // | ||
| // SPDX-FileCopyrightText: 2022 Stanford University and the project authors (see CONTRIBUTORS.md) | ||
| // | ||
| // SPDX-License-Identifier: MIT | ||
| // | ||
|
|
||
| import Foundation | ||
|
|
||
|
|
||
| /// A result builder used to aggregate multiple ``LLMFunction``s within the ``LLMOpenAI``. | ||
| @resultBuilder | ||
| public enum LLMFunctionBuilder { | ||
| /// If declared, provides contextual type information for statement expressions to translate them into partial results. | ||
| public static func buildExpression<L: LLMFunction>(_ expression: L) -> [L] { | ||
| [expression] | ||
| } | ||
|
|
||
| /// Required by every result builder to build combined results from statement blocks. | ||
| public static func buildBlock(_ children: [any LLMFunction]...) -> [any LLMFunction] { | ||
| children.flatMap { $0 } | ||
| } | ||
|
|
||
| /// Enables support for `if` statements that do not have an `else`. | ||
| public static func buildOptional(_ component: [any LLMFunction]?) -> [any LLMFunction] { | ||
| // swiftlint:disable:previous discouraged_optional_collection | ||
| // The optional collection is a requirement defined by @resultBuilder, we can not use a non-optional collection here. | ||
| component ?? [] | ||
| } | ||
|
|
||
| /// With buildEither(second:), enables support for 'if-else' and 'switch' statements by folding conditional results into a single result. | ||
| public static func buildEither(first: [any LLMFunction]) -> [any LLMFunction] { | ||
| first | ||
| } | ||
|
|
||
| /// With buildEither(first:), enables support for 'if-else' and 'switch' statements by folding conditional results into a single result. | ||
| public static func buildEither(second: [any LLMFunction]) -> [any LLMFunction] { | ||
| second | ||
| } | ||
|
|
||
| /// Enables support for 'for..in' loops by combining the results of all iterations into a single result. | ||
| public static func buildArray(_ components: [[any LLMFunction]]) -> [any LLMFunction] { | ||
| components.flatMap { $0 } | ||
| } | ||
|
|
||
| /// If declared, this will be called on the partial result of an 'if #available' block to allow the result builder to erase type information. | ||
| public static func buildLimitedAvailability(_ component: [any LLMFunction]) -> [any LLMFunction] { | ||
| component | ||
| } | ||
|
|
||
| /// If declared, this will be called on the partial result from the outermost block statement to produce the final returned result. | ||
| public static func buildFinalResult(_ component: [any LLMFunction]) -> _LLMFunctionCollection { | ||
| _LLMFunctionCollection(functions: component) | ||
| } | ||
| } |
25 changes: 25 additions & 0 deletions
25
Sources/SpeziLLMOpenAI/FunctionCalling/Helpers/LLMFunctionParameterCodingKey.swift
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| // | ||
| // This source file is part of the Stanford Spezi open source project | ||
| // | ||
| // SPDX-FileCopyrightText: 2022 Stanford University and the project authors (see CONTRIBUTORS.md) | ||
| // | ||
| // SPDX-License-Identifier: MIT | ||
| // | ||
|
|
||
| import Foundation | ||
|
|
||
|
|
||
| /// In case of a `DecodingError` of the called function parameters to the ``LLMFunction/Parameter``s, indicates where in the `DecodingError` occurred. | ||
| struct LLMFunctionParameterCodingKey: CodingKey { | ||
| let stringValue: String | ||
| var intValue: Int? | ||
|
|
||
|
|
||
| init(stringValue: String) { | ||
| self.stringValue = stringValue | ||
| } | ||
|
|
||
| init?(intValue: Int) { | ||
| self.stringValue = String(intValue) | ||
| } | ||
| } |
85 changes: 85 additions & 0 deletions
85
...eziLLMOpenAI/FunctionCalling/Helpers/LLMFunctionParameterIntermediateRepresentation.swift
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| // | ||
| // This source file is part of the Stanford Spezi open source project | ||
| // | ||
| // SPDX-FileCopyrightText: 2022 Stanford University and the project authors (see CONTRIBUTORS.md) | ||
| // | ||
| // SPDX-License-Identifier: MIT | ||
| // | ||
|
|
||
| import Foundation | ||
|
|
||
|
|
||
| /// Serves as an intermediary representation of the requested function call parameters in order to decode the parameters into the ``LLMFunction/Parameter``s. | ||
| enum LLMFunctionParameterIntermediary: Codable { | ||
| case null | ||
| case int(Int) | ||
| case bool(Bool) | ||
| case string(String) | ||
| case double(Double) | ||
| case array(Array<LLMFunctionParameterIntermediary>) | ||
| case dictionary([String: LLMFunctionParameterIntermediary]) | ||
|
|
||
|
|
||
| /// Provides a representation of the received JSON where each first-level parameter (the key) maps to the respective nested JSON `Data`. | ||
| var topLayerJSONRepresentation: [String: Data] { | ||
| get throws { | ||
| guard case let .dictionary(dictionary) = self else { | ||
| return [:] | ||
| } | ||
|
|
||
| return try dictionary.mapValues { | ||
| try JSONEncoder().encode($0) | ||
| } | ||
| } | ||
| } | ||
|
|
||
|
|
||
| init(from decoder: Decoder) throws { | ||
| let container = try decoder.singleValueContainer() | ||
|
|
||
| if container.decodeNil() { | ||
| self = .null | ||
| } else if let bool = try? container.decode(Bool.self) { | ||
| self = .bool(bool) | ||
| } else if let int = try? container.decode(Int.self) { | ||
| self = .int(int) | ||
| } else if let double = try? container.decode(Double.self) { | ||
| self = .double(double) | ||
| } else if let string = try? container.decode(String.self) { | ||
| self = .string(string) | ||
| } else if let array = try? container.decode([LLMFunctionParameterIntermediary].self) { | ||
| self = .array(array) | ||
| } else if let dictionary = try? container.decode([String: LLMFunctionParameterIntermediary].self) { | ||
| self = .dictionary(dictionary) | ||
| } else { | ||
| throw DecodingError.dataCorrupted( | ||
| DecodingError.Context( | ||
| codingPath: decoder.codingPath, | ||
| debugDescription: "Encountered unexpected JSON values within LLM Function Calling Parameters" | ||
| ) | ||
| ) | ||
| } | ||
| } | ||
|
|
||
|
|
||
| func encode(to encoder: Encoder) throws { | ||
| var container = encoder.singleValueContainer() | ||
|
|
||
| switch self { | ||
| case .null: | ||
| return | ||
| case let .int(int): | ||
| try container.encode(int) | ||
| case let .bool(bool): | ||
| try container.encode(bool) | ||
| case let .string(string): | ||
| try container.encode(string) | ||
| case let .double(double): | ||
| try container.encode(double) | ||
| case let .array(array): | ||
| try container.encode(array) | ||
| case let .dictionary(dictionary): | ||
| try container.encode(dictionary) | ||
| } | ||
| } | ||
| } |
25 changes: 25 additions & 0 deletions
25
Sources/SpeziLLMOpenAI/FunctionCalling/Helpers/_LLMFunctionCollection.swift
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| // | ||
| // This source file is part of the Stanford Spezi open-source project | ||
| // | ||
| // SPDX-FileCopyrightText: 2022 Stanford University and the project authors (see CONTRIBUTORS.md) | ||
| // | ||
| // SPDX-License-Identifier: MIT | ||
| // | ||
|
|
||
| import Foundation | ||
| import SwiftUI | ||
|
|
||
|
|
||
| /// Defines a collection of ``SpeziLLMOpenAI`` ``LLMFunction``s. | ||
| /// | ||
| /// You can not create a `_LLMFunctionCollection` yourself. Please use the ``LLMOpenAI`` that internally creates a `_LLMFunctionCollection` with the passed ``LLMFunction``s. | ||
| public struct _LLMFunctionCollection { // swiftlint:disable:this type_name | ||
| var functions: [String: LLMFunction] = [:] | ||
|
|
||
|
|
||
| init(functions: [any LLMFunction]) { | ||
| for function in functions { | ||
| self.functions[Swift.type(of: function).name] = function | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.