data structures management client
The library is both Golang api and command-line client for managing data structures.
"A data structure is a data organization, and storage format that is usually chosen for efficient access to data" - Wikipedia says.
Data structures are widely utilized across various domains in Computer Science and Software Engineering. Unlike key-value or relational datastores, data structures are an algebraic abstractions that implements a unique properties tailored to meet the specific needs of applications. This library eliminate the extra cost of converting application objects into database entities for each database operation.
This library provides remote access to sophisticated data structures, giving the simplicity of developing application with fewer lines of code to store and access data.
Install the command-line utility from source code. It requires Golang to be installed:
go install github.com/kshard/optimum/cmd/optimum@latestThe library usage requires access to api that provisions and operates data structures for you. Contact your provided for api details.
It is recommended to config environment variables for client usage:
export HOST=https://example.com
export ROLE=arn:aws:iam::000000000000:role/example-access-roleUsing data structures typically involves a following workflow:
- List existing data structures.
- Create a new instance of data structure.
- Write data.
- Read data.
- Remove the data structure instance.
A data structure can be seen as a typed algebraic abstraction that encompasses a collection of data values, the relationships between those values, and the operations or functions that can be applied to manipulate the data. In practical application development, each data structure must be uniquely identifiable to allow efficient access and manipulation. To facilitate this, the application uses a unique reference name called a CURIE (Compact Uniform Resource Identifier). The CURIE combines both the data structure type and a unique identifier, ensuring that the correct data structure is referenced throughout the workflow, enabling smooth interactions within the system.
See tutorials for example usage.
List all data structure instances. It fetches data structure instances of same type. For each provisioned instance it reports NAME, active VERSION, UPDATED AT timestamp, instance STATUS, PENDING version if any, and initialization PARAMS.
optimum <type> list -u $HOST
NAME VERSION UPDATED AT | STATUS PENDING | PARAMS
example1 NjqOYyOkpMHfg3.6 2024-08-18 10:40:34 | ACTIVE | {}
example2 2024-08-18 10:38:13 | PENDING NjqOYyOkpMHfg3.6 | {}Create new instance of data structure. See either documentation of supported
data structure or optimum help for details about configuration parameters.
optimum <type> create -u $HOST -n <name> -j path/to/config.jsonThe batch writing consist of two phases - data upload followed by a commit.
See either documentation of supported data structure or optimum help for
details about upload file format.
# Upload data into server.
optimum <type> upload -u $HOST -n <name> path/to/data.txt
# Commit uploaded data, making it available online.
optimum <type> commit -u $HOST -n <name>Use the REST API for any advanced reading use cases, as the client only supports
basic read operations. See either documentation of supported data structure or
optimum help for details about query formats.
optimum <type> query -u $HOST -n <name> path/to/query.txtThe command removes data structure instance. The operation is irreversible and results in the permanent destruction of all data.
optimum <type> remove -u $HOST -n <name>The command library supports following data structures:
Continue with examples and tutorials.
Note: the command line is only support basic operation for data structure manipulation. Use Golang API for any advanced scenario.
The latest version of the module is available at main branch. All development, including new features and bug fixes, take place on the main branch using forking and pull requests as described in contribution guidelines. The stable version is available via Golang modules.
Use go get to retrieve the library and add it as dependency to your application.
go get -u github.com/kshard/optimumThe client library support
optimumpackage is control plane to coordinate the instances lifecycle.optimum/surfacepackage is data plane for reading/writing Graph-based Nearest Neighbor N-dimensional Surface.optimum/sentencespackage is data plane for reading/writing natural language text and searching for nearest neighbor.
The example below shows usage of client for reading natural language text.
package main
import (
"github.com/kshard/optimum/sentences"
"github.com/fogfish/gurl/v2/http"
"github.com/fogfish/curie"
)
const (
host = "https://example.com"
cask = curie.IRI("text:example")
)
func main() {
// Create client, the library depends on
api := sentences.New(http.New(), host)
// Query the data structure
neighbors, err := api.Query(context.Background(), cask,
sentences.Query{Text: "hello world!"},
)
// Print results
fmt.Println("Nearest neighbors:", neighbors)
}The library is MIT licensed and accepts contributions via GitHub pull requests:
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Added some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
The build and testing process requires Go version 1.21 or later.
The commit message helps us to write a good release note, speed-up review process. The message should address two question what changed and why. The project follows the template defined by chapter Contributing to a Project of Git book.
If you experience any issues with the library, please let us know via GitHub issues. We appreciate detailed and accurate reports that help us to identity and replicate the issue.
