Skip to content

Commit

Permalink
renaming things to vaults
Browse files Browse the repository at this point in the history
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
  • Loading branch information
brunocalza committed Dec 14, 2023
1 parent 3d16294 commit 0c541e0
Show file tree
Hide file tree
Showing 17 changed files with 233 additions and 384 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
/basin
/vaults
/cover.out
.vscode/*
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ lint:

# Build
build:
go build -o basin cmd/basin/*
go build -o vaults cmd/vaults/*
.PHONY: build

# Test
Expand Down
78 changes: 39 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# basin-cli
# vaults-cli

[![License](https://img.shields.io/github/license/tablelandnetwork/basin-cli.svg)](./LICENSE)
[![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg)](https://github.com/RichardLitt/standard-readme)
Expand All @@ -12,20 +12,20 @@
- [Self-hosted](#self-hosted)
- [Amazon RDS](#amazon-rds)
- [Supabase](#supabase)
- [Create a publication](#create-a-publication)
- [Start replicating a publication](#start-replicating-a-publication)
- [Create a vault](#create-a-vault)
- [Start replicating a database](#start-replicating-a-database)
- [Upload a Parquet file](#upload-a-parquet-file)
- [Listing Publications](#listing-publications)
- [Listing Deals](#listing-deals)
- [Listing Vaults](#listing-vaults)
- [Listing Events](#listing-events)
- [Running](#running)
- [Run tests](#run-tests)
- [Retrieving](#retrieving)

# Background

Tableland Basin is a secure and verifiable open data platform. The Basin CLI is a tool that allows you to continuously replicate a table or view from your database to the network. Currently, only PostgreSQL is supported.
Textile Vaults is a secure and verifiable open data platform. The Vaults CLI is a tool that allows you to continuously replicate a table or view from your database to the network. Currently, only PostgreSQL is supported.

> 🚧 Basin is currently not in a production-ready state. Any data that is pushed to the network may be subject to deletion. 🚧
> 🚧 Vaults is currently not in a production-ready state. Any data that is pushed to the network may be subject to deletion. 🚧
# Usage

Expand All @@ -34,14 +34,14 @@ Tableland Basin is a secure and verifiable open data platform. The Basin CLI is
```bash
git clone https://github.com/tablelandnetwork/basin-cli.git
cd basin-cli
go install ./cmd/basin
go install ./cmd/vaults
```

## Postgres Setup

### Self-hosted

- Make sure you have access to a superuser role. For example, you can create a new role such as `CREATE ROLE basin WITH PASSWORD NULL LOGIN SUPERUSER;`.
- Make sure you have access to a superuser role. For example, you can create a new role such as `CREATE ROLE vaults WITH PASSWORD NULL LOGIN SUPERUSER;`.
- Check that your Postgres installation has the [wal2json](https://github.com/eulerto/wal2json) plugin installed.
- Check if logical replication is enabled:

Expand Down Expand Up @@ -69,7 +69,7 @@ go install ./cmd/basin
WHERE name = 'rds.logical_replication';
```

- If it's on, go to [Create a publication](#create-a-publication)
- If it's on, go to [Create a vault](#create-a-vault)
- If it's off, follow the next steps:
- [Create a custom RDS parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithDBInstanceParamGroups.html#USER_WorkingWithParamGroups.Creating)
- After creation, edit it and set the `rds.logical_replication` parameter to `1`
Expand All @@ -83,92 +83,92 @@ go install ./cmd/basin
- Log into the [Supabase](https://supabase.io/) dashboard and go to your project, or create a new one.
- Check if logical replication is enabled. This should be the default setting, so you shouldn't have to change anything. You can do this in the `SQL Editor` section on the left hand side of the Supabase dashboard by running `SHOW wal_level;` query, which should log `logical`.
- You can find the database connection information on the left hand side under `Project Settings` > `Database`. You will need the `Host`, `Port`, `Database`, `Username`, and `Password` to connect to your database.
- When you create a publication, the `--dburi` should follow this format:
- When you create a vault, the `--dburi` should follow this format:
```sh
postgresql://postgres:[PASSWORD]@db.[PROJECT_ID].supabase.co:5432/postgres
```

## Create a publication
## Create a vault

_Publications_ define the data you are pushing to Basin.
_Vaults_ define the place you push data into.

Basin uses public key authentication, so you will need an Ethereum style (ECDSA, secp256k1) wallet to create a new publication. You can use an existing wallet or set up a new one with `basin wallet create`. Your private key is only used locally for signing.
Vaults uses public key authentication, so you will need an Ethereum style (ECDSA, secp256k1) wallet to create a new vault. You can use an existing wallet or set up a new one with `vaults wallet create`. Your private key is only used locally for signing.

```bash
basin wallet create [FILENAME]
vaults wallet create [FILENAME]
```

A new private key will be written to `FILENAME`.

The name of a publication contains a `namespace` (e.g. `my_company`) and the name of an existing database relation (e.g. `my_table`), separated by a period (`.`). Use `basin publication create` to create a new publication. See `basin publication create --help` for more info.
The name of a vault contains a `namespace` (e.g. `my_company`) and the name of an existing database relation (e.g. `my_table`), separated by a period (`.`). Use `vaults create` to create a new vault. See `vaults create --help` for more info.

```bash
basin publication create --dburi [DBURI] --address [WALLET_ADDRESS] namespace.relation_name
vaults create --dburi [DBURI] --address [WALLET_ADDRESS] namespace.relation_name
```

🚧 Basin currently only replicates `INSERT` statements, which means that it only replicates append-only data (e.g., log-style data). Row updates and deletes will be ignored. 🚧
🚧 Vaults currently only replicates `INSERT` statements, which means that it only replicates append-only data (e.g., log-style data). Row updates and deletes will be ignored. 🚧

## Start replicating a publication
## Start replicating a database

Use `basin publication start` to start a daemon that will continuously push changes to the underlying table/view to the network. See `basin publication start --help` for more info.
Use `vaults stream` to start a daemon that will continuously push changes to the underlying table/view to the network. See `vaults stream --help` for more info.

```bash
basin publication start --private-key [PRIVATE_KEY] namespace.relation_name
vaults stream --private-key [PRIVATE_KEY] namespace.relation_name
```

## Upload a Parquet file

Before uploading a Parquet file, you need to [Create a publication](#create-a-publication), if not already created. You can omit the `--dburi` flag, in this case.
Before uploading a Parquet file, you need to [Create a vault](#create-a-vault), if not already created. You can omit the `--dburi` flag, in this case.

Then, use `basin publication upload` to upload a Parquet file.
Then, use `vaults write` to write a Parquet file.

```bash
basin publication upload --name [namespace.relation_name] --private-key [PRIVATE_KEY] filepath
vaults write --vault [namespace.relation_name] --private-key [PRIVATE_KEY] filepath
```

You can attach a timestamp to that file upload, e.g.

```bash
basin publication upload --name [namespace.relation_name] --private-key [PRIVATE_KEY] --timestamp 1699984703 filepath
vaults upload --vault [namespace.relation_name] --private-key [PRIVATE_KEY] --timestamp 1699984703 filepath
# or use data format
basin publication upload --name [namespace.relation_name] --private-key [PRIVATE_KEY] --timestamp 2006-01-02 filepath
vaults upload --vault [namespace.relation_name] --private-key [PRIVATE_KEY] --timestamp 2006-01-02 filepath
# or use RFC3339 format
basin publication upload --name [namespace.relation_name] --private-key [PRIVATE_KEY] --timestamp 2006-01-02T15:04:05Z07:00 filepath
vaults upload --vault [namespace.relation_name] --private-key [PRIVATE_KEY] --timestamp 2006-01-02T15:04:05Z07:00 filepath
```

If a timestamp is not provided, the CLI will assume the timestamp is the current client epoch in UTC.

## Listing Publications
## Listing Vaults

You can list the publications from an address by running:
You can list the vaults from an account by running:

```bash
basin publication list --address [ETH_ADDRESS]
vaults list --account [ETH_ADDRESS]
```

## Listing Deals
## Listing Events

You can list deals of a given publication by running:
You can list events of a given vault by running:

```bash
basin publication deals --publication [PUBLICATION] --latest 5
vaults events --vault [VAULT_NAME] --latest 5
```

Deals command accept `--before`,`--after` , and `--at` flags to filter deals by timestamp
Events command accept `--before`,`--after` , and `--at` flags to filter events by timestamp

```bash
# examples
basin publication deals --publication demotest.data --at 1699569502
basin publication deals --publication demotest.data --before 2023-11-09T19:38:23-03:00
basin publication deals --publication demotest.data --after 2023-11-09
vaults events --vault demotest.data --at 1699569502
vaults events --vault demotest.data --before 2023-11-09T19:38:23-03:00
vaults events --vault demotest.data --after 2023-11-09
```

## Retrieving

```bash
basin publication retrieve bafybeifr5njnrw67yyb2h2t7k6ukm3pml4fgphsxeurqcmgmeb7omc2vlq
vaults retrieve bafybeifr5njnrw67yyb2h2t7k6ukm3pml4fgphsxeurqcmgmeb7omc2vlq
```

# Development
Expand All @@ -185,7 +185,7 @@ PORT=8888 ./scripts/server.sh
./scripts/run.sh wallet create pk.out
# Start replicating
./scripts/run.sh publication start --private-key [PRIVATE_KEY] namespace.relation_name
./scripts/run.sh vaults stream --private-key [PRIVATE_KEY] namespace.relation_name
```

## Run tests
Expand Down
124 changes: 0 additions & 124 deletions cmd/basin/publication_test.go

This file was deleted.

69 changes: 0 additions & 69 deletions cmd/basin/wallet.go

This file was deleted.

Loading

0 comments on commit 0c541e0

Please sign in to comment.