Go application used to collect and return information about blockchain headers
Pulse is a go server which connects into BSV P2P network to gather and then serve information about all exisiting and new headers. It is build to work as a standaolne app or a module in bigger system.
The main functionality of the application is synchornization with peers and collecting all headers. After starting the server, it creates default objects and connects to BSV P2P network. Application has defined checkpoints (specific headers) which are used in synchronization. During this process, server is asking peers for headers (from checkpoint to checkpoint) in batches of 2000. Every header received from peers is saved in memory. After full synchronization, server is changing the operating mode and start to listening for new header. After when new block has been mined, this information should be sended from peers to our server.
We store everything in database and then we calculate any important information related to: confirmations, state etc. The only exception is when inserting a new header we check if he has any parent in database and if we don't have any parents we set the "isorphan" flag to true. Database schema below:
Headers table schema with sqlite database
- Go 1.19
- Install Go according to the installation instructions here: http://golang.org/doc/install
- Clone the repo
https://github.com/gignative-solutions/ba-p2p-headers.git
In the config.go is the configuration of the application. By changing variables you can adjust the work of our server
defaultConfigFilename = "p2p.conf"
defaultLogLevel = "info"
defaultLogDirname = "logs"
defaultLogFilename = "p2p.log"
defaultMaxPeers = 125
defaultMaxPeersPerIP = 5
defaultBanDuration = time.Hour * 24
defaultConnectTimeout = time.Second * 30
defaultTrickleInterval = peer.DefaultTrickleInterval
defaultExcessiveBlockSize = 128000000
defaultMinSyncPeerNetworkSpeed = 51200
defaultTargetOutboundPeers = uint32(8)
defaultBlocksToConfirmFork = 10
Settings related to database:
- DB_DSN=file:/data/blockheaders.db?_foreign_keys=true&pooled=true
- DB_SCHEMA_PATH=/migrations
- DB_PREPAREDDB=true
- DB_PREPAREDDBFILE_PATH="./data/blockheaders.xz"
DSN can be used to change the local database location - this should be a volume mount into the container while SQLite is the only db option, we will support more in future.
DB_SCHEMA_PATH should always be set to /migrations, that's the location within the container where the db migration files are head and will setup the database correctly.
DB_PREPAREDDB is used to define if application should use prepared db.
DB_PREPAREDDBFILE_PATH define path to prepared db.
Settings related to admin auth: - HTTP_SERVER_AUTHTOKEN=admin_only_afUMlv5iiDgQtj22O9n5fADeSb
This admin token should be used as a Bearer token in the Authorization header when dynamically creating secure tokens for applications to then use at the POST /api/v1/access endpoint.
go run ./cmd/ .or with Docker
docker compose up --buildFor endpoints documentation you can visit swagger which is exposed on port 8080 by default.
http://localhost:8080/swagger/index.html
The default assumes you want to use Authentication. This requires a single environment variable.
HTTP_SERVER_AUTHTOKEN=replace_me_with_token_you_want_to_use_as_admin_token
To disable authentication exposing all endpoints openly, set the following environment variable. This is available if you prefer to use your own authentication in a separate proxy or similar. We do not recommend you expose the server to the internet without authentication, as it would then be possible for anyone to prune your headers at will.
HTTP_SERVER_USEAUTH=false
After the setup of authentication you can use provided token to authenticate. To do it, just add the following header to all the requests to pulse
Authorization Bearer replace_me_with_token_you_want_to_use_as_admin_token
If you have a need for additional tokens to authenticate in pulse you can generate such with the following request:
POST https://{{pulse_url}}/api/v1/access
Authorization: Bearer replace_me_with_token_you_want_to_use_as_admin_tokenIn response you should receive something like
{
"token": "some_token_created_by_server",
"createdAt": "2023-05-11T10:20:16.227582Z",
"isAdmin": false
}Now you can put a value from "token" property from the response and use it in all requests to server by setting header:
Authorization: Bearer some_token_created_by_serverIf at some point you want to revoke this additional token you can make a request:
DELETE https://{{pulse_url}}/api/v1/access/{{some_token_created_by_server}}
Authorization: Bearer replace_me_with_token_you_want_to_use_as_admin_tokenAfter this request succeeded the token can't be used to authenticate in pulse.
Pulse can notify a client via websockets that new header was received and store by it.
Pulse use centrifugal/centrifuge to run a server. Therefore, to integrate you need to choose a client library matching a programming language of your choice.
Example how to subscribe using GO lang library centrifugal/centrifuge-go can be found in ./examples/ws-subscribe-to-new-headers/
Creating a new webhook is done via POST request
POST https://{{pulse_url}}/api/v1/webhookData which should be sent in body:
{
"url": "<server_url>",
"requiredAuth": {
"type": "BEARER|CUSTOM_HEADER",
"token": "<authorization_token>",
"header": "<custom_header_name>",
}
}
Information:
- If authorization is enabled this request also requires
Authorizationheader - url have to include http or https protocol example:
https://test-url.com - requiredAuth is used to define authorization for webhook
- type
BEARER- token will be placed inAuthorization: Bearer {{token}}header - type
CUSTOM_HEADER- authorization header will be build from given variables{{header}}: {{token}}
- type
Example response:
{
"url": "http://example.com/api/v1/webhook/new-header",
"createdAt": "2023-05-11T13:05:23.297808+02:00",
"lastEmitStatus": "",
"lastEmitTimestamp": "0001-01-01T00:00:00Z",
"errorsCount": 0,
"active": true
}After that webhook is created and will be informed about new headers.
To check webhook you can use the GET request which will return webhook object (same as when creating new webhook) from which you can get all the information
GET https://{{pulse_url}}/api/v1/webhook?url={{webhook_url}}If you want to revoke webhook you can use the following request:
DELETE https://{{pulse_url}}/api/v1/webhook?url={{webhook_url}}This request will delete webhook permanently
If the number of failed requests wil exceed WEBHOOK_MAXTRIES, webhook will be set to inactive. To refresh webhook you can use this same endpoint as for webhook creation.