diff --git a/docs/logging-guidelines.md b/docs/logging-guidelines.md new file mode 100644 index 0000000000..ffb1849046 --- /dev/null +++ b/docs/logging-guidelines.md @@ -0,0 +1,21 @@ +This page describes the logging mechanics and lists guidelines for clear logging. + +## Goal +There are 2 main goals of the logging: providing support and tracing down errors in development + +Logging with the support goal the is to inform semi-technical users about the state of the application. +This means all levels from INFO and up need to be clear to understand for everyone. + +Logging with the development goal is to find where errors originate from. This means every log message needs sufficient information to trace the application its steps. + +Use cases for the logs are: +- Support and development teams will receive an end-user log to investigate +- Run the application in DEBUG log-level to see what goes on in detail. + +## Levels +These are the log levels available to the logger: +- **CRITICAL** = app is unable to continue, it should exit after this message +- **ERROR** = app is unable to perform the requested action and stops trying +- **WARNING** = app is unable to perform the requested action but will continue to run, for instance try again later +- **INFO** = app performs a requested action +- **DEBUG** = gives extra details over other messages, always a following message diff --git a/docs/provider/capabilities.md b/docs/provider/capabilities.md new file mode 100644 index 0000000000..e5ddaf928e --- /dev/null +++ b/docs/provider/capabilities.md @@ -0,0 +1,11 @@ +# Provider agent capabilities List + +## Protocol + +| Capability | Yagna package version | Backwards-compatible? | Description | +|-------------------------------|-----------------------|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Multi-Activity Agreement | 0.5.0 | Yes | Negotiate ability to create multiple activities under single Agreement. Use (which?) property in Demand/Offer to indicate node's support for Multi-Activity. If counterparty does not support Multi-Activity, the node falls back to single Activity per Agreement behaviour. | +| Restart Proposal Negotiations | 0.7.0 | Yes | Agent is allowed to restart negotiations, by sending `Counter Proposal`, after he rejected Proposal at some point. Counter-party will receive regular `ProposalEvent` in this case. Only Agent rejecting Proposal has initiative in restarting negotiations, rejected Agent can only wait for this to happen. To indicate, that Proposal rejection isn't final and negotiations can be restarted later, Agent can set `golem.proposal.rejection.is-final` (bool) field in `Reason` structure. If this value is set to false, Agent can free any state related to this negotiation. The same field can be set in `Reason` sent in `Reject Agreement` operation. Requestor can send new counter Proposal after some period of time or propose the same Agreement for the second time. (No change to specification) | +| manifest-support | | | TODO | +| inet | | | TODO | +| start-entrypoint | | | TODO | diff --git a/docs/provider/exe-unit/exe-units.md b/docs/provider/exe-unit/exe-units.md new file mode 100644 index 0000000000..c9c640b0d1 --- /dev/null +++ b/docs/provider/exe-unit/exe-units.md @@ -0,0 +1,82 @@ +# ExeUnits + +## How Provider agent finds ExeUnits? + +`ya-provider` lists available ExeUnits by listing descriptors placed in [directory](../overview.md#provider-directories). + +Example directory content: +```commandline +nieznanysprawiciel@laptop:~$ tree ~/.local/lib/yagna/plugins +/home/nieznanysprawiciel/.local/lib/yagna/plugins +├── exe-unit +├── ya-runtime-vm +│   ├── runtime +│   │   ├── bios-256k.bin +│   │   ├── efi-virtio.rom +│   │   ├── initramfs.cpio.gz +│   │   ├── kvmvapic.bin +│   │   ├── linuxboot_dma.bin +│   │   ├── self-test.gvmi +│   │   ├── vmlinuz-virt +│   │   └── vmrt +│   └── ya-runtime-vm +├── ya-runtime-vm.json +├── ya-runtime-wasi +└── ya-runtime-wasi.json +``` +ExeUnit descriptors (`ya-runtime-vm.json` and `ya-runtime-wasi.json`) must be placed directly in plugins directory (not in nested structure). +Other ExeUnits' files can be moved into subdirectories, because descriptor is pointing to specific files. + +## Descriptor + +Example ExeUnit descriptor: +```json +[ + { + "name": "custom", + "version": "0.2.2", + "supervisor-path": "exe-unit", + "runtime-path": "custom-runtime/custom", + "description": "Custom runtime for documentation purposes.", + "extra-args": ["--runtime-managed-image"], + "properties": { + "golem.custom-runtime.enable" : false, + "golem.custom-runtime.config" : { + "value": 32 + } + }, + "config": { + "counters": { + "golem.usage.network.in-mib": { + "name": "in-network-traffic", + "description": "Incoming network traffic usage in MiB", + "price": true + }, + "golem.usage.network.out-mib": { + "name": "out-network-traffic", + "description": "Outgoing network traffic usage in MiB", + "price": true + }, + "golem.usage.duration_sec": { + "name": "duration", + "description": "Activity duration in seconds", + "price": true + } + } + } + } +] +``` + + +| property | optional | description | +|-----------------|----------|---------------------------------------------------------------------------------------------------------------------------------| +| name | No | Runtime name which will be placed in offer as `"golem.runtime.name"` | +| version | No | Runtime version following semantic versioning. Placed in Offer as `golem.runtime.version` | +| supervisor-path | No | Path to supervisor binary relative to this descriptor. | +| runtime-path | Yes | Path to runtime binary relative to this descriptor. | +| description | Yes | Human readable runtime description. | +| extra-args | Yes | Runtime specific arguments that will be appended to ExeUnit binary when starting. | +| properties | Yes | Properties that will be attached to Offer. Dictionary with keys used as a path in Offer which value can be any legal json type. | +| config | Yes | Runtime configuration that can be used by Provider. | +| config/counters | Yes | Dictionary of supported usage counters. | diff --git a/docs/provider/overview.md b/docs/provider/overview.md index e8bbb9aecf..06b2da9efe 100644 --- a/docs/provider/overview.md +++ b/docs/provider/overview.md @@ -22,13 +22,13 @@ To change default runtimes versions set env variables: Standard Provider installation uses files in following directories: -| name | ENV | command line | default configuration | description | comment | -| ----------------------- | ------------- | --------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Yagna data directory | YAGNA_DATADIR | yagna --datadir | ~/.local/share/yagna | Contains yagna daemon configuration and persistent files. | | -| Provider data directory | DATADIR | ya-provider --datadir | ~/.local/share/ya-provider | Provider agent configuration files, logs and ExeUnit directories. | | +| name | ENV | command line | default configuration | description | comment | +| ----------------------- | ------------- | --------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Yagna data directory | YAGNA_DATADIR | yagna --datadir | ~/.local/share/yagna | Contains yagna daemon configuration and persistent files. | | +| Provider data directory | DATADIR | ya-provider --datadir | ~/.local/share/ya-provider | Provider agent configuration files, logs and ExeUnit directories. | | | Runtimes directory | EXE_UNIT_PATH | ya-provider --exe-unit-path | ~/.local/lib/yagna/plugins/ya-*.json | Contains runtime binaries. | Regular expression pointing to ExeUnits descriptors (It's not directory). Warning:`golemsp` overrides this setting (issue: [#2689](https://github.com/golemfactory/yagna/issues/2689)). | -| ExeUnit cache | | | ~/.local/share/ya-provider/exe-unit/cache | Stores cached ExeUnit Runtime images. | | -| ExeUnit working dir | | | ~/.local/share/ya-provider/exe-unit/work | Directory used to store tasks data. For each Agreement ExeUnit creates directory named by Agreement Id. Inside there are directories created for each activity. VM runtime mounts image volumes inside this directory. | | -| Binaries | | | ~/.local/bin/yagna | Yagna daemon and agent binaries. | If yagna is already installed, installer will use previous directory instead. | -| Installer files | | | ~/.local/share/ya-installer | Directory used by installer to download files. | Files can be removed after installation is completed. | -| GSB unix socket | GSB_URL | | unix:/tmp/yagna.sock | Unix socket used by GSB for communication. | Can be configured to use TCP. | +| ExeUnit cache | | | ~/.local/share/ya-provider/exe-unit/cache | Stores cached ExeUnit Runtime images. | Always relative to Provider data directory `${YAGNA_DATADIR}/exe-unit/cache`. | +| ExeUnit working dir | | | ~/.local/share/ya-provider/exe-unit/work | Directory used to store tasks data. For each Agreement ExeUnit creates directory named by Agreement Id. Inside there are directories created for each activity. VM runtime mounts image volumes inside this directory. | Always relative to Provider data directory `${YAGNA_DATADIR}/exe-unit/work`. | +| Binaries | | | ~/.local/bin/yagna | Yagna daemon and agent binaries. | If yagna is already installed, installer will use previous directory instead. | +| Installer files | | | ~/.local/share/ya-installer | Directory used by installer to download files. | Files can be removed after installation is completed. | +| GSB unix socket | GSB_URL | | unix:/tmp/yagna.sock | Unix socket used by GSB for communication. | Can be configured to use TCP. | diff --git a/docs/yagna/capabilities.md b/docs/yagna/capabilities.md new file mode 100644 index 0000000000..2844de7103 --- /dev/null +++ b/docs/yagna/capabilities.md @@ -0,0 +1,11 @@ +# Yagna Capabilities List + +# Protocol capabilities + +Capabilities requiring Provider agent support are listed [here](../provider/capabilities.md). + +## Yagna API + +| Capability | Yagna package version | Backwards-compatible? | Description | +|-------------|-----------------------|-----------------------|----------------------------------------------------------------------------| +| Cors Policy | 0.12.0 | Yes | Yagna is able to respond with Cors headers. [Spec](./capabilities/cors.md) | diff --git a/docs/yagna/capabilities/cors.md b/docs/yagna/capabilities/cors.md new file mode 100644 index 0000000000..c907b82bcb --- /dev/null +++ b/docs/yagna/capabilities/cors.md @@ -0,0 +1,60 @@ +# Cors headers + +Why is it needed: + +- local tests by developers. +- convenient connection of applications in public arenas with the local golem node. + +### Examples + +#### Erigon + +**--http.corsdomain value** +Comma separated list of domains from which to accept cross origin requests (browser enforced) + +**--http.vhosts value** +Comma separated list of virtual hostnames from which to accept requests (server enforced). Accepts '*' wildcard. (default: "localhost") + +#### Lightouse + +***--http-allow-origin** +`` this server (e.g., http://localhost:5052). + +## Yagna Requirements + +### Ability to define default cors rules + +Add new commandline argument api-allow-origin (defined by env YAGNA_API_ALLOW_ORIGIN). + + +``` +$ yagna service run --api-allow-origin='*' +``` + + +### Rules per appkey + +Adding new appkey + +``` +$ yagna app-key create --id 0x578349a0d1dd825162fe8579a51efa220a9f4b17 --allow-origin https://dapps.golem.network dapp-portal +4f1cf7c363e9403b9ce15823cec182ff +$ yagna app-key list +┌───────────────┬──────────────────────────────────────────────┬───────────┬─────────────────────────────────┐ +│ name │ id │ role │ created │ +├───────────────┼──────────────────────────────────────────────┼───────────┼─────────────────────────────────┤ +│ dapp-portal │ 0x578349a0d1dd825162fe8579a51efa220a9f4b17 │ manager │ 2022-12-01T14:11:20.113596023 │ +└───────────────┴──────────────────────────────────────────────┴───────────┴─────────────────────────────────┘ +$ yagna app-key show dapp-portal +--- +name: dapp-portal +key: 4f1cf7c363e9403b9ce15823cec182ff +id: "0x578349a0d1dd825162fe8579a51efa220a9f4b17" +role: manager +created: 2022-12-01T14:11:20.113596023 +allowOrigin: + - https://dapps.golem.network +--- + + +