From c31b82f93749aff4c9c7c9f3408694f17455ba93 Mon Sep 17 00:00:00 2001 From: "nieznany.sprawiciel" Date: Tue, 31 Oct 2023 17:05:53 +0100 Subject: [PATCH 1/6] Documentation of ExeUnit directory and runtime descriptor --- docs/provider/exe-unit/exe-units.md | 82 +++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 docs/provider/exe-unit/exe-units.md diff --git a/docs/provider/exe-unit/exe-units.md b/docs/provider/exe-unit/exe-units.md new file mode 100644 index 0000000000..11573aa5d9 --- /dev/null +++ b/docs/provider/exe-unit/exe-units.md @@ -0,0 +1,82 @@ +# ExeUnits + +## How Provider agent finds ExeUnits? + +`ya-provider` lists available ExeUnits by listing descriptors placed in [directory](../overview.md#provider-directories). + +Example directory content: +```commandline +nieznanysprawiciel@laptop:~$ tree ~/.local/lib/yagna/plugins +/home/nieznanysprawiciel/.local/lib/yagna/plugins +├── exe-unit +├── ya-runtime-vm +│   ├── runtime +│   │   ├── bios-256k.bin +│   │   ├── efi-virtio.rom +│   │   ├── initramfs.cpio.gz +│   │   ├── kvmvapic.bin +│   │   ├── linuxboot_dma.bin +│   │   ├── self-test.gvmi +│   │   ├── vmlinuz-virt +│   │   └── vmrt +│   └── ya-runtime-vm +├── ya-runtime-vm.json +├── ya-runtime-wasi +└── ya-runtime-wasi.json +``` +ExeUnit descriptors must be placed directly in plugins directory (not in nested structure). +Other ExeUnits' files can be moved into subdirectories, because descriptor is pointing to specific files. + +## Descriptor + +Example ExeUnit descriptor: +```json +[ + { + "name": "custom", + "version": "0.2.2", + "supervisor-path": "exe-unit", + "runtime-path": "custom-runtime/custom", + "description": "Custom runtime for documentation purposes.", + "extra-args": ["--runtime-managed-image"], + "properties": { + "golem.custom-runtime.enable" : false, + "golem.custom-runtime.config" : { + "value": 32 + } + }, + "config": { + "counters": { + "golem.usage.network.in-mib": { + "name": "in-network-traffic", + "description": "Incoming network traffic usage in MiB", + "price": true + }, + "golem.usage.network.out-mib": { + "name": "out-network-traffic", + "description": "Outgoing network traffic usage in MiB", + "price": true + }, + "golem.usage.duration_sec": { + "name": "duration", + "description": "Activity duration in seconds", + "price": true + } + } + } + } +] +``` + + +| property | optional | description | +|-----------------|----------|---------------------------------------------------------------------------------------------------------------------------------| +| name | No | Runtime name which will be placed in offer as `"golem.runtime.name"` | +| version | No | Runtime version following semantic versioning. Placed in Offer as `golem.runtime.version` | +| supervisor-path | No | Path to supervisor binary relative to this descriptor. | +| runtime-path | Yes | Path to runtime binary relative to this descriptor. | +| description | Yes | Human readable runtime description. | +| extra-args | Yes | Runtime specific arguments that will be appended to ExeUnit binary when starting. | +| properties | Yes | Properties that will be attached to Offer. Dictionary with keys used as a path in Offer which value can be any legal json type. | +| config | Yes | Runtime configuration that can be used by Provider. | +| config/counters | Yes | Dictionary of supported usage counters. | From 3951750829dd89909d45310fc5baa889329cfcff Mon Sep 17 00:00:00 2001 From: "nieznany.sprawiciel" Date: Thu, 2 Nov 2023 15:30:26 +0100 Subject: [PATCH 2/6] Doc: ExeUnit cache and work is relative to provider datadir --- docs/provider/overview.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/provider/overview.md b/docs/provider/overview.md index e8bbb9aecf..06b2da9efe 100644 --- a/docs/provider/overview.md +++ b/docs/provider/overview.md @@ -22,13 +22,13 @@ To change default runtimes versions set env variables: Standard Provider installation uses files in following directories: -| name | ENV | command line | default configuration | description | comment | -| ----------------------- | ------------- | --------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Yagna data directory | YAGNA_DATADIR | yagna --datadir | ~/.local/share/yagna | Contains yagna daemon configuration and persistent files. | | -| Provider data directory | DATADIR | ya-provider --datadir | ~/.local/share/ya-provider | Provider agent configuration files, logs and ExeUnit directories. | | +| name | ENV | command line | default configuration | description | comment | +| ----------------------- | ------------- | --------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Yagna data directory | YAGNA_DATADIR | yagna --datadir | ~/.local/share/yagna | Contains yagna daemon configuration and persistent files. | | +| Provider data directory | DATADIR | ya-provider --datadir | ~/.local/share/ya-provider | Provider agent configuration files, logs and ExeUnit directories. | | | Runtimes directory | EXE_UNIT_PATH | ya-provider --exe-unit-path | ~/.local/lib/yagna/plugins/ya-*.json | Contains runtime binaries. | Regular expression pointing to ExeUnits descriptors (It's not directory). Warning:`golemsp` overrides this setting (issue: [#2689](https://github.com/golemfactory/yagna/issues/2689)). | -| ExeUnit cache | | | ~/.local/share/ya-provider/exe-unit/cache | Stores cached ExeUnit Runtime images. | | -| ExeUnit working dir | | | ~/.local/share/ya-provider/exe-unit/work | Directory used to store tasks data. For each Agreement ExeUnit creates directory named by Agreement Id. Inside there are directories created for each activity. VM runtime mounts image volumes inside this directory. | | -| Binaries | | | ~/.local/bin/yagna | Yagna daemon and agent binaries. | If yagna is already installed, installer will use previous directory instead. | -| Installer files | | | ~/.local/share/ya-installer | Directory used by installer to download files. | Files can be removed after installation is completed. | -| GSB unix socket | GSB_URL | | unix:/tmp/yagna.sock | Unix socket used by GSB for communication. | Can be configured to use TCP. | +| ExeUnit cache | | | ~/.local/share/ya-provider/exe-unit/cache | Stores cached ExeUnit Runtime images. | Always relative to Provider data directory `${YAGNA_DATADIR}/exe-unit/cache`. | +| ExeUnit working dir | | | ~/.local/share/ya-provider/exe-unit/work | Directory used to store tasks data. For each Agreement ExeUnit creates directory named by Agreement Id. Inside there are directories created for each activity. VM runtime mounts image volumes inside this directory. | Always relative to Provider data directory `${YAGNA_DATADIR}/exe-unit/work`. | +| Binaries | | | ~/.local/bin/yagna | Yagna daemon and agent binaries. | If yagna is already installed, installer will use previous directory instead. | +| Installer files | | | ~/.local/share/ya-installer | Directory used by installer to download files. | Files can be removed after installation is completed. | +| GSB unix socket | GSB_URL | | unix:/tmp/yagna.sock | Unix socket used by GSB for communication. | Can be configured to use TCP. | From 2cbd034b95bd35c306c60cec6c8d1d79bdf8c7a4 Mon Sep 17 00:00:00 2001 From: "nieznany.sprawiciel" Date: Thu, 2 Nov 2023 15:37:59 +0100 Subject: [PATCH 3/6] Docs: Improve ExeUnit directory descirption --- docs/provider/exe-unit/exe-units.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/provider/exe-unit/exe-units.md b/docs/provider/exe-unit/exe-units.md index 11573aa5d9..c9c640b0d1 100644 --- a/docs/provider/exe-unit/exe-units.md +++ b/docs/provider/exe-unit/exe-units.md @@ -24,7 +24,7 @@ nieznanysprawiciel@laptop:~$ tree ~/.local/lib/yagna/plugins ├── ya-runtime-wasi └── ya-runtime-wasi.json ``` -ExeUnit descriptors must be placed directly in plugins directory (not in nested structure). +ExeUnit descriptors (`ya-runtime-vm.json` and `ya-runtime-wasi.json`) must be placed directly in plugins directory (not in nested structure). Other ExeUnits' files can be moved into subdirectories, because descriptor is pointing to specific files. ## Descriptor From ff77587967328950734cdffbbab2b61edbc78d10 Mon Sep 17 00:00:00 2001 From: "nieznany.sprawiciel" Date: Wed, 18 Oct 2023 13:38:31 +0200 Subject: [PATCH 4/6] Move logging guidelines --- docs/logging-guidelines.md | 83 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 docs/logging-guidelines.md diff --git a/docs/logging-guidelines.md b/docs/logging-guidelines.md new file mode 100644 index 0000000000..338e530a5d --- /dev/null +++ b/docs/logging-guidelines.md @@ -0,0 +1,83 @@ +This page describes the logging mechanics and lists guidelines for clear logging. + +## Goal +There are 2 main goals of the logging: providing support and tracing down errors in development + +Logging with the support goal the is to inform semi-technical users about the state of the application. +This means all levels from INFO and up need to be clear to understand for everyone. + +Logging with the development goal is to find where errors originate from. This means every log message needs sufficient information to trace the application its steps. + +Use cases for the logs are: +- Support and development teams will receive an end-user log to investigate +- Run the application in DEBUG log-level to see what goes on in detail. + +## Levels +These are the log levels available to the logger: +- **CRITICAL** = app is unable to continue, it should exit after this message +- **ERROR** = app is unable to perform the requested action and stops trying +- **WARNING** = app is unable to perform the requested action but will continue to run, for instance try again later +- **INFO** = app performs a requested action +- **DEBUG** = gives extra details over other messages, always a following message + +## Logger initialization +A logger can be created at the top of the file, after all imports are completed like this: + +``` +... +import logging + +logger = logging.getLogger(__name__) +``` +- `logger` has to be the module level variable name. +- `__name__` has to be the logger name +- when there is a clear and good reason, `__name__` can be be replaced with: + - base level script like `golemapp` and `golemcli` + - 3rd part module name like `twisted` + - higher level module, like `golem.a.b` for `golem.a.b.c` + - help your future self and comment the reason + +## Log message structure +Log messages should start with a message that describes the action clearly. At the end there should be extra information, for example the operation, status or task_id. +``` +logger.info('Task has been started. id=%r', task_id) +logger.debug('task_header=%r', task_header) +``` +Debug messages are used to enrich the messages logged in higher levels. + +Some guidelines on what to ( and what not to ) log: +- Describe the action as short and clear as possible +- Always start with a message, print data as "`kwargs`" + - For example `Task deadline passed. id=%r, state=%r` +- Always print enough related data for the action you log +- Always use native logger formatting, no manual formatting. + - When a log-level is disabled there is no need to format all these strings. + - [PoC](https://gist.github.com/jiivan/eedb505574ccb2503af48fa2e3043036) +- "Silly" logs (multiline data, spam, json) should always be DEBUG +- Info all important milestones + - Listening ports + - Components load + - User interactions + - Task interactions + - Network interactions +- Include node names in messages, before node_id's +- Prefer to use `abrev` node_id ( `node_id[:8] + ‘…’ + node_id[-8:]` ) over the full id when you know there is sufficient data + - First contact ( subtask given, connected ) should print full id + - Repeating messages should print the abrev version (either `golem.core.common.node_info_str` or `golem.core.common.short_node_id`) + - Increase basic abrev function when network grows and 8 byte is not unique enough +- No JSON ( except DEBUG ) +- No Stack/Traceback ( on INFO or WARNING) + +## Defaults +By default the golem logger has 3 outputs: +- console -> level INFO +- `golem.log` -> level INFO, rotate daily, keep 5 +- `golem.error.log` -> level WARNING, no rotate + +External components are by default set to the WARNING level, since INFO logs are not always useful for our end users. + +## Arguments +`--log-level` is the argument to tweak the logs written by the application. When set it will change the INFO console and file handlers and all golem components. It shouldn't affect `golem.error.log`. + +`--loglevel` is an argument used by our 3rd party library crossbar, since this is a 3rd party log it only shows WARNING and up inside golem application logs. +Please note: Crossbar log level's are lower case, while golem log level's are upper case. Also `warn` in crossbar is equal to `WARNING` in golem. The rest of the levels are named the same. \ No newline at end of file From a6f1ad6faf9dddd8e9cc21fc2fa66fefd0500587 Mon Sep 17 00:00:00 2001 From: "nieznany.sprawiciel" Date: Wed, 18 Oct 2023 14:36:17 +0200 Subject: [PATCH 5/6] Move capabilities list; Add missing caps as todos --- docs/provider/capabilities.md | 11 ++++++ docs/yagna/capabilities.md | 11 ++++++ docs/yagna/capabilities/cors.md | 60 +++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+) create mode 100644 docs/provider/capabilities.md create mode 100644 docs/yagna/capabilities.md create mode 100644 docs/yagna/capabilities/cors.md diff --git a/docs/provider/capabilities.md b/docs/provider/capabilities.md new file mode 100644 index 0000000000..e203783c69 --- /dev/null +++ b/docs/provider/capabilities.md @@ -0,0 +1,11 @@ +# Yagna Capabilities List + +## Protocol + +| Capability | Yagna package version | Backwards-compatible? | Description | +|-------------------------------|-----------------------|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Multi-Activity Agreement | 0.5.0 | Yes | Negotiate ability to create multiple activities under single Agreement. Use (which?) property in Demand/Offer to indicate node's support for Multi-Activity. If counterparty does not support Multi-Activity, the node falls back to single Activity per Agreement behaviour. | +| Restart Proposal Negotiations | 0.7.0 | Yes | Agent is allowed to restart negotiations, by sending `Counter Proposal`, after he rejected Proposal at some point. Counter-party will receive regular `ProposalEvent` in this case. Only Agent rejecting Proposal has initiative in restarting negotiations, rejected Agent can only wait for this to happen. To indicate, that Proposal rejection isn't final and negotiations can be restarted later, Agent can set `golem.proposal.rejection.is-final` (bool) field in `Reason` structure. If this value is set to false, Agent can free any state related to this negotiation. The same field can be set in `Reason` sent in `Reject Agreement` operation. Requestor can send new counter Proposal after some period of time or propose the same Agreement for the second time. (No change to specification) | +| manifest-support | | | TODO | +| inet | | | TODO | +| start-entrypoint | | | TODO | diff --git a/docs/yagna/capabilities.md b/docs/yagna/capabilities.md new file mode 100644 index 0000000000..2844de7103 --- /dev/null +++ b/docs/yagna/capabilities.md @@ -0,0 +1,11 @@ +# Yagna Capabilities List + +# Protocol capabilities + +Capabilities requiring Provider agent support are listed [here](../provider/capabilities.md). + +## Yagna API + +| Capability | Yagna package version | Backwards-compatible? | Description | +|-------------|-----------------------|-----------------------|----------------------------------------------------------------------------| +| Cors Policy | 0.12.0 | Yes | Yagna is able to respond with Cors headers. [Spec](./capabilities/cors.md) | diff --git a/docs/yagna/capabilities/cors.md b/docs/yagna/capabilities/cors.md new file mode 100644 index 0000000000..c907b82bcb --- /dev/null +++ b/docs/yagna/capabilities/cors.md @@ -0,0 +1,60 @@ +# Cors headers + +Why is it needed: + +- local tests by developers. +- convenient connection of applications in public arenas with the local golem node. + +### Examples + +#### Erigon + +**--http.corsdomain value** +Comma separated list of domains from which to accept cross origin requests (browser enforced) + +**--http.vhosts value** +Comma separated list of virtual hostnames from which to accept requests (server enforced). Accepts '*' wildcard. (default: "localhost") + +#### Lightouse + +***--http-allow-origin** +`` this server (e.g., http://localhost:5052). + +## Yagna Requirements + +### Ability to define default cors rules + +Add new commandline argument api-allow-origin (defined by env YAGNA_API_ALLOW_ORIGIN). + + +``` +$ yagna service run --api-allow-origin='*' +``` + + +### Rules per appkey + +Adding new appkey + +``` +$ yagna app-key create --id 0x578349a0d1dd825162fe8579a51efa220a9f4b17 --allow-origin https://dapps.golem.network dapp-portal +4f1cf7c363e9403b9ce15823cec182ff +$ yagna app-key list +┌───────────────┬──────────────────────────────────────────────┬───────────┬─────────────────────────────────┐ +│ name │ id │ role │ created │ +├───────────────┼──────────────────────────────────────────────┼───────────┼─────────────────────────────────┤ +│ dapp-portal │ 0x578349a0d1dd825162fe8579a51efa220a9f4b17 │ manager │ 2022-12-01T14:11:20.113596023 │ +└───────────────┴──────────────────────────────────────────────┴───────────┴─────────────────────────────────┘ +$ yagna app-key show dapp-portal +--- +name: dapp-portal +key: 4f1cf7c363e9403b9ce15823cec182ff +id: "0x578349a0d1dd825162fe8579a51efa220a9f4b17" +role: manager +created: 2022-12-01T14:11:20.113596023 +allowOrigin: + - https://dapps.golem.network +--- + + + From 8c4c63c0c0800eb88787bda06279fa89a117396e Mon Sep 17 00:00:00 2001 From: nieznanysprawiciel Date: Mon, 6 Nov 2023 15:35:10 +0100 Subject: [PATCH 6/6] Apply suggestions from code review --- docs/logging-guidelines.md | 62 ----------------------------------- docs/provider/capabilities.md | 2 +- 2 files changed, 1 insertion(+), 63 deletions(-) diff --git a/docs/logging-guidelines.md b/docs/logging-guidelines.md index 338e530a5d..ffb1849046 100644 --- a/docs/logging-guidelines.md +++ b/docs/logging-guidelines.md @@ -19,65 +19,3 @@ These are the log levels available to the logger: - **WARNING** = app is unable to perform the requested action but will continue to run, for instance try again later - **INFO** = app performs a requested action - **DEBUG** = gives extra details over other messages, always a following message - -## Logger initialization -A logger can be created at the top of the file, after all imports are completed like this: - -``` -... -import logging - -logger = logging.getLogger(__name__) -``` -- `logger` has to be the module level variable name. -- `__name__` has to be the logger name -- when there is a clear and good reason, `__name__` can be be replaced with: - - base level script like `golemapp` and `golemcli` - - 3rd part module name like `twisted` - - higher level module, like `golem.a.b` for `golem.a.b.c` - - help your future self and comment the reason - -## Log message structure -Log messages should start with a message that describes the action clearly. At the end there should be extra information, for example the operation, status or task_id. -``` -logger.info('Task has been started. id=%r', task_id) -logger.debug('task_header=%r', task_header) -``` -Debug messages are used to enrich the messages logged in higher levels. - -Some guidelines on what to ( and what not to ) log: -- Describe the action as short and clear as possible -- Always start with a message, print data as "`kwargs`" - - For example `Task deadline passed. id=%r, state=%r` -- Always print enough related data for the action you log -- Always use native logger formatting, no manual formatting. - - When a log-level is disabled there is no need to format all these strings. - - [PoC](https://gist.github.com/jiivan/eedb505574ccb2503af48fa2e3043036) -- "Silly" logs (multiline data, spam, json) should always be DEBUG -- Info all important milestones - - Listening ports - - Components load - - User interactions - - Task interactions - - Network interactions -- Include node names in messages, before node_id's -- Prefer to use `abrev` node_id ( `node_id[:8] + ‘…’ + node_id[-8:]` ) over the full id when you know there is sufficient data - - First contact ( subtask given, connected ) should print full id - - Repeating messages should print the abrev version (either `golem.core.common.node_info_str` or `golem.core.common.short_node_id`) - - Increase basic abrev function when network grows and 8 byte is not unique enough -- No JSON ( except DEBUG ) -- No Stack/Traceback ( on INFO or WARNING) - -## Defaults -By default the golem logger has 3 outputs: -- console -> level INFO -- `golem.log` -> level INFO, rotate daily, keep 5 -- `golem.error.log` -> level WARNING, no rotate - -External components are by default set to the WARNING level, since INFO logs are not always useful for our end users. - -## Arguments -`--log-level` is the argument to tweak the logs written by the application. When set it will change the INFO console and file handlers and all golem components. It shouldn't affect `golem.error.log`. - -`--loglevel` is an argument used by our 3rd party library crossbar, since this is a 3rd party log it only shows WARNING and up inside golem application logs. -Please note: Crossbar log level's are lower case, while golem log level's are upper case. Also `warn` in crossbar is equal to `WARNING` in golem. The rest of the levels are named the same. \ No newline at end of file diff --git a/docs/provider/capabilities.md b/docs/provider/capabilities.md index e203783c69..e5ddaf928e 100644 --- a/docs/provider/capabilities.md +++ b/docs/provider/capabilities.md @@ -1,4 +1,4 @@ -# Yagna Capabilities List +# Provider agent capabilities List ## Protocol