Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cosmos] Revisit the encoding limitation of the newer version of cosmos #1033

Open
12 tasks
Olshansk opened this issue Jan 17, 2025 · 1 comment
Open
12 tasks
Assignees
Labels
on-chain On-chain business logic protocol General core protocol related changes tooling Tooling - CLI, scripts, helpers, off-chain, etc...

Comments

@Olshansk
Copy link
Member

Olshansk commented Jan 17, 2025

Objective

Understand what limitations the newer version of Cosmos imposes on us and adapt appropriately.

Origin Document

Image

Goals

  • Read, learn and understand the direction of the Cosmos SDK when it comes protobuf encoding
  • Follow best practices in how we are designing the structures (i.e. onchain state) and interfaces (gRPC queries & transactions) in poktroll
  • Prepare for the v0.52, v0.2 and beyond

Deliverables

  • Review the Zero Copy Encoding document from the Cosmos SDK:
  • API (gRPC) Quries & Transactions
  • Structures - Evaluate how we need to change any of our onchain structures to comply with:
    • Floats not being a supported scalar
    • Maps not being supported
    • Limitation of not having new fields
      The biggest change is that it will be invalid to add a new field to an existing message and a breaking change detector will need to be created which augments [buf breaking](https://docs.buf.build/breaking/overview) to detect this.
      
      The reasons for this are two-fold:
      
      1) from an API compatibility perspective, adding a new field to an existing message is actually a state machine breaking change which in [ADR 020](https://docs.cosmos.network/v0.52/build/architecture/adr-020-protobuf-transaction-encoding) required us to add an unknown field detector. Furthermore, in [ADR 054](https://docs.cosmos.network/v0.52/build/architecture/adr-054-semver-compatible-modules) this "feature" of protobuf poses one of the biggest problems for correct forward compatibility between different versions of the same module. 2) not allowing new fields in existing messages makes the generated code in languages like Rust (which is currently our highest priority target), much simpler and more performant because we can assume a fixed size struct gets allocated. If new fields can be added to existing messages, we need to encode the number of fields into the message and then do runtime checks. So this both increases memory layers and requires another layout of indirection. With the encoding proposed below, "plain old Rust structs" (used with some special field types) can be used.
      
      Instead of adding new fields to existing messages, APIs can add new messages to existing packages or create new packages with new versions of the messages. Also, we are not restricting the addition of cases to oneofs or values to enums. All of these cases are easier to detect at runtime with standard switch statements than the addition of new fields.
      

Non-goals / Non-deliverables

  • Forking the Cosmos SDK
  • Creating our own encoding rules / best practices
  • Trying to make the Cosmos SDK change based on our own opinions

General deliverables

  • Comments: Add/update TODOs and comments alongside the source code so it is easier to follow.
  • Testing: Add new tests (unit and/or E2E) to the test suite.
  • Makefile: Add new targets to the Makefile to make the new functionality easier to use.
  • Documentation: Update architectural or development READMEs; use mermaid diagrams where appropriate.

Creator: @Olshansk
Co-Owners: @red-0ne @bryanchriswhite

@Olshansk Olshansk added on-chain On-chain business logic protocol General core protocol related changes tooling Tooling - CLI, scripts, helpers, off-chain, etc... labels Jan 17, 2025
@Olshansk Olshansk added this to Shannon Jan 17, 2025
@github-project-automation github-project-automation bot moved this to 📋 Backlog in Shannon Jan 17, 2025
@Olshansk Olshansk self-assigned this Jan 17, 2025
@Olshansk Olshansk moved this from 📋 Backlog to 🔖 Ready in Shannon Jan 17, 2025
@Olshansk
Copy link
Member Author

For reference, here is the error that forced us to move rev_share from float to uint64: https://github.com/pokt-network/poktroll/actions/runs/12832899448/job/35787150432?pr=1028

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
on-chain On-chain business logic protocol General core protocol related changes tooling Tooling - CLI, scripts, helpers, off-chain, etc...
Projects
Status: 🔖 Ready
Development

No branches or pull requests

1 participant