feat(core): Plonky2 proofs onchain verification (part 1) #4442

ekovalev · 2025-01-15T19:01:58Z

Overview

This is the first part of a bigger change (#4393 ) in order to break down the original PR into several more or less independent and tractable parts. 
The ultimate goal is to provide a framework for zk-related apps on top of (currently) Vara blockchain. Specifically, it supports on-chain verification of Plonky2 proofs for arbitrary circuits. This would require speeding up of computationally greedy tasks in order to ensure any verify operation fits in a single block. 
The big picture looks like so:

                            +-------------------------------------------------------------------------+
                            |                                                          WASM           |
+------------+              |   +---------------------------+                ( Gear programs realm )  |
|            |     proof    |   |   Application Contract    |                                         |
|   Prover   | -------------|-> |                           |                                         |
|            |              |   |  (verifier_circuit_data)  |                                         |
+------------+              |   +---------------------------+                                         |
                            |                 |                                                       |
                            |                 | concatenate( verifier_circuit_data, proof )           |
                            |                 v                                                       |
                            |   +---------------------------+                                         |
                            |   |     Verifier Contract     |                                         |
                            |   |                           |         +---------------------------+   |
                            |   |     plonky2::verify()     |         |   GoldilocksField  impl   |   |
                            |   |       |                   |         |                           |   |
                            |   |       +- GF::poseidon() --+-------> |    Poseidon::permute()    |   |
                            |   +---------------------------+         +-------------|-------------+   |
                            +-------------------------------------------------------|-----------------+
                                                                                    | syscall
                                                                                    v
                                                                      +---------------------------+
                                                                      |        Vara runtime       |
                                                                      |                           |
                                                                      |        gr_permute()       |
                                                                      +-------------|-------------+
                                                                                    | host call
                                                                                    v
                                                                      +---------------------------+
                                                                      |           Client          |
                                                                      |                           |
                                                                      |  permute primitives impl  |
                                                                      +---------------------------+

Most of the workflow happens in the user-space. We define a set of contracts with (potentially) well-known addresses:

an Application contract contains the main app logic, and, therefore, stores the circuit-specific data; every proof sent to it for verification is prepended with the encoded circuit data;
the Verifier contract (can be a single instance for the entire chain) wraps the original Plonky2 code for proofs verification; however, the underlying primitives (GoldilocksField) are custom-implemented in such a way that heavy operations are offloaded to the host (through a chain of a syscall->host call).

The runtime part includes a syscall implementation to export heavy computing outside of WASM environment, as well as a runtime-interface to proxy a call to the host.

On the host we only maintain a self-sustained set of primitives (independent of the Plonky implementation) that ensure a client will always be able to carry out the Poseidon permute operation without requiring additional dependencies.

Changes

 This PR only concerns the runtime part and the “user-space” part for the sake of keeping the PR size under control:

A syscall for Poseidon permutation computation;
Example verifier contract with a custom GoldilocksField implementation to plug into the out-of-the-box plonky2 verifier;
Runtime tests.

The host call is replaced with a mock - the actual host call comes in a separate PR.

StackOverflowExcept1on · 2025-01-18T12:48:15Z

utils/wasm-proc/src/main.rs

@@ -114,6 +114,8 @@ const RT_ALLOWED_IMPORTS: [&str; 76] = [
    // From `GearBls12_381`
    "ext_gear_bls_12_381_aggregate_g1_version_1",
    "ext_gear_bls_12_381_map_to_g2affine_version_1",
+    // From `PoseidonHash`
+    "ext_poseidon_hash_poseidon_version_1",


mb rename to ext_gear_poseidon_hash_poseidon_version_1? like GearBls12_381 does

Makes sense

mertwole

Generally, LGTM

mertwole · 2025-01-20T12:13:50Z

examples/plonky2-verifier/src/goldilocks_field.rs

+    fn order() -> BigUint {
+        Self::ORDER.into()
+    }
+    fn characteristic() -> BigUint {
+        Self::order()
+    }


Suggested change

fn order() -> BigUint {

Self::ORDER.into()

}

fn characteristic() -> BigUint {

Self::order()

}

fn order() -> BigUint {

GoldilocksField::order()

}

fn characteristic() -> BigUint {

GoldilocksField::characteristic()

}

mertwole · 2025-01-20T12:16:41Z

examples/plonky2-verifier/src/goldilocks_field.rs

+    ///
+    /// The following code has been adapted from
+    /// winterfell/math/src/field/f64/mod.rs located at <https://github.com/facebook/winterfell>.
+    fn try_inverse(&self) -> Option<Self> {


Cannot GoldilocksField::try_inverse be reused here?

Absolutely! Good catch

mertwole · 2025-01-20T12:21:04Z

examples/plonky2-verifier/src/goldilocks_field.rs

+        // Using the fact that `GoldilocksFieldWrapper` is a newtype around `u64`.
+        let data: [u64; 12] = unsafe { core::mem::transmute(input) };


Maybe input.map(...)? I'm sure that it will be no-op in the compiled code but will not require unsafe

Clearly, unsafe is safe in this case.
If we replace type transmutation with a map, some overhead will definitely be there (at least in terms of the code size): compare. And in a contract the size is crucial.

Not sure if it has any runtime impact, but definitely this doesn't make anything better, in my opinion.

mertwole · 2025-01-20T12:30:18Z

pallets/gear/src/benchmarking/tests/syscalls_integrity.rs

+    vec![SyscallName::PoseidonPermute].into_iter().for_each(|syscall| {
+    // SyscallName::all().for_each(|syscall| {


Seems like previous version is correct

mertwole · 2025-01-20T12:31:31Z

pallets/gear/src/benchmarking/tests/syscalls_integrity.rs

+{
+    run_tester::<T, _, _, T::AccountId>(|_, _| {
+        let input = [1_u64; 12];
+        // TODO: uncomment when the actual host implementatoin of Poseidon permute is ready


Will it be implemented in the following PR?

Yes. It has already been implemented and tested. Goes in a separate PR for the sake of the changes readability.

mertwole · 2025-01-20T12:36:20Z

gcore/src/exec.rs

+/// Perform input data transformation and return data in the same format.
+/// Primarily designed for the Poseidon permutation of Goldilocks field arrays,


Not very clear - isn't this fn only for poseidon permutation?

You're right. The description must be a leftover from an earlier version, apparently.

mertwole · 2025-01-20T12:36:54Z

gcore/src/utils.rs

+// pub(crate) trait AsRawPtr: AsRef<[u8]> + AsMut<[u8]> {
+//     fn as_ptr(&self) -> *const [u8; 32] {
+//         self.as_ref().as_ptr() as *const _
+//     }
+
+//     fn as_mut_ptr(&mut self) -> *mut [u8; 32] {
+//         self.as_mut().as_mut_ptr() as *mut _
+//     }
+// }


Suggested change

// pub(crate) trait AsRawPtr: AsRef<[u8]> + AsMut<[u8]> {

// fn as_ptr(&self) -> *const [u8; 32] {

// self.as_ref().as_ptr() as *const _

// }

// fn as_mut_ptr(&mut self) -> *mut [u8; 32] {

// self.as_mut().as_mut_ptr() as *mut _

// }

// }

…imes

ekovalev added A0-pleasereview PR is ready to be reviewed by the team D1-core Gear Core labels Jan 15, 2025

ekovalev requested review from mertwole and breathx January 15, 2025 19:01

ekovalev force-pushed the ek-plonky2-verification-part-1 branch 2 times, most recently from fa3b359 to 971d106 Compare January 17, 2025 22:09

StackOverflowExcept1on reviewed Jan 18, 2025

View reviewed changes

mertwole reviewed Jan 20, 2025

View reviewed changes

ekovalev force-pushed the ek-plonky2-verification-part-1 branch 2 times, most recently from 35ab836 to 0e854c9 Compare January 23, 2025 13:26

ekovalev mentioned this pull request Jan 23, 2025

feat(node): Plonky2 proofs onchain verification (part 2) #4465

Open

ekovalev and others added 13 commits January 24, 2025 12:47

Plonky2 proofs verification

d7aad4e

Move custom Goldilocks field implementation into the verifier contract

89c2497

Mock the GoldilocksField and Poseidon impl on the host

1693c3b

fix(crates-io): publish of core-processor with new dependency (#4429)

4cbcbba

Temporarily disable a test that is expected to fail

e7aada4

Rename syscall

99f8cb9

Use gear-runtime-interface in core-processor only for non-ethexe runt…

f2caecb

…imes

Rename crate with host primitives

48e810f

Adapt Poseidon permute syscall tests to mocked host implementation

4e7d743

Rename runtime interface

d77a0a3

Streamline GoldilocksField implementation in a verifier's contract

94b665b

Tidy up

98ce75c

Undo changes to the node, mock the runtime interface

ff75a21

ekovalev force-pushed the ek-plonky2-verification-part-1 branch from 0e854c9 to ff75a21 Compare January 24, 2025 11:47

ekovalev changed the title ~~feat(core, node): Plonky2 proofs onchain verification (part 1)~~ feat(core): Plonky2 proofs onchain verification (part 1) Jan 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(core): Plonky2 proofs onchain verification (part 1) #4442

feat(core): Plonky2 proofs onchain verification (part 1) #4442

ekovalev commented Jan 15, 2025 •

edited

Loading

StackOverflowExcept1on Jan 18, 2025

ekovalev Jan 20, 2025

mertwole left a comment

mertwole Jan 20, 2025

mertwole Jan 20, 2025

ekovalev Jan 21, 2025

mertwole Jan 20, 2025

ekovalev Jan 21, 2025 •

edited

Loading

mertwole Jan 20, 2025

mertwole Jan 20, 2025

ekovalev Jan 21, 2025 •

edited

Loading

mertwole Jan 20, 2025

ekovalev Jan 21, 2025

mertwole Jan 20, 2025

		// Using the fact that `GoldilocksFieldWrapper` is a newtype around `u64`.
		let data: [u64; 12] = unsafe { core::mem::transmute(input) };

		vec![SyscallName::PoseidonPermute].into_iter().for_each(\|syscall\| {
		// SyscallName::all().for_each(\|syscall\| {

		/// Perform input data transformation and return data in the same format.
		/// Primarily designed for the Poseidon permutation of Goldilocks field arrays,

feat(core): Plonky2 proofs onchain verification (part 1) #4442

Are you sure you want to change the base?

feat(core): Plonky2 proofs onchain verification (part 1) #4442

Conversation

ekovalev commented Jan 15, 2025 • edited Loading

Overview

Changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mertwole left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekovalev Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekovalev Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekovalev commented Jan 15, 2025 •

edited

Loading

ekovalev Jan 21, 2025 •

edited

Loading

ekovalev Jan 21, 2025 •

edited

Loading