added rate limiters and updated Readme

bobanetwork · Nov 17, 2023 · 0954f2a · 0954f2a
1 parent ec726f6
commit 0954f2a
Show file tree

Hide file tree

Showing 6 changed files with 126 additions and 89 deletions.
diff --git a/boba_community/fraud-detector/README.md b/boba_community/fraud-detector/README.md
@@ -6,11 +6,11 @@ description: Learn how to help detect operator fraud
 
 ## Basics
 
-The `boba_community/fraud-detector` repo contains Docker scripts and python source code for running a *Verifier*, a *DTL* (data transport layer), and a *fraud-detector* service. The allows you to:
+The `boba_community/fraud-detector` repo contains Docker scripts and python source code for running a _Verifier_, a _DTL_ (data transport layer), and a _fraud-detector_ service. The allows you to:
 
-1. Run your own Boba geth L2 on your computer. In this case, the geth L2 will run in its `Verifier` mode. In `Verifier` mode, the geth will sync from L1 and use the transaction data from the L1 contracts to compute what the state roots should be, *if the operator is honest*.
+1. Run your own Boba geth L2 on your computer. In this case, the geth L2 will run in its `Verifier` mode. In `Verifier` mode, the geth will sync from L1 and use the transaction data from the L1 contracts to compute what the state roots should be, _if the operator is honest_.
 
-2. A separate service, the *fraud-detector*, can then be used to discover potential fraud. Briefly, the fraud detection process consists of requesting a state root from Boba and requesting a state root from your Verifier. If those state roots match, then, the operator has been honest. If they do not match, then, that **might** be due to fraud, or, could also indicate indexing errors, timestamp errors, or chain configuration errors.
+2. A separate service, the _fraud-detector_, can then be used to discover potential fraud. Briefly, the fraud detection process consists of requesting a state root from Boba and requesting a state root from your Verifier. If those state roots match, then, the operator has been honest. If they do not match, then, that **might** be due to fraud, or, could also indicate indexing errors, timestamp errors, or chain configuration errors.
 
 The central idea is that if two (or more) geths injects the same transactions, then they should write the same blocks with the same state roots. If they don't, then there is a problem somewhere. Fundamentally, the security of rollups has little to do with math or cryptography - rather, security arises from the operator publicly depositing transactions and their corresponding state roots, and then, **having many independent nodes check those data for possible discrepancies**.
 
@@ -20,7 +20,7 @@ Congratulations! The security of the L2 depends on community monitoring of the o
 
 ## Running the Fraud Detector, the Verifier, and the Data Transport Layer (DTL) from local images
 
-**Requirements**: you will need a command line and Docker. Before filing GitHub issues, please make sure Docker is installed and *running*.
+**Requirements**: you will need a command line and Docker. Before filing GitHub issues, please make sure Docker is installed and _running_.
 
 **Open a terminal window**. First, clone the project and install needed dependencies:
 
@@ -30,11 +30,12 @@ $ cd boba
 $ yarn install
 $ yarn build
 $ cd ops
-$ docker-compose build
+$ docker compose build --no-cache
 ```
 
-Next, navigate to `boba_community/fraud-detector` and set the RELEASE_VERSION environment variable:
+Please note that to ensure a fresh build you will have to stop existing docker containers and remove existing docker images that were previously built.
 
+Next, navigate to `boba_community/fraud-detector` and set the RELEASE_VERSION environment variable:
 
 ```
 $ cd boba_community/fraud-detector
@@ -56,7 +57,7 @@ x-l1_node_web3_url: &l1_node_web3_url
 Next, spin up the `Fraud Detector` and other neccessary services (the `Verifier L2 Geth` and the `Data Transport Layer`)
 
 ```
-$ docker-compose up
+$ docker compose up
 ```
 
 Finally, **Open another terminal window** and upload the `addresses.json` to the `data transport layer` service.
@@ -128,8 +129,8 @@ verifier_dtl_1     | {"level":30,"time":1636134645380,"highestSyncedL1Block":135
 
 ## Known Errors and State Root Mismatches in Boba-V1
 
-* This directory contains a "docker-compose-v1_mainnet.yml" file which is configured to process the original chain prior to the October 2021 regenesis event. The DTL and l2geth images supporting this era are available from dockerhub, or may be built from the https://github.com/omgnetwork/optimism repository. The fraud-detector may be built from this repository or (if available) from a dockerhub image built after May 2023 which includes support for the V1 name of the OVM_StateCommitmentChain.
+- This directory contains a "docker-compose-v1_mainnet.yml" file which is configured to process the original chain prior to the October 2021 regenesis event. The DTL and l2geth images supporting this era are available from dockerhub, or may be built from the https://github.com/omgnetwork/optimism repository. The fraud-detector may be built from this repository or (if available) from a dockerhub image built after May 2023 which includes support for the V1 name of the OVM_StateCommitmentChain.
 
-* For the first 10 blocks of the V1 chain (between L1 heights of 13011896 and 13502893), the chainID was set (incorrectly) to 28 rather than 288. Therefore, the EIP155 signatures fail for those blocks, and the Verifier cannot sync those blocks. This has been addressed by overriding the chain ID of those blocks in a modified DTL (the rc1.0-surgery tag). In the fraud-detector log, these 10 blocks will show a mismatch but the stateroots should re-synchronize at block 11.
+- For the first 10 blocks of the V1 chain (between L1 heights of 13011896 and 13502893), the chainID was set (incorrectly) to 28 rather than 288. Therefore, the EIP155 signatures fail for those blocks, and the Verifier cannot sync those blocks. This has been addressed by overriding the chain ID of those blocks in a modified DTL (the rc1.0-surgery tag). In the fraud-detector log, these 10 blocks will show a mismatch but the stateroots should re-synchronize at block 11.
 
-* There is one state root mismatch at L2 block 155, arising from a two second discrepancy in a timestamp, that was ultimately caused by a too-small setting for the number of confirmations (DATA_TRANSPORT_LAYER__CONFIRMATIONS). This value was therefore increased. This is also handled by the rc1.0-surgery DTL.
+- There is one state root mismatch at L2 block 155, arising from a two second discrepancy in a timestamp, that was ultimately caused by a too-small setting for the number of confirmations (DATA_TRANSPORT_LAYER\_\_CONFIRMATIONS). This value was therefore increased. This is also handled by the rc1.0-surgery DTL.
diff --git a/boba_community/fraud-detector/docker-compose-goerli.yml b/boba_community/fraud-detector/docker-compose-goerli.yml
@@ -1,4 +1,4 @@
-version: "3"
+version: '3'
 
 x-l1_rpc_dtl: &l1_rpc_dtl
   DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT: https://rpc.ankr.com/eth_goerli
@@ -10,7 +10,7 @@ services:
   verifier_dtl:
     image: bobanetwork/data-transport-layer:${RELEASE_VERSION}
     env_file:
-      -  ../../ops/envs/dtl.env
+      - ../../ops/envs/dtl.env
     environment:
       <<: *l1_rpc_dtl
       DATA_TRANSPORT_LAYER__CONFIRMATIONS: 8
@@ -27,10 +27,10 @@ services:
       - ./state-dumps/goerli/:/opt/optimism/packages/data-transport-layer/state-dumps/
     #  - ./<YOUR PERSISTED STORAGE>:/db/
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${DTL_PORT:-7878}:7878
       - ${REGISTRY_PORT:-8080}:8081
@@ -54,28 +54,28 @@ services:
       ROLLUP_VERIFIER_ENABLE: 'true'
       RETRIES: 1000
       # no need to keep this secret, only used internally to sign blocks
-      BLOCK_SIGNER_KEY: "6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27"
-      BLOCK_SIGNER_ADDRESS: "0x00000398232E2064F896018496b4b44b3D62751F"
-      ROLLUP_POLL_INTERVAL_FLAG: "10s"
+      BLOCK_SIGNER_KEY: '6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27'
+      BLOCK_SIGNER_ADDRESS: '0x00000398232E2064F896018496b4b44b3D62751F'
+      ROLLUP_POLL_INTERVAL_FLAG: '10s'
       ROLLUP_ENFORCE_FEES: 'true'
       # turing
-      TURING_CREDIT_ADDRESS: "0x4200000000000000000000000000000000000020"
+      TURING_CREDIT_ADDRESS: '0x4200000000000000000000000000000000000020'
       # fee token
-      L2_BOBA_TOKEN_ADDRESS: "0x4200000000000000000000000000000000000023"
-      BOBA_GAS_PRICE_ORACLE_ADDRESS: "0x4200000000000000000000000000000000000024"
+      L2_BOBA_TOKEN_ADDRESS: '0x4200000000000000000000000000000000000023'
+      BOBA_GAS_PRICE_ORACLE_ADDRESS: '0x4200000000000000000000000000000000000024'
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${VERIFIER_HTTP_PORT:-8547}:8545
       - ${VERIFIER_WS_PORT:-8548}:8546
 
   fraud-detector:
     depends_on:
-     - verifier_dtl
-     - verifier_l2geth
+      - verifier_dtl
+      - verifier_l2geth
     image: bobanetwork/fraud-detector:${RELEASE_VERSION}
     deploy:
       replicas: 1
@@ -88,13 +88,14 @@ services:
       ADDRESS_MANAGER_ADDRESS: '0x6FF9c8FF8F0B6a0763a3030540c21aFC721A9148'
       L1_DEPLOYMENT_BLOCK: 7867494
       L2_START_BLOCK: 1
+      RATE_LIMITER_MAX_CALLS: 5
+      RATE_LIMITER_PERIOD: 1
     #volumes:
     #  - ./<YOUR PERSISTED STORAGE>:/db/
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${FRAUD_CHECKER_HTTP_PORT:-8555}:8555
-
diff --git a/boba_community/fraud-detector/docker-compose-v1_mainnet.yml b/boba_community/fraud-detector/docker-compose-v1_mainnet.yml
@@ -1,4 +1,4 @@
-version: "3"
+version: '3'
 
 x-l1_rpc_dtl: &l1_rpc_dtl
   DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT: 'https://mainnet.gateway.tenderly.co'
@@ -10,7 +10,7 @@ services:
   verifier_dtl:
     image: omgx/data-transport-layer:rc1.0-surgery
     env_file:
-      -  ../../ops/envs/dtl.env
+      - ../../ops/envs/dtl.env
     environment:
       <<: *l1_rpc_dtl
       DATA_TRANSPORT_LAYER__CONFIRMATIONS: 8
@@ -24,10 +24,10 @@ services:
       - ./state-dumps/v1_mainnet/:/opt/optimism/packages/data-transport-layer/state-dumps/
     #  - ./<YOUR PERSISTED STORAGE>:/db/
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${DTL_PORT:-7878}:7878
       - ${REGISTRY_PORT:-8080}:8081
@@ -51,31 +51,31 @@ services:
       ROLLUP_VERIFIER_ENABLE: 'true'
       RETRIES: 60
       # no need to keep this secret, only used internally to sign blocks
-      BLOCK_SIGNER_KEY: "6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27"
-      BLOCK_SIGNER_ADDRESS: "0x00000398232E2064F896018496b4b44b3D62751F"
-      ROLLUP_POLL_INTERVAL_FLAG: "10s"
+      BLOCK_SIGNER_KEY: '6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27'
+      BLOCK_SIGNER_ADDRESS: '0x00000398232E2064F896018496b4b44b3D62751F'
+      ROLLUP_POLL_INTERVAL_FLAG: '10s'
       ROLLUP_ENFORCE_FEES: 'true'
       DEV: true
       CHAIN_ID: 288
       NETWORK_ID: 288
-      ETH1_L1_CROSS_DOMAIN_MESSENGER_ADDRESS: "0x6D4528d192dB72E282265D6092F4B872f9Dff69e"
-      ROLLUP_ADDRESS_MANAGER_OWNER_ADDRESS: "0x1f2414D0af8741Bc822dBc2f88069c2b2907a840"
-      ETH1_L1_STANDARD_BRIDGE_ADDRESS: "0xdc1664458d2f0B6090bEa60A8793A4E66c2F1c00"
+      ETH1_L1_CROSS_DOMAIN_MESSENGER_ADDRESS: '0x6D4528d192dB72E282265D6092F4B872f9Dff69e'
+      ROLLUP_ADDRESS_MANAGER_OWNER_ADDRESS: '0x1f2414D0af8741Bc822dBc2f88069c2b2907a840'
+      ETH1_L1_STANDARD_BRIDGE_ADDRESS: '0xdc1664458d2f0B6090bEa60A8793A4E66c2F1c00'
       ETH1_L1_FEE_WALLET_ADDRESS: 0xbF159Ba5E5917551c70E377ADf21eD3736209fCE
       ROLLUP_GAS_PRICE_ORACLE_OWNER_ADDRESS: 0xd86D22c02E301BE7C35e3Ef20962f614cAf32B76
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${VERIFIER_HTTP_PORT:-8547}:8545
       - ${VERIFIER_WS_PORT:-8548}:8546
 
   fraud-detector:
     depends_on:
-     - verifier_dtl
-     - verifier_l2geth
+      - verifier_dtl
+      - verifier_l2geth
     image: bobanetwork/fraud-detector:${RELEASE_VERSION}
     deploy:
       replicas: 1
@@ -89,12 +89,14 @@ services:
       L1_DEPLOYMENT_BLOCK: 13011896
       L2_START_BLOCK: 1
       SCC_NAME: 'OVM_StateCommitmentChain'
+      RATE_LIMITER_MAX_CALLS: 5
+      RATE_LIMITER_PERIOD: 1
     #volumes:
     #  - ./<YOUR PERSISTED STORAGE>:/db/
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${FRAUD_CHECKER_HTTP_PORT:-8555}:8555
diff --git a/boba_community/fraud-detector/docker-compose.yml b/boba_community/fraud-detector/docker-compose.yml
@@ -1,4 +1,4 @@
-version: "3"
+version: '3'
 
 x-l1_rpc_dtl: &l1_rpc_dtl
   DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT: 'https://mainnet.gateway.tenderly.co'
@@ -10,7 +10,7 @@ services:
   verifier_dtl:
     image: bobanetwork/data-transport-layer:${RELEASE_VERSION}
     env_file:
-      -  ../../ops/envs/dtl.env
+      - ../../ops/envs/dtl.env
     environment:
       <<: *l1_rpc_dtl
       DATA_TRANSPORT_LAYER__CONFIRMATIONS: 8
@@ -27,10 +27,10 @@ services:
       - ./state-dumps/mainnet/:/opt/optimism/packages/data-transport-layer/state-dumps/
     #  - ./<YOUR PERSISTED STORAGE>:/db/
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${DTL_PORT:-7878}:7878
       - ${REGISTRY_PORT:-8080}:8081
@@ -54,28 +54,28 @@ services:
       ROLLUP_VERIFIER_ENABLE: 'true'
       RETRIES: 60
       # no need to keep this secret, only used internally to sign blocks
-      BLOCK_SIGNER_KEY: "6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27"
-      BLOCK_SIGNER_ADDRESS: "0x00000398232E2064F896018496b4b44b3D62751F"
-      ROLLUP_POLL_INTERVAL_FLAG: "10s"
+      BLOCK_SIGNER_KEY: '6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27'
+      BLOCK_SIGNER_ADDRESS: '0x00000398232E2064F896018496b4b44b3D62751F'
+      ROLLUP_POLL_INTERVAL_FLAG: '10s'
       ROLLUP_ENFORCE_FEES: 'true'
       # turing
-      TURING_CREDIT_ADDRESS: "0xF8D2f1b0292C0Eeef80D8F47661A9DaCDB4b23bf"
+      TURING_CREDIT_ADDRESS: '0xF8D2f1b0292C0Eeef80D8F47661A9DaCDB4b23bf'
       # fee token
-      L2_BOBA_TOKEN_ADDRESS: "0xa18bF3994C0Cc6E3b63ac420308E5383f53120D7"
-      BOBA_GAS_PRICE_ORACLE_ADDRESS: "0xeE06ee2F239d2ab11792D77f3C347d919ddA0d51"
+      L2_BOBA_TOKEN_ADDRESS: '0xa18bF3994C0Cc6E3b63ac420308E5383f53120D7'
+      BOBA_GAS_PRICE_ORACLE_ADDRESS: '0xeE06ee2F239d2ab11792D77f3C347d919ddA0d51'
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${VERIFIER_HTTP_PORT:-8547}:8545
       - ${VERIFIER_WS_PORT:-8548}:8546
 
   fraud-detector:
     depends_on:
-     - verifier_dtl
-     - verifier_l2geth
+      - verifier_dtl
+      - verifier_l2geth
     image: bobanetwork/fraud-detector:${RELEASE_VERSION}
     deploy:
       replicas: 1
@@ -88,12 +88,14 @@ services:
       ADDRESS_MANAGER_ADDRESS: '0x8376ac6C3f73a25Dd994E0b0669ca7ee0C02F089'
       L1_DEPLOYMENT_BLOCK: 13502893
       L2_START_BLOCK: 1
+      RATE_LIMITER_MAX_CALLS: 5
+      RATE_LIMITER_PERIOD: 1
     #volumes:
     #  - ./<YOUR PERSISTED STORAGE>:/db/
     logging:
-      driver: "json-file"
+      driver: 'json-file'
       options:
-        max-file: "5"
-        max-size: "10m"
+        max-file: '5'
+        max-size: '10m'
     ports:
       - ${FRAUD_CHECKER_HTTP_PORT:-8555}:8555
diff --git a/boba_community/fraud-detector/docker/Dockerfile.fraud-detector b/boba_community/fraud-detector/docker/Dockerfile.fraud-detector
@@ -1,5 +1,6 @@
 FROM python:3.8-slim
 RUN pip3 install --no-cache-dir web3==6.11.1
+RUN pip3 install --no-cache-dir ratelimiter==1.2.0.post0
 COPY boba_community/fraud-detector/fraud-detector.py /
 COPY boba_community/fraud-detector/packages/jsonrpclib /jsonrpclib
 COPY /packages/contracts/artifacts/contracts/L1/rollup/StateCommitmentChain.sol/StateCommitmentChain.json /contracts/StateCommitmentChain.json