Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: mock HTTPS outcalls in state machine tests #89

Merged
merged 30 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0a46cd2
Refactor to use same HTTP request logic in JSON- and Candid-RPC endpo…
rvanasa Nov 20, 2023
6761c5f
Set up 'MockOutcall'
rvanasa Nov 20, 2023
1ea698a
Set up HTTPS outcall mocking in state machine tests
rvanasa Nov 21, 2023
db29161
Add 'mock' feature to generated Wasm file
rvanasa Nov 21, 2023
b59d80a
Misc
rvanasa Nov 21, 2023
ec26702
Improve readability of JSON literals
rvanasa Nov 21, 2023
62fa023
Use mocking for both test env and 'mock' flag
rvanasa Nov 21, 2023
f447105
Refactor
rvanasa Nov 21, 2023
3d96260
Rename 'MessageFlow' -> 'CallFlow'
rvanasa Nov 21, 2023
c17764e
Appease type system
rvanasa Nov 21, 2023
cbfef1e
Add 'no pending HTTP request' expect message
rvanasa Nov 21, 2023
fbad30d
Add initial cycles to canister in state machine tests
rvanasa Nov 21, 2023
d27496c
Implement HTTP request mocking
rvanasa Nov 21, 2023
dc7d182
Refactor
rvanasa Nov 21, 2023
c3942d6
Simplify HTTP outcall testing logic
rvanasa Nov 21, 2023
d1515ac
Merge branch 'main' of https://github.com/internet-computer-protocol/…
rvanasa Nov 21, 2023
6ab1cdd
Remove unused Cargo dependency
rvanasa Nov 21, 2023
f8671b9
Reformat
rvanasa Nov 21, 2023
280a274
Fix linter warnings
rvanasa Nov 21, 2023
56fe2fb
Rename local variable
rvanasa Nov 21, 2023
e808116
Simplify
rvanasa Nov 21, 2023
a4c9135
Merge branch 'main' into mock-https-outcalls
rvanasa Nov 22, 2023
c8142a9
Update unit test names
rvanasa Nov 22, 2023
1fcd52e
Update src/http.rs
rvanasa Nov 22, 2023
6626872
Test each individual part of mock HTTPS outcalls
rvanasa Nov 22, 2023
bad4f78
Refactor
rvanasa Nov 22, 2023
529351f
Add 'should_request_succeed_with_url' test
rvanasa Nov 22, 2023
b2de719
Test JSON canonicalization
rvanasa Nov 22, 2023
a75b741
Merge branch 'mock-https-outcalls' of https://github.com/internet-com…
rvanasa Nov 22, 2023
c83f855
Adjust HTTPS outcall transform source code
rvanasa Nov 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 27 additions & 26 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 1 addition & 8 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,19 +38,12 @@ async-trait = "0.1"
hex = "0.4"

[dev-dependencies]
# assert_matches = "1.5.0"
# ethers-core = "2.0.8"
ic-ic00-types = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
ic-base-types = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
ic-config = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
# ic-crypto-test-utils-reproducible-rng = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
# ic-icrc1-ledger = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
ic-state-machine-tests = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
ic-test-utilities-load-wasm = { git = "https://github.com/dfinity/ic", rev = "release-2023-09-27_23-01" }
# maplit = "1"
# proptest = "1.0"
# rand = "0.8"
# scraper = "0.17.1"
assert_matches = "1.5"

[workspace.dependencies]
candid = { version = "0.9", features = ["parser"] }
Expand Down
4 changes: 2 additions & 2 deletions lib/rust/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ pub use ethers_core as core;

pub use rpc::{call_contract, get_provider, request};

#[ic_cdk_macros::query(name = "__transform_ic_evm_rpc")]
pub fn transform_evm_rpc(args: TransformArgs) -> HttpResponse {
#[ic_cdk_macros::query(name = "__ic_evm_transform_json_rpc")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dumb question: why have another name as the method's name (__ic_evm_transform_json_rpc vs transform_json_rpc)

Copy link
Collaborator Author

@rvanasa rvanasa Nov 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code above is from a separate Rust library (in lib/rust) which was intended as an alternative to calling the EVM RPC canister. At one point, the canister used this library to perform JSON-RPC requests, but this has changed for the current MVP design.

__ic_evm_transform_json_rpc is provided by the ic_evm library whereas __transform_json_rpc is specific to the canister. They're essentially doing the same thing, so I used arbitrarily different names to distinguish these from each other. I'd be happy to change the names if we can come up with something better.

pub fn transform_json_rpc(args: TransformArgs) -> HttpResponse {
HttpResponse {
status: args.response.status,
body: args.response.body,
Expand Down
2 changes: 1 addition & 1 deletion lib/rust/src/rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ pub async fn request<'a, T: Serialize>(
headers: request_headers,
body: Some(json_rpc_payload.as_bytes().to_vec()),
transform: Some(TransformContext::from_name(
"__transform_ic_evm_rpc".to_string(),
"__ic_evm_transform_json_rpc".to_string(),
vec![],
)),
};
Expand Down
1 change: 1 addition & 0 deletions src/constants.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ pub const DEFAULT_ETHEREUM_PROVIDER: EthereumProvider = EthereumProvider::Ankr;
pub const DEFAULT_SEPOLIA_PROVIDER: SepoliaProvider = SepoliaProvider::PublicNode;

pub const CONTENT_TYPE_HEADER: &str = "Content-Type";
pub const CONTENT_TYPE_VALUE: &str = "application/json";

pub const ETH_MAINNET_CHAIN_ID: u64 = 1;
pub const ETH_SEPOLIA_CHAIN_ID: u64 = 11155111;
Expand Down
21 changes: 15 additions & 6 deletions src/http.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
use cketh_common::eth_rpc::{HttpOutcallError, ProviderError, RpcError, ValidationError};
use ic_canister_log::log;
use ic_cdk::api::management_canister::http_request::{
http_request as make_http_request, CanisterHttpRequestArgument, HttpHeader, HttpMethod,
HttpResponse, TransformContext,
CanisterHttpRequestArgument, HttpHeader, HttpMethod, HttpResponse, TransformArgs,
TransformContext,
};
use num_traits::ToPrimitive;

Expand All @@ -19,7 +19,6 @@ pub async fn do_http_request(
inc_metric!(request_err_no_permission);
return Err(ProviderError::NoPermission.into());
}
let cycles_available = ic_cdk::api::call::msg_cycles_available128();
let cost = get_request_cost(&source, json_rpc_payload, max_response_bytes);
let (api, provider) = match source {
ResolvedSource::Api(api) => (api, None),
Expand All @@ -39,6 +38,7 @@ pub async fn do_http_request(
return Err(ValidationError::HostNotAllowed(host.to_string()).into());
}
if !is_authorized(&caller, Auth::FreeRpc) {
let cycles_available = ic_cdk::api::call::msg_cycles_available128();
if cycles_available < cost {
return Err(ProviderError::TooFewCycles {
expected: cost,
Expand All @@ -62,7 +62,7 @@ pub async fn do_http_request(
inc_metric_entry!(host_requests, host.to_string());
let mut request_headers = vec![HttpHeader {
name: CONTENT_TYPE_HEADER.to_string(),
value: "application/json".to_string(),
value: CONTENT_TYPE_VALUE.to_string(),
}];
request_headers.extend(api.headers);
let request = CanisterHttpRequestArgument {
Expand All @@ -72,11 +72,11 @@ pub async fn do_http_request(
headers: request_headers,
body: Some(json_rpc_payload.as_bytes().to_vec()),
transform: Some(TransformContext::from_name(
"__transform_evm_rpc".to_string(),
"__transform_json_rpc".to_string(),
vec![],
)),
};
match make_http_request(request, cost).await {
match ic_cdk::api::management_canister::http_request::http_request(request, cost).await {
Ok((response,)) => Ok(response),
Err((code, message)) => {
inc_metric!(request_err_http);
Expand All @@ -85,6 +85,15 @@ pub async fn do_http_request(
}
}

pub fn do_transform_http_request(args: TransformArgs) -> HttpResponse {
HttpResponse {
status: args.response.status,
body: canonicalize_json(&args.response.body).unwrap_or(args.response.body),
// Remove headers (which may contain a timestamp) for consensus
headers: vec![],
}
}

pub fn get_http_response_status(status: candid::Nat) -> u16 {
status.0.to_u16().unwrap_or(u16::MAX)
}
Expand Down
12 changes: 3 additions & 9 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use ic_canister_log::log;
use ic_canisters_http_types::{
HttpRequest as AssetHttpRequest, HttpResponse as AssetHttpResponse, HttpResponseBuilder,
};
use ic_cdk::api::management_canister::http_request::{HttpHeader, HttpResponse, TransformArgs};
use ic_cdk::api::management_canister::http_request::{HttpResponse, TransformArgs};
use ic_cdk::{query, update};
use ic_nervous_system_common::{serve_logs, serve_logs_v2, serve_metrics};

Expand Down Expand Up @@ -222,15 +222,9 @@ async fn withdraw_accumulated_cycles(provider_id: u64, canister_id: Principal) {
};
}

#[query(name = "__transform_evm_rpc")]
#[query(name = "__transform_json_rpc")]
fn transform(args: TransformArgs) -> HttpResponse {
HttpResponse {
status: args.response.status,
body: canonicalize_json(&args.response.body).unwrap_or(args.response.body),
// Strip headers as they contain the Date which is not necessarily the same
// and will prevent consensus on the result.
headers: Vec::<HttpHeader>::new(),
}
do_transform_http_request(args)
}

#[ic_cdk::init]
Expand Down
Loading