Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nexus] Support Bundle background task #7063

Merged
merged 116 commits into from
Jan 13, 2025
Merged
Changes from 3 commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
cd3aabd
Add support bundle API skeleton to Nexus
smklein Nov 7, 2024
3774ecb
fmt
smklein Nov 7, 2024
33787b8
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 7, 2024
0b3fd29
EXPECTORATE
smklein Nov 7, 2024
9eb91bf
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 8, 2024
0f9ff1c
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 8, 2024
ec897f7
Add database schema for support bundles
smklein Nov 8, 2024
38b6ea8
Add failure reason
smklein Nov 8, 2024
a938a82
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 8, 2024
908f661
Merge reason for failure
smklein Nov 8, 2024
aa3978f
Fix SQL, moving forward on tests
smklein Nov 12, 2024
d6d53c9
Refactor state machine
smklein Nov 13, 2024
483801a
Clippy, schema changes
smklein Nov 13, 2024
5f78af9
more clippy
smklein Nov 13, 2024
8dba975
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 13, 2024
69e04ae
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 13, 2024
92b7d62
Strongly-typed UUID
smklein Nov 13, 2024
bc10ee1
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 13, 2024
410ff5b
Skeleton of background task
smklein Nov 13, 2024
2234301
less bp
smklein Nov 14, 2024
6c529d1
Add list-by-nexus method, test it
smklein Nov 14, 2024
aeec097
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 14, 2024
6972b2f
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 25, 2024
adf956c
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 25, 2024
11a88d6
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 25, 2024
e61365a
merge
smklein Nov 25, 2024
07159ad
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 25, 2024
01949c9
Merge UUID changes
smklein Nov 25, 2024
5d02715
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 25, 2024
81c7121
Merge UUID changes
smklein Nov 25, 2024
8052bbe
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 25, 2024
744e351
Implement storage, use sled agent APIs
smklein Nov 26, 2024
9baae20
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 27, 2024
60f92dd
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 27, 2024
f6f28a0
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 27, 2024
7c33f19
fix typed uuid merge
smklein Nov 27, 2024
affa4c7
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 27, 2024
10c3745
more typed UUID merges
smklein Nov 27, 2024
4feff4b
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 27, 2024
8b8fab7
update omdb output
smklein Nov 27, 2024
2b42d0e
API updates pulled from 7187 as I built it
smklein Nov 28, 2024
2371630
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Nov 28, 2024
4f6d2d2
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Nov 28, 2024
356225c
Merge branch 'main' into nexus-support-bundles-api
smklein Nov 28, 2024
1fccea8
expectorate
smklein Dec 2, 2024
b33ed31
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 2, 2024
0f8f750
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Dec 2, 2024
417cab9
update description
smklein Dec 2, 2024
d10e034
big one, should be split up
smklein Dec 5, 2024
3c5047d
Fixing clippy
smklein Dec 5, 2024
b77c23a
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 6, 2024
0b939f2
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 6, 2024
f9a2efd
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Dec 6, 2024
59612b0
Fix (and test) ordering, output paths
smklein Dec 6, 2024
0461260
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 9, 2024
490d648
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 9, 2024
366204f
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Dec 9, 2024
569a28c
Wire up bundles to blueprints, more tests
smklein Dec 10, 2024
83bdc2e
Support bundle re-assignment is no-op without any bundles to fail
smklein Dec 10, 2024
a8ed3d3
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 10, 2024
abedbb2
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 10, 2024
6661475
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Dec 10, 2024
dcf4bed
comment cleaning
smklein Dec 10, 2024
01bc1c1
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 13, 2024
9795bfe
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 13, 2024
5866f48
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 13, 2024
55e9c24
use path params more
smklein Dec 13, 2024
7d87a27
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 13, 2024
efca596
Review feedback
smklein Dec 13, 2024
e29733d
Add to uncovered endpoints, since these aren't impl'd yet
smklein Dec 13, 2024
0464f25
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 13, 2024
2232690
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 16, 2024
f97bfc5
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 16, 2024
5cc668e
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Dec 16, 2024
51be611
Handle bundle cleanup from deleted zpool
smklein Dec 16, 2024
adea822
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 17, 2024
0ad466d
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 17, 2024
fa25cd2
Merge branch 'support-bundles-crdb' into support-bundle-bg-task
smklein Dec 17, 2024
80edd57
Make zpool_get_sled more paranoid
smklein Dec 17, 2024
d16fda0
Use a shared support bundle implementation for the simulated sled agent
smklein Dec 17, 2024
60c27d3
cleanup println
smklein Dec 17, 2024
900c670
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Dec 17, 2024
2a24dd2
Merge branch 'main' into nexus-support-bundles-api
smklein Dec 19, 2024
50917c7
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Dec 19, 2024
053adca
Merge branch 'support-bundles-crdb' into support-bundle-simulated-imp…
smklein Dec 19, 2024
5f3f507
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Dec 19, 2024
28130e5
Merge branch 'main' into nexus-support-bundles-api
smklein Jan 6, 2025
9bf57a4
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Jan 6, 2025
28795f6
Merge branch 'support-bundles-crdb' into support-bundle-simulated-imp…
smklein Jan 6, 2025
a39f6a8
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 6, 2025
1506c87
merging
smklein Jan 6, 2025
a0f45c1
Merge branch 'support-bundles-crdb' into support-bundle-simulated-imp…
smklein Jan 6, 2025
9d416b1
Fix simulated nested datasets, add tests
smklein Jan 6, 2025
8557ad0
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 6, 2025
a8b263a
Merge branch 'main' into nexus-support-bundles-api
smklein Jan 6, 2025
bdaab3e
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Jan 6, 2025
8b58c16
Merge branch 'support-bundles-crdb' into support-bundle-simulated-imp…
smklein Jan 6, 2025
14d39fd
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 6, 2025
917759d
config.test.toml
smklein Jan 6, 2025
7d166f1
Error logging
smklein Jan 6, 2025
bcf6f8c
Propagate unexpected errors during bundle activation
smklein Jan 7, 2025
76be221
remove printlns
smklein Jan 7, 2025
d23826e
Use BufReader for reading entries
smklein Jan 7, 2025
5746fbf
Merge branch 'main' into nexus-support-bundles-api
smklein Jan 8, 2025
7c1457f
Merge branch 'nexus-support-bundles-api' into support-bundles-crdb
smklein Jan 8, 2025
69e0f46
Merge branch 'support-bundles-crdb' into support-bundle-simulated-imp…
smklein Jan 8, 2025
1baad89
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 8, 2025
39394ff
Update to deal with structured support bundle APIs from sled agent
smklein Jan 8, 2025
53dcbe6
Use /var/tmp
smklein Jan 9, 2025
ac36cf6
Merge branch 'main' into support-bundle-simulated-implementation
smklein Jan 9, 2025
f40f1a4
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 9, 2025
f73f081
Merge branch 'main' into support-bundle-simulated-implementation
smklein Jan 13, 2025
dcedfd3
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 13, 2025
a54097c
simpler buffering
smklein Jan 13, 2025
1a8a21d
Merge branch 'main' into support-bundle-simulated-implementation
smklein Jan 13, 2025
7c5e396
Merge branch 'support-bundle-simulated-implementation' into support-b…
smklein Jan 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions clients/sled-agent-client/src/lib.rs
Original file line number Diff line number Diff line change
@@ -75,6 +75,8 @@ progenitor::generate_api!(
RouterVersion = omicron_common::api::internal::shared::RouterVersion,
SledRole = nexus_sled_agent_shared::inventory::SledRole,
SourceNatConfig = omicron_common::api::internal::shared::SourceNatConfig,
SupportBundleGetQueryParams = omicron_common::api::external::SupportBundleGetQueryParams,
SupportBundleQueryType = omicron_common::api::external::SupportBundleQueryType,
SwitchLocation = omicron_common::api::external::SwitchLocation,
TypedUuidForDatasetKind = omicron_uuid_kinds::DatasetUuid,
TypedUuidForInstanceKind = omicron_uuid_kinds::InstanceUuid,
18 changes: 18 additions & 0 deletions common/src/api/external/mod.rs
Original file line number Diff line number Diff line change
@@ -3090,6 +3090,24 @@ pub enum ImportExportPolicy {
Allow(Vec<oxnet::IpNet>),
}

/// Query parameters for reading the support bundle
#[derive(Deserialize, Serialize, JsonSchema)]
pub struct SupportBundleGetQueryParams {
pub query_type: SupportBundleQueryType,
}

/// Describes the type of access to the support bundle
#[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum SupportBundleQueryType {
/// Access the whole support bundle
Whole,
/// Access the names of all files within the support bundle
Index,
/// Access a specific file within the support bundle
Path { file_path: String },
}

#[cfg(test)]
mod test {
use serde::Deserialize;
1 change: 1 addition & 0 deletions nexus/external-api/output/nexus_tags.txt
Original file line number Diff line number Diff line change
@@ -33,6 +33,7 @@ probe_view GET /experimental/v1/probes/{probe
support_bundle_create POST /experimental/v1/system/support-bundles
support_bundle_delete DELETE /experimental/v1/system/support-bundles/{support_bundle}
support_bundle_download GET /experimental/v1/system/support-bundles/{support_bundle}/download
support_bundle_head HEAD /experimental/v1/system/support-bundles/{support_bundle}/download
support_bundle_list GET /experimental/v1/system/support-bundles
support_bundle_view GET /experimental/v1/system/support-bundles/{support_bundle}

17 changes: 15 additions & 2 deletions nexus/external-api/src/lib.rs
Original file line number Diff line number Diff line change
@@ -2793,6 +2793,19 @@ pub trait NexusExternalApi {
async fn support_bundle_download(
rqctx: RequestContext<Self::Context>,
path_params: Path<params::SupportBundlePath>,
body: TypedBody<SupportBundleGetQueryParams>,
) -> Result<Response<Body>, HttpError>;

/// Download the metadata of a single support bundle
#[endpoint {
method = HEAD,
path = "/experimental/v1/system/support-bundles/{support_bundle}/download",
tags = ["hidden"], // system/support-bundles: only one tag is allowed
}]
async fn support_bundle_head(
rqctx: RequestContext<Self::Context>,
path_params: Path<params::SupportBundlePath>,
body: TypedBody<SupportBundleGetQueryParams>,
) -> Result<Response<Body>, HttpError>;

/// Create a new support bundle
@@ -2803,7 +2816,7 @@ pub trait NexusExternalApi {
}]
async fn support_bundle_create(
rqctx: RequestContext<Self::Context>,
) -> Result<HttpResponseOk<shared::SupportBundleInfo>, HttpError>;
) -> Result<HttpResponseCreated<shared::SupportBundleInfo>, HttpError>;

/// Delete an existing support bundle
///
@@ -2817,7 +2830,7 @@ pub trait NexusExternalApi {
async fn support_bundle_delete(
rqctx: RequestContext<Self::Context>,
path_params: Path<params::SupportBundlePath>,
) -> Result<HttpResponseOk<shared::SupportBundleInfo>, HttpError>;
) -> Result<HttpResponseDeleted, HttpError>;

// Probes (experimental)

30 changes: 28 additions & 2 deletions nexus/src/external_api/http_entrypoints.rs
Original file line number Diff line number Diff line change
@@ -81,6 +81,7 @@ use omicron_common::api::external::NameOrId;
use omicron_common::api::external::Probe;
use omicron_common::api::external::RouterRoute;
use omicron_common::api::external::RouterRouteKind;
use omicron_common::api::external::SupportBundleGetQueryParams;
use omicron_common::api::external::SwitchPort;
use omicron_common::api::external::SwitchPortSettings;
use omicron_common::api::external::SwitchPortSettingsView;
@@ -6049,6 +6050,31 @@ impl NexusExternalApi for NexusExternalApiImpl {
async fn support_bundle_download(
rqctx: RequestContext<Self::Context>,
_path_params: Path<params::SupportBundlePath>,
_body: TypedBody<SupportBundleGetQueryParams>,
) -> Result<Response<Body>, HttpError> {
let apictx = rqctx.context();
let handler = async {
let nexus = &apictx.context.nexus;

let opctx =
crate::context::op_context_for_external_api(&rqctx).await?;

Err(nexus
.unimplemented_todo(&opctx, crate::app::Unimpl::Public)
.await
.into())
};
apictx
.context
.external_latencies
.instrument_dropshot_handler(&rqctx, handler)
.await
}

async fn support_bundle_head(
rqctx: RequestContext<Self::Context>,
_path_params: Path<params::SupportBundlePath>,
_body: TypedBody<SupportBundleGetQueryParams>,
) -> Result<Response<Body>, HttpError> {
let apictx = rqctx.context();
let handler = async {
@@ -6071,7 +6097,7 @@ impl NexusExternalApi for NexusExternalApiImpl {

async fn support_bundle_create(
rqctx: RequestContext<Self::Context>,
) -> Result<HttpResponseOk<shared::SupportBundleInfo>, HttpError> {
) -> Result<HttpResponseCreated<shared::SupportBundleInfo>, HttpError> {
let apictx = rqctx.context();
let handler = async {
let nexus = &apictx.context.nexus;
@@ -6094,7 +6120,7 @@ impl NexusExternalApi for NexusExternalApiImpl {
async fn support_bundle_delete(
rqctx: RequestContext<Self::Context>,
_path_params: Path<params::SupportBundlePath>,
) -> Result<HttpResponseOk<shared::SupportBundleInfo>, HttpError> {
) -> Result<HttpResponseDeleted, HttpError> {
let apictx = rqctx.context();
let handler = async {
let nexus = &apictx.context.nexus;
2 changes: 1 addition & 1 deletion nexus/types/src/external_api/params.rs
Original file line number Diff line number Diff line change
@@ -89,8 +89,8 @@ path_param!(SshKeyPath, ssh_key, "SSH key");
path_param!(AddressLotPath, address_lot, "address lot");
path_param!(ProbePath, probe, "probe");
path_param!(CertificatePath, certificate, "certificate");
path_param!(SupportBundlePath, support_bundle, "support bundle");

id_path_param!(SupportBundlePath, support_bundle, "support bundle");
id_path_param!(GroupPath, group_id, "group");

// TODO: The hardware resources should be represented by its UUID or a hardware
145 changes: 128 additions & 17 deletions openapi/nexus.json
Original file line number Diff line number Diff line change
@@ -366,8 +366,8 @@
"summary": "Create a new support bundle",
"operationId": "support_bundle_create",
"responses": {
"200": {
"description": "successful operation",
"201": {
"description": "successful creation",
"content": {
"application/json": {
"schema": {
@@ -396,10 +396,11 @@
{
"in": "path",
"name": "support_bundle",
"description": "Name or ID of the support bundle",
"description": "ID of the support bundle",
"required": true,
"schema": {
"$ref": "#/components/schemas/NameOrId"
"type": "string",
"format": "uuid"
}
}
],
@@ -433,23 +434,17 @@
{
"in": "path",
"name": "support_bundle",
"description": "Name or ID of the support bundle",
"description": "ID of the support bundle",
"required": true,
"schema": {
"$ref": "#/components/schemas/NameOrId"
"type": "string",
"format": "uuid"
}
}
],
"responses": {
"200": {
"description": "successful operation",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SupportBundleInfo"
}
}
}
"204": {
"description": "successful deletion"
},
"4XX": {
"$ref": "#/components/responses/Error"
@@ -471,13 +466,63 @@
{
"in": "path",
"name": "support_bundle",
"description": "Name or ID of the support bundle",
"description": "ID of the support bundle",
"required": true,
"schema": {
"$ref": "#/components/schemas/NameOrId"
"type": "string",
"format": "uuid"
}
}
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SupportBundleGetQueryParams"
}
}
},
"required": true
},
"responses": {
"default": {
"description": "",
"content": {
"*/*": {
"schema": {}
}
}
}
}
},
"head": {
"tags": [
"hidden"
],
"summary": "Download the metadata of a single support bundle",
"operationId": "support_bundle_head",
"parameters": [
{
"in": "path",
"name": "support_bundle",
"description": "ID of the support bundle",
"required": true,
"schema": {
"type": "string",
"format": "uuid"
}
}
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SupportBundleGetQueryParams"
}
}
},
"required": true
},
"responses": {
"default": {
"description": "",
@@ -20199,6 +20244,18 @@
"items"
]
},
"SupportBundleGetQueryParams": {
"description": "Query parameters for reading the support bundle",
"type": "object",
"properties": {
"query_type": {
"$ref": "#/components/schemas/SupportBundleQueryType"
}
},
"required": [
"query_type"
]
},
"SupportBundleInfo": {
"type": "object",
"properties": {
@@ -20248,6 +20305,60 @@
"items"
]
},
"SupportBundleQueryType": {
"description": "Describes the type of access to the support bundle",
"oneOf": [
{
"description": "Access the whole support bundle",
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": [
"whole"
]
}
},
"required": [
"type"
]
},
{
"description": "Access the names of all files within the support bundle",
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": [
"index"
]
}
},
"required": [
"type"
]
},
{
"description": "Access a specific file within the support bundle",
"type": "object",
"properties": {
"file_path": {
"type": "string"
},
"type": {
"type": "string",
"enum": [
"path"
]
}
},
"required": [
"file_path",
"type"
]
}
]
},
"SupportBundleState": {
"oneOf": [
{
19 changes: 2 additions & 17 deletions sled-agent/api/src/lib.rs
Original file line number Diff line number Diff line change
@@ -702,23 +702,8 @@ pub struct SupportBundleGetHeaders {
range: String,
}

/// Query parameters for reading the support bundle
#[derive(Deserialize, Serialize, JsonSchema)]
pub struct SupportBundleGetQueryParams {
pub query_type: SupportBundleQueryType,
}

/// Describes the type of access to the support bundle
#[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum SupportBundleQueryType {
/// Access the whole support bundle
Whole,
/// Access the names of all files within the support bundle
Index,
/// Access a specific file within the support bundle
Path { file_path: String },
}
pub use omicron_common::api::external::SupportBundleGetQueryParams;
pub use omicron_common::api::external::SupportBundleQueryType;

#[derive(Deserialize, Debug, Serialize, JsonSchema, PartialEq)]
#[serde(rename_all = "snake_case")]