Skip to content

Commit

Permalink
Update to version v1.5.0
Browse files Browse the repository at this point in the history
  • Loading branch information
tabdunabi committed Jan 24, 2022
1 parent 843a7b4 commit bc71277
Show file tree
Hide file tree
Showing 20 changed files with 1,697 additions and 442 deletions.
15 changes: 14 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,19 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.5.0] - 2022-01-24

### Added

- A new pipeline to deploy [Amazon SageMaker Clarify Model Bias Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-model-monitor-bias-drift.html). The new pipeline monitors predictions for bias on a regular basis, and generates
alerts if bias beyond a certain threshold is detected.
- A new pipeline to deploy [Amazon SageMaker Clarify Explainability (Feature Attribution Drift) Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-model-monitor-feature-attribution-drift.html). The new pipeline helps data scientists and ML engineers
monitor predictions for feature attribution drift on a regular basis.

### Updated

- The solution's name was changed from "AWS MLOps Framework" to "MLOps Workload Orchestrator".

## [1.4.1] - 2021-12-20

### Added
Expand All @@ -23,7 +36,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- A new pipeline to deploy [AWS SageMaker Model Quality Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html). The new pipeline monitors the performance of a deployed model by comparing the
- A new pipeline to deploy [Amazon SageMaker Model Quality Monitor](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html). The new pipeline monitors the performance of a deployed model by comparing the
predictions that the model makes with the actual ground truth labels that the model attempts to predict.

### Updated
Expand Down
24 changes: 11 additions & 13 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,23 @@ documentation, we greatly value feedback and contributions from our community.
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
information to effectively respond to your bug report or contribution.


## Reporting Bugs/Feature Requests

We welcome you to use the GitHub issue tracker to report bugs or suggest features.

When filing an issue, please check [existing open](https://github.com/awslabs/aws-mlops-solution/issues), or [recently closed](https://github.com/awslabs/aws-mlops-solution/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
When filing an issue, please check [existing open](https://github.com/aws-solutions/mlops-workload-orchestrator/issues), or [recently closed](https://github.com/aws-solutions/mlops-workload-orchestrator/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:

* A reproducible test case or series of steps
* The version of our code being used
* Any modifications you've made relevant to the bug
* Anything unusual about your environment or deployment

- A reproducible test case or series of steps
- The version of our code being used
- Any modifications you've made relevant to the bug
- Anything unusual about your environment or deployment

## Contributing via Pull Requests

Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:

1. You are working against the latest source on the *master* branch.
1. You are working against the latest source on the _main_ branch.
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.

Expand All @@ -40,23 +39,22 @@ To send us a pull request, please:
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).


## Finding contributions to work on
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/aws-mlops-solution/labels/help%20wanted) issues is a great place to start.

Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-solutions/mlops-workload-orchestrator/labels/help%20wanted) issues is a great place to start.

## Code of Conduct

This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
opensource-codeofconduct@amazon.com with any additional questions or comments.


## Security issue notifications
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue.

If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue.

## Licensing

See the [LICENSE](https://github.com/awslabs/aws-mlops-solution/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
See the [LICENSE](https://github.com/aws-solutions/mlops-workload-orchestrator/blob/main/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.

We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.
4 changes: 2 additions & 2 deletions NOTICE.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
aws-mlops-framework
Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
mlops-workload-orchestrator
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except
in compliance with the License. A copy of the License is located at http://www.apache.org/licenses/
or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS,
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# aws-mlops-framework
# mlops-workload-orchestrator

The machine learning (ML) lifecycle is an iterative and repetitive process that involves
changing models over time and learning from new data. As ML applications gain popularity,
Expand All @@ -16,9 +16,9 @@ pipeline for building and registering Docker images for custom algorithms that c
deployment on an [Amazon SageMaker](https://aws.amazon.com/sagemaker/) endpoint.

You can use batch and real-time data inferences to configure the pipeline for your business context.
You can also provision multiple data quality and model quality Monitor pipelines to periodically monitor the quality of deployed
Amazon SageMaker ML models. This solution increases your team’s agility and efficiency by allowing them
to repeat successful processes at scale.
You can also provision multiple data quality, model quality, model bias, and model explainability Monitor
pipelines to periodically monitor the quality of deployed Amazon SageMaker ML models. This solution
increases your team’s agility and efficiency by allowing them to repeat successful processes at scale.

#### Benefits

Expand Down Expand Up @@ -122,8 +122,8 @@ chmod +x ./build-s3-dist.sh
- Upload the distributable assets to your Amazon S3 bucket in your account. Note: ensure that you own the Amazon S3 bucket before uploading the assets. To upload the assets to the S3 bucket, you can use the AWS Console or the AWS CLI as shown below.

```
aws s3 cp ./global-s3-assets/ s3://my-bucket-name-<aws_region>/aws-mlops-framework/<my-version>/ --recursive --acl bucket-owner-full-control --profile aws-cred-profile-name
aws s3 cp ./regional-s3-assets/ s3://my-bucket-name-<aws_region>/aws-mlops-framework/<my-version>/ --recursive --acl bucket-owner-full-control --profile aws-cred-profile-name
aws s3 cp ./global-s3-assets/ s3://my-bucket-name-<aws_region>/mlops-workload-orchestrator/<my-version>/ --recursive --acl bucket-owner-full-control --profile aws-cred-profile-name
aws s3 cp ./regional-s3-assets/ s3://my-bucket-name-<aws_region>/mlops-workload-orchestrator/<my-version>/ --recursive --acl bucket-owner-full-control --profile aws-cred-profile-name
```

---
Expand All @@ -132,17 +132,17 @@ aws s3 cp ./regional-s3-assets/ s3://my-bucket-name-<aws_region>/aws-mlops-frame

```
$DIST_OUTPUT_BUCKET - This is the global name of the distribution. For the bucket name, the AWS Region is added to the global name (example: 'my-bucket-name-us-east-1') to create a regional bucket. The lambda artifact should be uploaded to the regional buckets for the CloudFormation template to pick it up for deployment.
$SOLUTION_NAME - The name of This solution (example: aws-mlops-framework)
$SOLUTION_NAME - The name of This solution (example: mlops-workload-orchestrator)
$VERSION - The version number of the change
```

## Uninstall the solution

Please refer to the [Uninstall the solution section](https://docs.aws.amazon.com/solutions/latest/aws-mlops-framework/uninstall-the-solution.html) in the [solution's implementation guide](https://docs.aws.amazon.com/solutions/latest/aws-mlops-framework/welcome.html).
Please refer to the [Uninstall the solution section](https://docs.aws.amazon.com/solutions/latest/mlops-workload-orchestrator/uninstall-the-solution.html) in the [solution's implementation guide](https://docs.aws.amazon.com/solutions/latest/mlops-workload-orchestrator/welcome.html).

## Collection of operational metrics

This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the [implementation guide](https://docs.aws.amazon.com/solutions/latest/aws-mlops-framework/operational-metrics.html).
This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the [implementation guide](https://docs.aws.amazon.com/solutions/latest/mlops-workload-orchestrator/operational-metrics.html).

## Known Issues

Expand All @@ -157,7 +157,7 @@ For more information regarding this issue and short-term and long-term fixes, re

---

Copyright 2020-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at

Expand Down
40 changes: 24 additions & 16 deletions deployment/build-s3-dist.sh
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ echo "cdk synth DataQualityModelMonitorStack > lib/blueprints/byom/byom_data_qua
cdk synth DataQualityModelMonitorStack > lib/blueprints/byom/byom_data_quality_monitor.yaml --path-metadata false --version-reporting false
echo "cdk synth ModelQualityModelMonitorStack > lib/blueprints/byom/byom_model_quality_monitor.yaml --path-metadata false --version-reporting false"
cdk synth ModelQualityModelMonitorStack > lib/blueprints/byom/byom_model_quality_monitor.yaml --path-metadata false --version-reporting false
echo "cdk synth ModelBiasModelMonitorStack > lib/blueprints/byom/byom_model_bias_monitor.yaml --path-metadata false --version-reporting false"
cdk synth ModelBiasModelMonitorStack > lib/blueprints/byom/byom_model_bias_monitor.yaml --path-metadata false --version-reporting false
echo "cdk synth ModelExplainabilityModelMonitorStack > lib/blueprints/byom/byom_model_explainability_monitor.yaml --path-metadata false --version-reporting false"
cdk synth ModelExplainabilityModelMonitorStack > lib/blueprints/byom/byom_model_explainability_monitor.yaml --path-metadata false --version-reporting false
echo "cdk synth SingleAccountCodePipelineStack > lib/blueprints/byom/single_account_codepipeline.yaml --path-metadata false --version-reporting false"
cdk synth SingleAccountCodePipelineStack > lib/blueprints/byom/single_account_codepipeline.yaml --path-metadata false --version-reporting false
echo "cdk synth MultiAccountCodePipelineStack > lib/blueprints/byom/multi_account_codepipeline.yaml --path-metadata false --version-reporting false"
Expand All @@ -132,6 +136,10 @@ echo "sed -i -e $replace lib/blueprints/byom/byom_data_quality_monitor.yaml"
sed -i -e $replace lib/blueprints/byom/byom_data_quality_monitor.yaml
echo "sed -i -e $replace lib/blueprints/byom/byom_model_quality_monitor.yaml"
sed -i -e $replace lib/blueprints/byom/byom_model_quality_monitor.yaml
echo "sed -i -e $replace lib/blueprints/byom/byom_model_bias_monitor.yaml"
sed -i -e $replace lib/blueprints/byom/byom_model_bias_monitor.yaml
echo "sed -i -e $replace lib/blueprints/byom/byom_model_explainability_monitor.yaml"
sed -i -e $replace lib/blueprints/byom/byom_model_explainability_monitor.yaml
echo "sed -i -e $replace lib/blueprints/byom/byom_realtime_inference_pipeline.yaml"
sed -i -e $replace lib/blueprints/byom/byom_realtime_inference_pipeline.yaml
echo "sed -i -e $replace lib/blueprints/byom/single_account_codepipeline.yaml"
Expand All @@ -144,10 +152,10 @@ echo "sed -i -e $replace lib/blueprints/byom/byom_batch_pipeline.yaml"
sed -i -e $replace lib/blueprints/byom/byom_batch_pipeline.yaml

# Run 'cdk synth' for main templates to generate raw solution outputs
echo "cdk synth aws-mlops-single-account-framework --path-metadata false --version-reporting false --output=$staging_dist_dir"
cdk synth aws-mlops-single-account-framework --path-metadata false --version-reporting false --output=$staging_dist_dir
echo "cdk synth aws-mlops-multi-account-framework --path-metadata false --version-reporting false --output=$staging_dist_dir"
cdk synth aws-mlops-multi-account-framework --path-metadata false --version-reporting false --output=$staging_dist_dir
echo "cdk synth mlops-workload-orchestrator-single-account --path-metadata false --version-reporting false --output=$staging_dist_dir"
cdk synth mlops-workload-orchestrator-single-account --path-metadata false --version-reporting false --output=$staging_dist_dir
echo "cdk synth mlops-workload-orchestrator-multi-account --path-metadata false --version-reporting false --output=$staging_dist_dir"
cdk synth mlops-workload-orchestrator-multi-account --path-metadata false --version-reporting false --output=$staging_dist_dir

# Remove unnecessary output files
echo "cd $staging_dist_dir"
Expand Down Expand Up @@ -187,20 +195,20 @@ cd $template_dist_dir
echo "Updating code source bucket in template with $1"
replace="s/%%BUCKET_NAME%%/$1/g"

echo "sed -i -e $replace $template_dist_dir/aws-mlops-single-account-framework.template"
sed -i -e $replace $template_dist_dir/aws-mlops-single-account-framework.template
echo "sed -i -e $replace $template_dist_dir/aws-mlops-multi-account-framework.template"
sed -i -e $replace $template_dist_dir/aws-mlops-multi-account-framework.template
echo "sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-single-account.template"
sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-single-account.template
echo "sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-multi-account.template"
sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-multi-account.template
replace="s/%%SOLUTION_NAME%%/$2/g"
echo "sed -i -e $replace $template_dist_dir/aws-mlops-single-account-framework"
sed -i -e $replace $template_dist_dir/aws-mlops-single-account-framework.template
echo "sed -i -e $replace $template_dist_dir/aws-mlops-multi-account-framework.template"
sed -i -e $replace $template_dist_dir/aws-mlops-multi-account-framework.template
echo "sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-single-account"
sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-single-account.template
echo "sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-multi-account.template"
sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-multi-account.template
replace="s/%%VERSION%%/$3/g"
echo "sed -i -e $replace $template_dist_dir/aws-mlops-single-account-framework.template"
sed -i -e $replace $template_dist_dir/aws-mlops-single-account-framework.template
echo "sed -i -e $replace $template_dist_dir/aws-mlops-multi-account-framework.template"
sed -i -e $replace $template_dist_dir/aws-mlops-multi-account-framework.template
echo "sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-single-account.template"
sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-single-account.template
echo "sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-multi-account.template"
sed -i -e $replace $template_dist_dir/mlops-workload-orchestrator-multi-account.template


echo "------------------------------------------------------------------------------"
Expand Down
Loading

0 comments on commit bc71277

Please sign in to comment.