Update to version v1.0.0

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,34 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior.
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Please complete the following information about the solution:**
+- [ ] Version: [e.g. v1.0.0]
+
+To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "(SO0136) - AWS MLOps Framework. Version v1.0.0".
+
+- [ ] Region: [e.g. us-east-1]
+- [ ] Was the solution modified from the version published on this repository?
+- [ ] If the answer to the previous question was yes, are the changes available on GitHub?
+- [ ] Have you checked your [service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) for the sevices this solution uses?
+- [ ] Were there any errors in the CloudWatch Logs?
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem (please **DO NOT include sensitive information**).
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this solution
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the feature you'd like**
+A clear and concise description of what you want to happen.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,5 @@
+*Issue #, if available:*
+
+*Description of changes:*
+
+By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

.gitignore

@@ -0,0 +1,31 @@
+*.egg
+*.egg-info
+__pycache__
+*.js
+!jest.config.js
+*.d.ts
+node_modules
+!source/.typescript/lambda/**/*.js
+!source/lambda/**/*.js
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out
+
+!**/cdk-solution-helper/index.js
+
+deployment/global-s3-assets
+deployment/regional-s3-assets
+open-source/
+.vscode/settings.json
+
+source/lambda/layers/*
+
+.DS_Store
+
+# Temporary folders
+tmp/
+temp/
+
+### VisualStudioCode Path ###
+.vscode

CHANGELOG.md

@@ -0,0 +1,13 @@
+# Change Log
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] - 2020-11-05
+### Added
+  - Initiates a pre-configured pipeline through an API call or a Git repository
+  - Automatically deploys a trained model and provides an inference endpoint
+  - Supports running your own integration tests to ensure that the deployed model meets expectations
+  - Allows to provision multiple environments to support ML model's life cycle
+  - Notifies users of the pipeline outcome though SMS or email

CODE_OF_CONDUCT.md

@@ -1,4 +1,4 @@
 ## Code of Conduct
-This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
-For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
 opensource-codeofconduct@amazon.com with any additional questions or comments.

CONTRIBUTING.md

@@ -11,7 +11,7 @@ information to effectively respond to your bug report or contribution.
 
 We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
-When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
+When filing an issue, please check [existing open](https://github.com/awslabs/aws-mlops-solution/issues), or [recently closed](https://github.com/awslabs/aws-mlops-solution/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
 reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
 
 * A reproducible test case or series of steps
@@ -31,17 +31,18 @@ To send us a pull request, please:
 
 1. Fork the repository.
 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
-3. Ensure local tests pass.
-4. Commit to your fork using clear commit messages.
-5. Send us a pull request, answering any default questions in the pull request interface.
-6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+3. Ensure all build processes execute successfully (see README.md for additional guidance).
+4. Ensure all unit, integration, and/or snapshot tests pass, as applicable.
+5. Commit to your fork using clear commit messages.
+6. Send us a pull request, answering any default questions in the pull request interface.
+7. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
 
 GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
 [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
 
 
 ## Finding contributions to work on
-Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/aws-mlops-solution/labels/help%20wanted) issues is a great place to start.
 
 
 ## Code of Conduct
@@ -51,9 +52,11 @@ opensource-codeofconduct@amazon.com with any additional questions or comments.
 
 
 ## Security issue notifications
-If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue.
 
 
 ## Licensing
 
-See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+See the [LICENSE](https://github.com/awslabs/aws-mlops-solution/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

LICENSE → LICENSE.txt

@@ -172,4 +172,4 @@
       of any other Contributor, and only if You agree to indemnify,
       defend, and hold each Contributor harmless for any liability
       incurred by, or claims asserted against, such Contributor by reason
-      of your accepting any such warranty or additional liability.
+      of your accepting any such warranty or additional liability.

NOTICE.txt

@@ -0,0 +1,25 @@
+aws-mlops-framework
+Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except
+in compliance with the License. A copy of the License is located at http://www.apache.org/licenses/
+or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+**********************
+THIRD PARTY COMPONENTS
+**********************
+This software includes third party software subject to the following copyrights:
+
+aws-cdk under the Apache License Version 2.0
+AWS SDK under the Apache License Version 2.0
+boto3 under the Apache License Version 2.0
+pytest-env under the Massachusetts Institute of Technology (MIT) license
+crhelper under the Apache License Version 2.0
+jsii under the Apache License Version 2.0
+pytest under the Massachusetts Institute of Technology (MIT) license
+pytest-cov under the Massachusetts Institute of Technology (MIT) license
+pytest-mock under the Massachusetts Institute of Technology (MIT) license
+requests under the Apache License Version 2.0
+sagemaker under the Apache License Version 2.0
+urllib3 under the Massachusetts Institute of Technology (MIT) license

README.md

@@ -1,17 +1,129 @@
-## My Project
+# aws-mlops-framework
+The machine learning (ML) lifecycle is an iterative and repetitive process that involves
+changing models over time and learning from new data. As ML applications gain popularity,
+organizations are building new and better applications for a wide range of use cases
+including optimized email campaigns, forecasting tools, recommendation engines, self-
+driving vehicles, virtual personal assistants, and more. While operational and pipelining
+processes vary greatly across projects and organizations, the processes contain
+commonalities across use cases.
 
-TODO: Fill this README out!
+The AWS MLOps Framework solution helps you streamline and enforce architecture best
+practices for machine learning (ML) model productionization. This solution is an extendable
+framework that provides a standard interface for managing ML pipelines for AWS ML
+services and third-party services. The solution’s template allows customers to upload their
+trained models, configure the orchestration of the pipeline, trigger the start of the deployment
+process, move models through different stages of deployment, and monitor the successes
+and failures of the operations.
+You can use batch and real-time data inferences to configure the pipeline for your business
+context. This solution increases your team’s agility and efficiency by allowing them to
+repeat successful processes at scale.
 
-Be sure to:
+***
 
-* Change the title in this README
-* Edit your repository description on GitHub
+## Architecture
+The AWS CloudFormation template deploys a Pipeline Provisioning framework that
+provisions a machine learning pipeline (Bring Your Own Model for SageMaker). The
+template includes the AWS Lambda functions and AWS Identity and Access Management
+(IAM) roles necessary to set up your account, and it creates an Amazon Simple Storage
+Service (Amazon S3) bucket that contains the CloudFormation templates that set up the
+pipelines.The template also creates an Amazon API Gateway instance, an additional
+Lambda function, and an AWS CodePipeline instance.
+The provisioned pipeline includes four stages: source, build, deploy, and share.
 
-## Security
+![architecture](source/architecture.png)
 
-See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
+***
 
-## License
+## File Structure
 
-This project is licensed under the Apache-2.0 License.
+Upon successfully cloning the repository into your local development environment but **prior** to running the initialization script, you will see the following file structure in your editor:
 
+```
+├── deployment                            [folder containing build scripts]
+│   ├── cdk-solution-helper               [A helper function to help deploy lambda function code through S3 buckets]
+│   ├── build-s3-dist.sh                  [A script to prepare the solution for deploying from source code]
+├── source                                [source code containing CDK App and lambda functions]
+│   ├── lambdas                           [folder containing source code the lambda functions]
+│   │   ├── custom_resource               [lambda function to copy necessary resources from aws solutions bucket]
+│   │   ├── pipeline_orchestration        [lambda function to provision ML pipelines]
+│   └── lib
+│       ├── blueprints                    [folder containing implementations of different types of ML pipelines supported by this solution]
+│       │   ├── byom                      [implementation of bring-your-own-model ML pipeline]
+│       │   │   ├── lambdas               [folder containing source code the lambda functions]
+│       │   │   └── pipeline_definitions  [folder containing CDK implementation of pipeline stages in BYOM]
+│       ├── aws_mlops_stack.py            [CDK implementation of the main framework ]
+│       └── conditional_resource.py       [a helper file to enable conditional resource provisioning in CDK]
+├── .gitignore
+├── CHANGELOG.md                          [required for every solution to include changes based on version to auto[uild release notes]
+├── CODE_OF_CONDUCT.md                    [standardized open source file for all solutions]
+├── CONTRIBUTING.md                       [standardized open source file for all solutions]
+├── LICENSE.txt                           [required open source file for all solutions - should contain the Apache 2.0 license]
+├── NOTICE.txt                            [required open source file for all solutions - should contain references to all 3rd party libraries]
+└── README.md                             [required file for all solutions]
+
+* Note: Not all languages are supported at this time. Actual appearance may vary depending on release.
+```
+
+## Building the solution
+
+### 1. Get source code
+
+Clone this git repository.
+
+`git clone https://github.com/awslabs/<repository_name>`
+
+***
+
+### 2. Running Unit Tests
+
+The `/source/run-all-tests.sh` script is the centralized script for running all unit, integration, and snapshot tests for both the CDK project as well as any associated Lambda functions or other source code packages.
+
+- Note: It is the developer's responsibility to ensure that all test commands are called in this script, and that it is kept up to date.
+
+This script is called from the solution build scripts to ensure that specified tests are passing while performing build, validation and publishing tasks via the pipeline.
+
+***
+
+### 3. Building Project Distributable
+* Configure the bucket name of your target Amazon S3 distribution bucket
+```
+export DIST_OUTPUT_BUCKET=my-bucket-name # bucket where customized code will reside
+export SOLUTION_NAME=my-solution-name
+export VERSION=my-version # version number for the customized code
+```
+_Note:_ You would have to create an S3 bucket with the prefix 'my-bucket-name-<aws_region>'; aws_region is where you are testing the customized solution. Also, the assets in bucket should be publicly accessible.
+
+* Now build the distributable:
+```
+chmod +x ./build-s3-dist.sh \n
+./build-s3-dist.sh $DIST_OUTPUT_BUCKET $SOLUTION_NAME $VERSION \n
+```
+
+* Deploy the distributable to an Amazon S3 bucket in your account. _Note:_ you must have the AWS Command Line Interface installed.
+```
+aws s3 cp ./dist/ s3://my-bucket-name-<aws_region>/$SOLUTION_NAME/$VERSION/ --recursive --acl bucket-owner-full-control --profile aws-cred-profile-name \n
+```
+
+* Get the link of the solution template uploaded to your Amazon S3 bucket.
+* Deploy the solution to your account by launching a new AWS CloudFormation stack using the link of the solution template in Amazon S3.
+
+## Known Issues
+
+### Pipeline may fail in custom model container build due to Docker Hub rate limits
+
+When building custom model container that pulls public docker images from Docker Hub in short time period, you may occasionally face throttling errors with an error message such as:
+``` toomanyrequests  You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit```
+
+This is due to Docker Inc. [limiting the rate at which images are pulled under Docker Hub anonymous and free plans](https://docs.docker.com/docker-hub/download-rate-limit/). Under the new limits of Dockerhub, free plan anonymous use is limited to 100 pulls per six hours, free plan authenticated accounts limited to 200 pulls per six hours, and Pro and Team accounts do not see any rate limits.
+
+For more information regarding this issue and short-term and long-term fixes, refer to this AWS blog post: [Advice for customers dealing with Docker Hub rate limits, and a Coming Soon announcement](https://aws.amazon.com/blogs/containers/advice-for-customers-dealing-with-docker-hub-rate-limits-and-a-coming-soon-announcement/)
+
+***
+
+Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at
+
+    http://www.apache.org/licenses/
+
+or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License.