Skip to content

Latest commit

 

History

History
238 lines (213 loc) · 22.5 KB

feature-release.md

File metadata and controls

238 lines (213 loc) · 22.5 KB

Feature Release Process

Mattermost core team works on a monthly release process, with a new version shipping on the 16th of each month in binary form.

This document outlines the development process for the Mattermost core team, which draws from what we find works best for us from Agile, Scrum, and Software Development Lifecycle approaches.

Recommended pre-reading:

Release Timeline

Notes:

A. (Code complete date of previous release) Beginning of Release

Pre-work for the current release begins at the code complete date of the previous release. See "Code Complete" section below for details.

B. (T-minus 30 working days) Judgment Day & Release Branch Cut

Day when Leads and PMs decide which major features are included in the release, and which are postponed. Stabilization period begins when all features for release have been committed. During this period, only bugs can be committed to the release branch. Non-bug pull requests are tagged for the next version. Exceptions can be made by the Release Manager during triage. Review the Release Features & Bugs Quality Gate Guidelines

  1. Release Manager:
    • Post this checklist in Release Checklist channel
    • Begin daily triage of tickets with the team
      • Also triage tickets in the backlog
    • Prior to the cut to Self-Managed, the Release manager reviews new feature flags and provides a report to the PM/QA team for their review. Essentially a feature will be included in a Self-Managed release once the feature flag has been removed.
    • Confirm with PMs that each Enterprise feature is in the correct pricing SKU
    • Review any features that are currently in beta and confirm with PMs if there are any to be promoted
    • Review the Jira tickets remaining in the current release fix version and push those that won't make it to the next fix version
    • After mobile release branch is cut, ask dev to cut an RN build
    • Queue a list of MVP candidates in alphabetical order to the MVP Discussion channel See example
    • Draft changelog in a WIP PR with updates for highlights, feature additions, known issues, compatibility updates for deprecated features, config.json, database changes, API changes, WebSocket event changes, and Go runtime version; see example
    • Review that software requirements and server software versions are up-to-date based on these step-by-step guidelines
    • Ask PMs and Dev Leads if there are any notable breaking changes or deprecated features in the release
      • Update Upgrade Guide with any special notes for upgrading to the new version
      • If there are any breaking compatibility changes in the release, open an issue in the GitLab Omnibus to make sure GitLab is aware. Post a link to the issue in the Release: Self-Managed channel
    • Start posting a daily list of open and submitted bugs and PRs
    • Submit NOTICE.txt PRs for any new libraries added from dev, and ensure they are cherry-picked to feature release branch
    • Confirm that NOTICE.txt PRs, Translations PRs, database upgrade, and any needed pre-packaged plugins, backports, and recent regression fixes are included in the release
    • Create meta issue for release in GitHub (see example)
    • Post a reminder in the Localization channel about the due date for translations, similar to this example
    • Prepare bullet points for release announcement and share for PMs to work on. Refer to the process here
    • Ask PMs and Tech Writer to complete release documentation by T-9 deadline. Example request
    • Schedule a meeting with PMs and QAs to discuss upcoming features in the next feature release
  2. Dev:
    • Cut release branches for server and mobile
      • Merge database upgrade before cutting the branch
      • Update model/version.go
      • Point CI servers to the release branch after cutting
      • Update https://prev.test.mattermost.com to the previous release version
        • Prioritize reviewing, updating, and merging of pull requests for current release until there are no more tickets in the pull request queue marked for the current release
        • After the cut-off, any PRs that include significant code changes require approval of the Release Manager before merging
  3. QA:
    • Prioritize testing PRs and resolved tickets for this release
    • Ensure that new features are also properly tested on mobile apps
    • Prioritize updating test cases as needed in test management and automated tests

C. (T-minus 9 working days) RC1 Cut and RC Testing Begins

  1. Release Manager:
    • Post this checklist in Release Checklist channel
    • Verify all items in the last posted release checklist are complete
    • Post in the Release: Self-Managed channel the rough timing when the release candidate will be cut
    • Cut a Release Candidate using this process and check CI servers running on release branch
    • Generate a list of contributors for changelog
    • Confirm doc submission
    • Post list of tickets to be fixed to the Release: Self-Managed channel (see example)
    • Update the GitHub meta issue:
      • Include a link to the changelog on the documentation branch
      • Update download links and testing server links to the latest RCs
    • After build is cut, tweet announcement that RC1 is ready (see example)
    • Update Open Source Components and Release Lifecycle docs
    • Confirm changelog reflects any changes since it was merged (including known issues and contributors from all repositories)
      • Confirm translators have been added
      • Confirm Open Source Components changes have been added
    • Find www-gitlab-com merge request for latest GitLab release blog post and make request for adding GitLab Mattermost update (see example request, example update). Post to Release: Self-Managed channel with link to request
    • Confirm the required URLs for the blog/release announcement were prepared
  2. Logistics @hanna.park:
    • Mail out contributor and security researcher mugs
      • Space out the ordering of mugs over the next three weeks to prevent mistakes being made by the supplier (e.g., If there are 12 contributors to order mugs for, place an order every 2nd or 3rd day over the next three weeks)
    • Update Team page with new contributors
    • Generate an E20 5000 seat test licence and email to Lindy for release testing
  3. QA:
    • Confirm up to date with testing PRs and resolved tickets
    • Confirm up to date with test updates and known issues in release test management and automated tests
    • Assign release testing areas to team members
    • After RC1 is cut: Update test server invite links in Release Testing instructions
    • After RC1 is cut: Run automated regression tests
    • Post reminder in Release Announcements channel for any unfinished RC testing, at-mentioning tester(s)
      • Find QA or other teammates to help finish unfinished tests if needed
      • Verify all release tests are finished, bring any concerns to Triage/Release meeting
      • Go through test cycles and verify all issues and comments have been filed or updated in Jira as needed
  4. Build:
    • Review all TODO notes, including one for uncommenting upgrade code
    • Confirm all PRs in /enterprise repo have been merged
    • Master is tagged and branched and Release Candidate 1 is cut (e.g. 3.5.0-RC1) according to the Release Candidate Checklist in mattermost/process
    • Update version for each Mattermost Helm chart
    • Make PRs for bug fixes to the release branch
    • Review PRs made from release branch and merge changes into the release branch as required
    • Run daily automated upgrade tests to avoid catching upgrade bugs late
  5. PM:
    • Finish draft of blog post for mattermost.com and all art work (screenshots, GIFs, and Twitter banners) used for the blog post
      • Add links to Admin guide in the release blog post where needed

D. (T-minus 5 working days) Code Freeze

Day after which only S1 regressions should be fixed (crashes, security vulnerabilities) and no other code changes should be committed. Exceptions can be made by the Release Manager during triage on a case-by-case basis. Review the Bug Severity Guidelines.

  1. Release Manager:
    • Post this checklist in Release Checklist channel
    • Verify all items in the last posted release checklist are complete
    • Verify that the final translations PR for the release is submitted
    • Update Known Issues section with any significant issues that were found and not fixed for the final release
  2. QA:
    • Verify all Jira tickets other than newly-filed bugs have been tested, verified, and closed
    • As bug fixes are merged and RCs are cut, verify fixes on new RCs, and post in Release Channel after testing
    • After all tickets are verified and closed, run smoke tests on webapp/server, desktop app, and RN apps as appropriate
  3. Docs:

E. (T-minus 2 working days) Release Build Cut

The final release is cut - RC cuts and bug fixes should be completed by this date. Only urgent and critical issues are considered for fixing.

  1. Release Manager:
    • Post this checklist in Release Checklist channel
    • Verify all items in the last posted release checklist are complete
    • Work with a developer to submit GitLab MR following this process and test the upgrade once the GitLab MR is merged and included in their RC
    • Close GitHub meta ticket for the release
    • Add the download links, SHA-256 hash, and GPG signature to the version archive
    • Merge changelog PR after review is complete
      • If there is a security fix, confirm the changelog recommends upgrade, with a note mentioning the security level and thanking the security researcher (security process doc for example)
    • Update the Mattermost server download page with the links to the Enterprise Edition and Team Edition sections
      • Test the download links before and after updating the page
    • Check security issues and confirm disclosure text
    • Update the security mugs spreadsheet
    • Check the security researcher was added to the Responsible Disclosure Policy page
    • Update deprecated feature list in mattermost.com with new and scheduled deprecations
    • Draft Mattermost Security Updates, but do not post until 30 days after official release
      • Add a placeholder text saying "Details on the security update will be posted here on X date, as per our Responsible Disclosure Policy"
  2. QA:
    • Verify all PRs and tickets for the release have been tested/closed
    • Post QA approval in Release: Self-Managed channel and playbook channel for that release
  3. Build:
    • Tags a new release (e.g. 1.1.0) and runs an official build which should be essentially identical to the last RC
    • Posts SHA key, md5 sum, and GPG signatures of the final build to Release: Self-Managed channel
    • Post in Release: Self-Managed with links to the Enterprise Edition and Team Edition sections
  4. Dev:
    • Verify the hashes (SHA-1, SHA-256, and md5) and GPG signatures are correct for both Enterprise Edition and Team Edition
    • Test upgrade from previous version to current version, following the Upgrade Guide with database upgrades on both MySQL and Postgres
    • Test upgrade from Team Edition to Enterprise Edition based on the Upgrade Guide
    • Test fresh install of current version, following the Install Guide
    • Review any changes made to install guides, and test if necessary
    • Ensure Security Policies page has been updated
    • Update dependancies after release branch is cut in mattermost-server, mattermost-webapp, desktop, mattermost-mobile, and mattermost-redux
  5. Logistics @hanna.park:
    • Order the release MVP winner's coaster
  6. Docs:
    • Remind PMs and Tech Writer to finalize docs
    • Merge the docs release branch to master and verify all changes on docs.mattermost.com once the build is up
    • Submit a correction PR for any incorrect formatting or other errors missed during the initial review
  7. Marketing:
    • Receive sign off on final version of MailChimp email blast and Twitter announcement and schedule for 08:00 PST on the date of marketing announcement
    • Finalize blog post for mattermost.com, test on mobile view, and set timer for 08:00 PST on the day of release

F. (T-minus 0 working days) Release Day

  1. Release Manager:
    • Post this checklist in Release Checklist channel
    • Verify all items in the last posted release checklist are complete
    • Update the server upgrade in-product notice
    • Schedule a release retrospective meeting, to be held within five days of the release
    • Prepare and post release metrics
    • Finalize release summary slide deck to prepare for posting for CSMs
    • Review and update company roadmap with which major features made it into the release
    • Update the Mattermost Wikipedia page with the latest version
    • Post the MVP winner announcement in the Contributors channel
      • Update MVP page with the most valued professional of the release
    • Add new release fix versions in Jira for the next few releases
    • Post key dates for the next release in the Release: Self-Managed channel and remove links to RC candidates and testing spreadsheet from the header
      • Make sure that statutory holidays for Canada and US are accounted for in the release dates
    • Check for any UserVoice feature suggestions that were completed in the current release
      • Find the release tweet and insert a link to the tweet next to the feature that shipped with the release
    • Close the release in Jira both for webapp and mobile (releases page)
    • Confirm that daily triage / release update meetings are scheduled.
    • Prepare tickets for the next release, with a corresponding vX.X prefix, and put the tickets in the appropriate sprints as follows:
      • The week RC is cut:
      • The week RC is cut:
      • The week RC is cut (for GitLab dev owner):
        • Test RC1 with the latest GitLab build during release testing cycle
      • Release week (for dependencies owner)
        • Upgrade dependencies for webapp, server, and Redux
      • Week after release (for GitLab dev owner)
      • The week before code complete (one for Apps and for Mobile):
        • If an existing ESR is going out of support next month, update the in-app prompts (mobile and desktop) to start detecting for the new minimum supported ESR version
        • If a new ESR is released next month, update the prompts to recommend upgrading to that version instead of the older ESR
    • Confirm that mattermost-docker has been updated to the latest version (contact the maintainer via direct message on community server if necessary)
    • Contact owner of Yunohost community installer to update install version number. See example
    • Turn on CrazyEgg for blog post page
    • Confirm marketing has been posted (mail announcement, tweets, blog posts)
      • Post in the Announcements channel
    • Update @mattermost Twitter profile with the next release date
    • Prepare retweet of GitLab release tweet (see example here)
  2. Docs:
    • Create a new branch on docs for the next release - vX.X-documentation
      • Submit a PR for changelog against the vX.X-documentation branch and add a Work in Progress label for it
      • Submit a PR to change version number in docs/source/conf.py against the vX.X-documentation branch
  3. Build:
  4. QA:
    • Merge any updates made to Selenium tests during release testing
    • Update RN server URLs to Rainforest

G. (T-plus 30 days) Update Mattermost Security Page

  1. Release Manager:
    • Post Mattermost Security Updates after reviewing with security lead
      • If a dot release is shipping with security fixes, do not post new details until T-plus 10 working days from the dot release ship date