-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 4714877
Showing
120 changed files
with
10,743 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
name: SonarCloud OpenLMIS-template-service Pipeline | ||
on: | ||
push: | ||
branches: | ||
- master | ||
pull_request: | ||
types: [opened, synchronize, reopened] | ||
jobs: | ||
build: | ||
name: SonarCloud Analyze | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
with: | ||
fetch-depth: 0 | ||
- name: Set up JDK 17 | ||
uses: actions/setup-java@v3 | ||
with: | ||
java-version: 17 | ||
distribution: 'zulu' | ||
- name: Cache SonarCloud packages | ||
uses: actions/cache@v3 | ||
with: | ||
path: ~/.sonar/cache | ||
key: ${{ runner.os }}-sonar | ||
restore-keys: ${{ runner.os }}-sonar | ||
- name: Cache Gradle packages | ||
uses: actions/cache@v3 | ||
with: | ||
path: ~/.gradle/caches | ||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }} | ||
restore-keys: ${{ runner.os }}-gradle | ||
- name: Build with Docker Compose | ||
continue-on-error: true | ||
run: | | ||
curl -o .env -L https://raw.githubusercontent.com/OpenLMIS/openlmis-ref-distro/master/settings-sample.env | ||
docker-compose -f docker-compose.builder.yml run builder | ||
sudo chown -R $(whoami) ./ | ||
cp ./build/reports/jacoco/test/jacocoTestReport.xml report.xml | ||
rm -rf ./build | ||
- name: Analyze | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} | ||
run: | | ||
sudo chown -R $(whoami) ./ | ||
./gradlew sonarqube --info |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
*~ | ||
*.iml | ||
*.ipr | ||
*.iws | ||
.gradle/ | ||
build/ | ||
out/ | ||
.idea/ | ||
.env | ||
.tx/ | ||
node_modules/ | ||
demo-data/*.sql |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
FROM openlmis/service-base:4 | ||
|
||
COPY build/libs/*.jar /service.jar | ||
COPY src/main/resources/db/demo-data/*.csv /demo-data/ | ||
COPY build/schema /schema | ||
COPY build/consul /consul |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
# Writing Schema Migrations (Using Flyway) | ||
|
||
This document serves as a set of guidelines / best practices for writing Flyway schema migrations | ||
for an OpenLMIS Service. | ||
|
||
## Intro to Flyway migrating a Database | ||
|
||
* Creates a table in the schema called `schema_version`. | ||
* Is smart enough to only apply new migrations to the database. | ||
* Goes through the Flyway configured folders, looking for migration scripts. | ||
* If found, compares them to the migrations already applied to see if any new migration scripts | ||
need to be applied. | ||
* If new migration scripts are found, applies them to the database and adds them to the | ||
`schema_version` table. | ||
* Flyway can also perform other database tasks, such as clean, validate, repair, etc.. | ||
* There is a Spring Boot Flyway plugin, which OpenLMIS services use, so that on Spring Boot | ||
application startup, Flyway is automatically run. | ||
|
||
## General Guidelines | ||
|
||
* Think about semantics in a migration; keep the changes conceptually self-contained. A migration | ||
should do one "thing" well similar to best practices with Git commits. | ||
* Keep a migration relatively small; it should only have changes that are all related to each other. | ||
* Try to keep a migration "tied to the code"; meaning it should be added to the codebase around | ||
the same time as its related code changes. Ideally, this would be in the same source control commit. | ||
* A migration should generally only have schema changes. The only time a migration would have | ||
data changes (i.e. INSERT/DELETE/etc. statements) is if it is considered "bootstrap" data. | ||
* **Note:** See [Bootstrap Data Guidelines](#bootstrap) below. | ||
* Migration scripts shouldn't have to check if a table/column exists. The state of the DB (schema) | ||
is already set by the migration scripts that preceed it. | ||
* Migration scripts shouldn't change. Especially not after others may have run them. NEVER after | ||
they have made it into a release. | ||
* Do not create reverse migrations (a migration that reverses an earlier migration). This | ||
guideline is to keep things simple if the migrations in a dot release upgrade are only partially | ||
successful. In this case, the recommended upgrade set of steps would be: | ||
* Perform a full backup of the database. | ||
* Do an in-place upgrade. | ||
* If the upgrade fails, revert the deployed service and revert the database to the backup. | ||
* Do not use database dumps or diffs to auto-generate a migration script. (NO pg_dumps!) | ||
* Try to provide comments on table columns. This is especially important as reports access | ||
database tables directly. | ||
* Use the `gradle generateMigration` task to create the migration script. This will give the | ||
correct format for the script name (`timestamp__migration_name.sql`). | ||
* When naming a migration script, try to be descriptive of what is in the migration script. For | ||
example: if you add an email column to a users table, the script might be called | ||
`20170123120000000__add_email_to_users.sql`. | ||
|
||
## <a name="bootstrap">Guidelines about Loading Bootstrap Data</a> | ||
|
||
There is a certain set of data without which the system cannot function. In OpenLMIS, this is | ||
called "bootstrap" data. This is the only kind of data changes that are allowed in migrations. | ||
|
||
There are two ways that bootstrap data needs to be loaded into the system: | ||
|
||
1. Inter-Service (within a Service)- a Service on startup needs to load data into its own database. | ||
* An example would be the Reference Data Service needing to insert rights into its own database | ||
for the system to use. | ||
2. Intra-Service (from one Service to another) - a Service on startup needs to load data into | ||
another Service's database. | ||
* An example would be the Requisition Service needs to insert requisition-specific rights into | ||
the Reference Data Service's database for the system to use. | ||
|
||
Below are the guidelines of how bootstrap data can be loaded for both ways: | ||
|
||
1. Within a Service the standard Flyway migrations may load bootstrap data and "upgrade" it as | ||
the Service matures. | ||
* A notable exception is the first "admin" user in the system. It is assumed that this user | ||
will be modified or deleted and should not be updated in future dot releases. | ||
2. A service loading bootstrap data into another service is more complex to solve as it requires | ||
service-to-service communication. For 3.0, since the only situation where this is expected is the | ||
creation of rights, all rights will be created by the Reference Data Service, including rights "owned" by other services. | ||
* For future releases after 3.0, an implementation will be provided where a Service contacts the | ||
Reference Data Service to create rights, but the design of this has not yet been determined. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
#!/usr/bin/env groovy | ||
import hudson.tasks.test.AbstractTestResultAction | ||
|
||
properties([ | ||
[ | ||
$class: 'ThrottleJobProperty', | ||
categories: ['pipeline'], | ||
throttleEnabled: true, | ||
throttleOption: 'category' | ||
] | ||
]) | ||
pipeline { | ||
agent none | ||
options { | ||
buildDiscarder(logRotator( | ||
numToKeepStr: env.BRANCH_NAME.equals("master") ? '15' : '3', | ||
daysToKeepStr: env.BRANCH_NAME.equals("master") || env.BRANCH_NAME.startsWith("rel-") ? '' : '7', | ||
artifactDaysToKeepStr: env.BRANCH_NAME.equals("master") || env.BRANCH_NAME.startsWith("rel-") ? '' : '3', | ||
artifactNumToKeepStr: env.BRANCH_NAME.equals("master") || env.BRANCH_NAME.startsWith("rel-") ? '' : '1' | ||
)) | ||
disableConcurrentBuilds() | ||
skipStagesAfterUnstable() | ||
} | ||
environment { | ||
COMPOSE_PROJECT_NAME = "template${BRANCH_NAME}" | ||
} | ||
stages { | ||
stage('Preparation') { | ||
agent any | ||
steps { | ||
withCredentials([usernamePassword( | ||
credentialsId: "cad2f741-7b1e-4ddd-b5ca-2959d40f62c2", | ||
usernameVariable: "USER", | ||
passwordVariable: "PASS" | ||
)]) { | ||
sh 'set +x' | ||
sh 'docker login -u $USER -p $PASS' | ||
} | ||
script { | ||
CURRENT_BRANCH = env.GIT_BRANCH // needed for agent-less stages | ||
def properties = readProperties file: 'gradle.properties' | ||
if (!properties.serviceVersion) { | ||
error("serviceVersion property not found") | ||
} | ||
VERSION = properties.serviceVersion | ||
STAGING_VERSION = properties.serviceVersion | ||
if (CURRENT_BRANCH != 'master' || (CURRENT_BRANCH == 'master' && !VERSION.endsWith("SNAPSHOT"))) { | ||
STAGING_VERSION += "-STAGING" | ||
} | ||
currentBuild.displayName += " - " + VERSION | ||
} | ||
} | ||
post { | ||
failure { | ||
script { | ||
notifyAfterFailure() | ||
} | ||
} | ||
} | ||
} | ||
stage('Build') { | ||
agent any | ||
environment { | ||
PATH = "/usr/local/bin/:$PATH" | ||
STAGING_VERSION = "${STAGING_VERSION}" | ||
} | ||
steps { | ||
withCredentials([file(credentialsId: '8da5ba56-8ebb-4a6a-bdb5-43c9d0efb120', variable: 'ENV_FILE')]) { | ||
script { | ||
try { | ||
sh( script: "./ci-buildImage.sh" ) | ||
currentBuild.result = processTestResults('SUCCESS') | ||
} | ||
catch (exc) { | ||
currentBuild.result = processTestResults('FAILURE') | ||
if (currentBuild.result == 'FAILURE') { | ||
error(exc.toString()) | ||
} | ||
} | ||
} | ||
} | ||
} | ||
post { | ||
success { | ||
archive 'build/libs/*.jar,build/resources/main/api-definition.html, build/resources/main/ version.properties' | ||
} | ||
unstable { | ||
script { | ||
notifyAfterFailure() | ||
} | ||
} | ||
failure { | ||
script { | ||
notifyAfterFailure() | ||
} | ||
} | ||
cleanup { | ||
script { | ||
sh "sudo rm -rf ${WORKSPACE}/{*,.*} || true" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
post { | ||
fixed { | ||
script { | ||
BRANCH = "${env.GIT_BRANCH}".trim() | ||
if (BRANCH.equals("master") || BRANCH.startsWith("rel-")) { | ||
slackSend color: 'good', message: "${env.JOB_NAME} - #${env.BUILD_NUMBER} Back to normal" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
|
||
def notifyAfterFailure() { | ||
messageColor = 'danger' | ||
if (currentBuild.result == 'UNSTABLE') { | ||
messageColor = 'warning' | ||
} | ||
BRANCH = "${env.GIT_BRANCH}".trim() | ||
if (BRANCH.equals("master") || BRANCH.startsWith("rel-")) { | ||
slackSend color: "${messageColor}", message: "${env.JOB_NAME} - #${env.BUILD_NUMBER} ${env.STAGE_NAME} ${currentBuild.result} (<${env.BUILD_URL}|Open>)" | ||
} | ||
emailext subject: "${env.JOB_NAME} - #${env.BUILD_NUMBER} ${env.STAGE_NAME} ${currentBuild.result}", | ||
body: """<p>${env.JOB_NAME} - #${env.BUILD_NUMBER} ${env.STAGE_NAME} ${currentBuild.result}</p><p>Check console <a href="${env.BUILD_URL}">output</a> to view the results.</p>""", | ||
recipientProviders: [[$class: 'CulpritsRecipientProvider'], [$class: 'DevelopersRecipientProvider']] | ||
} | ||
|
||
def processTestResults(status) { | ||
checkstyle pattern: '**/build/reports/checkstyle/*.xml' | ||
pmd pattern: '**/build/reports/pmd/*.xml' | ||
junit '**/build/test-results/*/*.xml' | ||
|
||
AbstractTestResultAction testResultAction = currentBuild.rawBuild.getAction(AbstractTestResultAction.class) | ||
if (testResultAction != null) { | ||
failuresCount = testResultAction.failCount | ||
echo "Failed tests count: ${failuresCount}" | ||
if (failuresCount > 0) { | ||
echo "Setting build unstable due to test failures" | ||
status = 'UNSTABLE' | ||
} | ||
} | ||
|
||
return status | ||
} |
Oops, something went wrong.