Skip to content

Commit

Permalink
OLMIS-8003: Init repo
Browse files Browse the repository at this point in the history
  • Loading branch information
sradziszewski committed Oct 6, 2024
0 parents commit 4714877
Show file tree
Hide file tree
Showing 120 changed files with 10,743 additions and 0 deletions.
47 changes: 47 additions & 0 deletions .github/workflows/sonar-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: SonarCloud OpenLMIS-template-service Pipeline
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
build:
name: SonarCloud Analyze
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: 17
distribution: 'zulu'
- name: Cache SonarCloud packages
uses: actions/cache@v3
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Gradle packages
uses: actions/cache@v3
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- name: Build with Docker Compose
continue-on-error: true
run: |
curl -o .env -L https://raw.githubusercontent.com/OpenLMIS/openlmis-ref-distro/master/settings-sample.env
docker-compose -f docker-compose.builder.yml run builder
sudo chown -R $(whoami) ./
cp ./build/reports/jacoco/test/jacocoTestReport.xml report.xml
rm -rf ./build
- name: Analyze
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: |
sudo chown -R $(whoami) ./
./gradlew sonarqube --info
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
*~
*.iml
*.ipr
*.iws
.gradle/
build/
out/
.idea/
.env
.tx/
node_modules/
demo-data/*.sql
6 changes: 6 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM openlmis/service-base:4

COPY build/libs/*.jar /service.jar
COPY src/main/resources/db/demo-data/*.csv /demo-data/
COPY build/schema /schema
COPY build/consul /consul
331 changes: 331 additions & 0 deletions ERROR_HANDLING.md

Large diffs are not rendered by default.

73 changes: 73 additions & 0 deletions FLYWAY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Writing Schema Migrations (Using Flyway)

This document serves as a set of guidelines / best practices for writing Flyway schema migrations
for an OpenLMIS Service.

## Intro to Flyway migrating a Database

* Creates a table in the schema called `schema_version`.
* Is smart enough to only apply new migrations to the database.
* Goes through the Flyway configured folders, looking for migration scripts.
* If found, compares them to the migrations already applied to see if any new migration scripts
need to be applied.
* If new migration scripts are found, applies them to the database and adds them to the
`schema_version` table.
* Flyway can also perform other database tasks, such as clean, validate, repair, etc..
* There is a Spring Boot Flyway plugin, which OpenLMIS services use, so that on Spring Boot
application startup, Flyway is automatically run.

## General Guidelines

* Think about semantics in a migration; keep the changes conceptually self-contained. A migration
should do one "thing" well similar to best practices with Git commits.
* Keep a migration relatively small; it should only have changes that are all related to each other.
* Try to keep a migration "tied to the code"; meaning it should be added to the codebase around
the same time as its related code changes. Ideally, this would be in the same source control commit.
* A migration should generally only have schema changes. The only time a migration would have
data changes (i.e. INSERT/DELETE/etc. statements) is if it is considered "bootstrap" data.
* **Note:** See [Bootstrap Data Guidelines](#bootstrap) below.
* Migration scripts shouldn't have to check if a table/column exists. The state of the DB (schema)
is already set by the migration scripts that preceed it.
* Migration scripts shouldn't change. Especially not after others may have run them. NEVER after
they have made it into a release.
* Do not create reverse migrations (a migration that reverses an earlier migration). This
guideline is to keep things simple if the migrations in a dot release upgrade are only partially
successful. In this case, the recommended upgrade set of steps would be:
* Perform a full backup of the database.
* Do an in-place upgrade.
* If the upgrade fails, revert the deployed service and revert the database to the backup.
* Do not use database dumps or diffs to auto-generate a migration script. (NO pg_dumps!)
* Try to provide comments on table columns. This is especially important as reports access
database tables directly.
* Use the `gradle generateMigration` task to create the migration script. This will give the
correct format for the script name (`timestamp__migration_name.sql`).
* When naming a migration script, try to be descriptive of what is in the migration script. For
example: if you add an email column to a users table, the script might be called
`20170123120000000__add_email_to_users.sql`.

## <a name="bootstrap">Guidelines about Loading Bootstrap Data</a>

There is a certain set of data without which the system cannot function. In OpenLMIS, this is
called "bootstrap" data. This is the only kind of data changes that are allowed in migrations.

There are two ways that bootstrap data needs to be loaded into the system:

1. Inter-Service (within a Service)- a Service on startup needs to load data into its own database.
* An example would be the Reference Data Service needing to insert rights into its own database
for the system to use.
2. Intra-Service (from one Service to another) - a Service on startup needs to load data into
another Service's database.
* An example would be the Requisition Service needs to insert requisition-specific rights into
the Reference Data Service's database for the system to use.

Below are the guidelines of how bootstrap data can be loaded for both ways:

1. Within a Service the standard Flyway migrations may load bootstrap data and "upgrade" it as
the Service matures.
* A notable exception is the first "admin" user in the system. It is assumed that this user
will be modified or deleted and should not be updated in future dot releases.
2. A service loading bootstrap data into another service is more complex to solve as it requires
service-to-service communication. For 3.0, since the only situation where this is expected is the
creation of rights, all rights will be created by the Reference Data Service, including rights "owned" by other services.
* For future releases after 3.0, an implementation will be provided where a Service contacts the
Reference Data Service to create rights, but the design of this has not yet been determined.
147 changes: 147 additions & 0 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
#!/usr/bin/env groovy
import hudson.tasks.test.AbstractTestResultAction

properties([
[
$class: 'ThrottleJobProperty',
categories: ['pipeline'],
throttleEnabled: true,
throttleOption: 'category'
]
])
pipeline {
agent none
options {
buildDiscarder(logRotator(
numToKeepStr: env.BRANCH_NAME.equals("master") ? '15' : '3',
daysToKeepStr: env.BRANCH_NAME.equals("master") || env.BRANCH_NAME.startsWith("rel-") ? '' : '7',
artifactDaysToKeepStr: env.BRANCH_NAME.equals("master") || env.BRANCH_NAME.startsWith("rel-") ? '' : '3',
artifactNumToKeepStr: env.BRANCH_NAME.equals("master") || env.BRANCH_NAME.startsWith("rel-") ? '' : '1'
))
disableConcurrentBuilds()
skipStagesAfterUnstable()
}
environment {
COMPOSE_PROJECT_NAME = "template${BRANCH_NAME}"
}
stages {
stage('Preparation') {
agent any
steps {
withCredentials([usernamePassword(
credentialsId: "cad2f741-7b1e-4ddd-b5ca-2959d40f62c2",
usernameVariable: "USER",
passwordVariable: "PASS"
)]) {
sh 'set +x'
sh 'docker login -u $USER -p $PASS'
}
script {
CURRENT_BRANCH = env.GIT_BRANCH // needed for agent-less stages
def properties = readProperties file: 'gradle.properties'
if (!properties.serviceVersion) {
error("serviceVersion property not found")
}
VERSION = properties.serviceVersion
STAGING_VERSION = properties.serviceVersion
if (CURRENT_BRANCH != 'master' || (CURRENT_BRANCH == 'master' && !VERSION.endsWith("SNAPSHOT"))) {
STAGING_VERSION += "-STAGING"
}
currentBuild.displayName += " - " + VERSION
}
}
post {
failure {
script {
notifyAfterFailure()
}
}
}
}
stage('Build') {
agent any
environment {
PATH = "/usr/local/bin/:$PATH"
STAGING_VERSION = "${STAGING_VERSION}"
}
steps {
withCredentials([file(credentialsId: '8da5ba56-8ebb-4a6a-bdb5-43c9d0efb120', variable: 'ENV_FILE')]) {
script {
try {
sh( script: "./ci-buildImage.sh" )
currentBuild.result = processTestResults('SUCCESS')
}
catch (exc) {
currentBuild.result = processTestResults('FAILURE')
if (currentBuild.result == 'FAILURE') {
error(exc.toString())
}
}
}
}
}
post {
success {
archive 'build/libs/*.jar,build/resources/main/api-definition.html, build/resources/main/ version.properties'
}
unstable {
script {
notifyAfterFailure()
}
}
failure {
script {
notifyAfterFailure()
}
}
cleanup {
script {
sh "sudo rm -rf ${WORKSPACE}/{*,.*} || true"
}
}
}
}
}
post {
fixed {
script {
BRANCH = "${env.GIT_BRANCH}".trim()
if (BRANCH.equals("master") || BRANCH.startsWith("rel-")) {
slackSend color: 'good', message: "${env.JOB_NAME} - #${env.BUILD_NUMBER} Back to normal"
}
}
}
}
}

def notifyAfterFailure() {
messageColor = 'danger'
if (currentBuild.result == 'UNSTABLE') {
messageColor = 'warning'
}
BRANCH = "${env.GIT_BRANCH}".trim()
if (BRANCH.equals("master") || BRANCH.startsWith("rel-")) {
slackSend color: "${messageColor}", message: "${env.JOB_NAME} - #${env.BUILD_NUMBER} ${env.STAGE_NAME} ${currentBuild.result} (<${env.BUILD_URL}|Open>)"
}
emailext subject: "${env.JOB_NAME} - #${env.BUILD_NUMBER} ${env.STAGE_NAME} ${currentBuild.result}",
body: """<p>${env.JOB_NAME} - #${env.BUILD_NUMBER} ${env.STAGE_NAME} ${currentBuild.result}</p><p>Check console <a href="${env.BUILD_URL}">output</a> to view the results.</p>""",
recipientProviders: [[$class: 'CulpritsRecipientProvider'], [$class: 'DevelopersRecipientProvider']]
}

def processTestResults(status) {
checkstyle pattern: '**/build/reports/checkstyle/*.xml'
pmd pattern: '**/build/reports/pmd/*.xml'
junit '**/build/test-results/*/*.xml'

AbstractTestResultAction testResultAction = currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
if (testResultAction != null) {
failuresCount = testResultAction.failCount
echo "Failed tests count: ${failuresCount}"
if (failuresCount > 0) {
echo "Setting build unstable due to test failures"
status = 'UNSTABLE'
}
}

return status
}
Loading

0 comments on commit 4714877

Please sign in to comment.