Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-12219 Add Xodus Flow History Storage with Migration #7885

Merged
merged 1 commit into from
Oct 18, 2023

Conversation

exceptionfactory
Copy link
Contributor

Summary

NIFI-12219 Backports Flow Configuration History persistent storage using JetBrains Xodus as implemented for NIFI-12206 and adds migration capabilities to the support branch for subsequent NiFi version 1 releases.

The EntityStoreAuditService class contains the same implementation as merged for NIFI-12219, and the new FlowConfigurationHistoryMigrator supports automated migration from H2 to Xodus on startup. The migrator checks for the presence of the H2 database file named nifi-flow-audit.mv.db and proceeds with migration when found. The migration process includes checking for older H2 binary formats, supporting all previous versions of H2 from 1.4 through 2.2. The migration process then uses available paging capabilities from the H2-based AuditService to transfer records. Following successful migration, the FlowConfigurationHistoryMigrator renames nifi-flow-audit.mv.db with a backup timestamp extension.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 21

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

- Added EntityStoreAuditService implementation using JetBrains Xodus libraries
- Added FlowConfigurationHistoryMigrator for checking and migrating H2 databases containing Flow Configuration History to Xodus persistent store implementation
@markap14
Copy link
Contributor

Thanks for putting this together, @exceptionfactory! Code looks good. Did a quick test and everything appears to have been migrated properly. Able to see all of the events that I expected, including before/after values, timestamps, action taken, user... +1 will merge to support/nifi-1.x!

@markap14 markap14 merged commit 1a2d769 into apache:support/nifi-1.x Oct 18, 2023
8 checks passed
@exceptionfactory exceptionfactory added the hacktoberfest-accepted Hacktoberfest Accepted label Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hacktoberfest-accepted Hacktoberfest Accepted
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants