Release v0.197.0

[0.197.0] - 2024-08-22

Changed

• Breaking: Using DataFusion's enable_ident_normalization = false setting to work with upper case identifiers without needing to put quotes everywhere. This may impact your root and derivative datasets.
• Datafusion transform engine was updated to latest version and includes JSON extensions
• Breaking: Push ingest from csv format will default to header: true in case schema was not explicitly provided
• Access token with duplicate names can be created if such name exists but was revoked
• Many examples were simplified due to ident normalization changes

Fixed

• Crash in kamu login command on 5XX server responses

Added

• HTTP sources now include User-Agent header that defaults to kamu-cli/{major}.{minor}.{patch}
• Externalized configuration of HTTP source parameters like timeouts and redirects
• CI: build sqlx-cli image if it is missing

Release v0.196.0

[0.196.0] - 2024-08-19

Added

• The /ingest endpoint will try to infer the media type of file by extension if not specified explicitly during upload.
  This resolves the problem with 415 Unsupported Media Type errors when uploading .ndjson files from the Web UI.
• Private Datasets, preparation work:
  • Added SQLite-specific implementation of ReBAC repository
  • Added SQLite-specific implementation of DatasetEntryRepository
• internal-error crate:
  • Added InternalError::reason() to get the cause of an error
  • Added methods to ResultIntoInternal:
    • map_int_err() - shortcut for result.int_err().map_err(...) combination
    • context_int_err() - ability to add a context message to an error
• Added macro database_transactional_test!() to minimize boilerplate code

Changed

• sqlx v0.8
• Renamed setConfigSchedule GQL api to setConfigIngest. Also extended
  setConfigIngest with new field fetchUncacheable which indicates to ingone cache
  during ingest step

Release v0.195.1

[0.195.1] - 2024-08-16

Fixed

• Add reset ENUM variant to dataset_flow_type in postgres migration

Release v0.195.0

[0.195.0] - 2024-08-16

Added

• Reliable transaction-based internal cross-domain message passing component (MessageOutbox), replacing EventBus
  • Metadata-driven producer/consumer annotations
  • Immediate and transaction-backed message delivery
  • Background transactional message processor, respecting client idempotence
• Persistent storage for flow configuration events

Changed

• Upgraded to datafusion v41 (#713)
• Introduced use case layer, encapsulating authorization checks and action validations, for first 6 basic dataset scenarios
  (creating, creating from snapshot, deleting, renaming, committing an event, syncing a batch of events),
• Separated DatasetRepository on read-only and read-write parts
• Isolated time-source library

Fixed

• E2E: added additional force off colors to exclude sometimes occurring ANSI color sequences
• E2E: modify a workaround for MySQL tests

Release v0.194.1

[0.194.1] - 2024-08-14

Fixed

• Add recursive field to Reset flow configurations in GQL Api which triggers HardCompaction in KeepMetadataOnly mode flow for each owned downstream dependency

Release v0.194.0

[0.194.0] - 2024-08-13

Changed

• Change mode argument for DatasetEnvVarsConfig to enabled: Option<bool>

Added

• New Reset flow in GQL Api which can be triggered manually for Root and Derivative datasets
• Private Datasets, preparation work:
  • Added in-mem implementation of ReBAC repository
  • Added in-mem implementation of DatasetEntryRepository

Release v0.193.1

[0.193.1] - 2024-08-09

Fixed

• Panic for EXECUTE_TRANSFORM flow without dataset env vars enabled feature

Release v0.193.0

[0.193.0] - 2024-08-07

Added

• kamu add command accepts optional --name argument to add a snapshot under a different name

Release v0.192.0

[0.192.0] - 2024-08-07

Added

• kamu --no-color to disable color output in the terminal.

Changed

• New recursive flag for CompactionConditionFull input to trigger
  Hard compaction with keep metadata only mode for each derived dataset
• E2E: Reorganized work with tests that call kamu-cli:
  • Added kamu-cli-puppet crate to allow kamu-cli to be run as a separate process from tests
  • Removed past kamu-cli wrapper that ran in-process.
  • Some of kamu-cli tests that are inherently E2E are moved and adapted to E2E scope (in-mem area)
  • For convenience, the test run macros are now procedural
  • Various Windows-related tweaks & fixes

Fixed

• Return RootDatasetCompacted error for manual triggered EXECUTE_TRANSFROM flows
• Using new Spark image that fixes serialization errors when working with large GIS datasets
• Fixed container runtime for systems using SELinux

Release v0.191.5

[0.191.5] - 2024-07-30

Fixed

• Ingest flow panic with database api mode