Release v0.204.2

[0.204.2] - 2024-09-26

Fixed

• kamu init: fixed regression in case of using exists_ok flag

Release v0.204.1

[0.204.1] - 2024-09-25

Fixed

• Fixed build regression, in case web-ui feature flag is used

Release v0.203.1

[0.203.1] - 2024-09-24

Added

• Added database migration & scripting to create an application user with restricted permissions
• kamu delete command will respect dependency graph ordering allowing to delete multiple datasets without encountering dangling reference

Release v0.203.0

[0.203.0] - 2024-09-22

Added

• Support List and Struct arrow types in json and json-aoa encodings

Release v0.202.1

[0.202.1] - 2024-09-20

Fixed

• Open Telemetry integration fixes

Release v0.202.0

[0.202.0] - 2024-09-20

Changed

• Major dependency upgrades:
  • DataFusion 42
  • HTTP stack v.1
  • Axum 0.7
  • latest AWS SDK
  • latest versions of all remaining libs we depend on
• Outbox refactoring towards true parallelism via Tokio spaned tasks instead of futures

Fixed

• Failed flows should still propagate finishedAt time
• Eliminate span.enter, replaced with instrument everywhere

Release v0.201.0

[0.201.0] - 2024-09-18

Added

• REST API: New /verify endpoint allows verification of query commitment as per documentation (#831)

Changed

• Outbox main loop was revised to minimize the number of transactions:
  • split outbox into planner and consumption jobs components
  • planner analyzes current state and loads bunch of unprocessed messages within a 1 transaction only
  • consumption jobs invoke consumers and detect their failures
• Detecting concurrent modifications in flow and task event stores
• Improved and cleaned handling of flow abortions at different stages of processing
• Revised implementation of flow scheduling to avoid in-memory time wheel:
  • recording FlowEventScheduledForActivation event (previously, placement moment into the time wheel)
  • replaced binary heap based time wheel operations with event store queries
  • Postgres/SQlite event stores additionally track activation time for the waiting flows
  • in-memory event store keeps prepared map-based lookup structures for activation time

Release v0.200.0

[0.200.0] - 2024-09-13

Added

• Added first integration of Prometheus metrics starting with Outbox
• Added --metrics CLI flag that will dump metrics into a file after command execution

Changed

• Telemetry improvements:
  • Improved data collected around transactional code
  • Revised associating span objects with large JSON structures such as messages
  • Suppressed several noisy, but not very useful events
• Improved Outbox stability when message consumers fail
• Similarly, Task Executor keeps executing next tasks in case running a task results in an internal error

Release v0.199.3

[0.199.3] - 2024-09-11

Fixed

• Associating correct input dataset that was hard compacted with the error during transformation of derived dataset

Release v0.199.2

[0.199.2] - 2024-09-09

Added

• REST API: The /query endpoint now supports response proofs via reproducibility and signing (#816)
• REST API: New /{dataset}/metadata endpoint for retrieving schema, description, attachments etc. (#816)

Fixed

• Fixed unguaranteed ordering of events when restoring event sourcing aggregates
• Enqueuing and cancelling future flows should be done with transactions taken into account (via Outbox)