Releases: treeverse/lakeFS
v0.40.3
v0.40.2
v0.40.1
v0.40.0
Changelog
This is a big release for lakeFS with many notable improvements.
Some of these are breaking changes. It's always a tough decision to introduce a change that isn't backwards compatible,
but we felt that at this stage they represent a significant enough benefit to be worth it.
Going forward, our goal is to make as few of those as possible, as we near a 1.0.0 release.
Here are the most notable changes:
lakeFS is now OpenAPI 3.0 compliant ✨
The lakeFS API has been migrated from OpenAPI 2.0 to OpenAPI 3.0.
OpenAPI 3.0 includes many improvements over the previous version: Cookie based authentication, reusable query parameters, better JSON Schema support and more.
While homegrown clients that simply use the lakeFS API as a REST inteface will continue to work,
client that relied on OpenAPI 2.0 specific behaviors will stop working.
This includes the previously recommended bravado based client for Python. For that reason, we're also releasing an officially supported Python client:
lakeFS now ships with a native Python client ✨
It's now as simple as:
$ pip install lakefs-client~=0.40.0And then:
import lakefs_client
from lakefs_client.client import LakeFSClient
lakefs = LakeFSClient(lakefs_client.Configuration(
username='AKIAIOSFODNN7EXAMPLE',
password='...',
host='http://lakefs.example.com'))
lakefs.branches.list_branches(repository='my-repo') # Or any other API actionThis client is officially supported and distributed by the lakeFS team, and will be released in conjunction with lakeFS releases, so it should always align in capabilities with the latest lakeFS versions.
For more information, see the Python Client Documentation.
Native Spark client, allowing to export a commit (or set of commits) to another object store ✨
Using Apache Spark, lakeFS users can now quickly export the contents of a branch to an external location (say, S3 bucket). Exporting committed data will be parallelized using Spark workers to support copying millions of objects in minutes.
This is the first feature released based on lakeFS' Spark integration (soon to be followed by data retention for stale objects), and a native lakefs:// filesystem support for Spark).
For more information, see the Export Job configuration Documentation.
lakeFS standardized URIs ✨
The lakeFS CLI now supports a standardized URI in the form: lakefs://<repository>/<ref>/<path>.
Additionally, the CLI now allows setting a $LAKECTL_BASE_URI environment variable that, if set, will prefix any relative URI used.
For example, instead of:
$ lakectl diff lakefs://my-repository/my-branch lakefs://my-repository/main
$ lakectl fs ls lakefs://my-repository/my-branch/path/It's now possible to simply do:
$ export LAKECTL_BASE_URI="lakefs://my-repository/"
$ lakectl diff my-branch main
$ lakectl fs ls mybranch/path/For more information, see the CLI Command Reference Documentation.
Complete UI Overhaul
Making it faster, more responsive and contains many improvements to pagination, commit browsing and action views.
Full Feature list
[UI]Complete UI overhaul 💅 (#1766)[Spark]Spark client that allows exporting from lakeFS to an object store ✨ (#1658)[Metastore]Support metastore copy between two different hive metastores ✨ (#1704)[API Gateway]BREAKING: Migrated to OpenAPI 3.0 💣 (#1667)[Python SDK]Native lakeFS Python Client ✨ (#1725)[Graveler]BREAKING: commit parents order for merge-commits are now [destination, source] instead of [source, destination] 💣 (#1754)[CLI]BREAKING:lakefs://URIs are now standard, replacing@with/to denote ref 💣 (#1717)[CLI]$LAKECTL_BASE_URIprefixes all lakectl URIs for more a human-friendly CLI 🥰 (#1717)[CLI]Support non-seekable stdin (-arg) in "fs upload" command 🥰 (#1672)
Bug Fixes
[S3 Gateway]Avoid logging v2 sigs on failure 🔒 (#1679)[Graveler]Limit length of Graveler serialization 🐞 (#1682)[Graveler]Fix merge large changes performance (#1652)[S3 Gateway]Handle no path for delete objects in gateway 🐞 (#1708)[API Gateway]API merge message is optional 🐞 (#1710)[API Gateway]Fix auth pagination 🐞 (#1755)[API Gateway]List repository actions should not check branch existence 🐞 (#1743)
As always, we hang around at #help on the lakeFS Slack to assist and answer questions!
v0.33.1
Changelog
b253f81 Actions tab UI (hook runs) (#1564)
cd8dbce Adding arm64 to our release binaries (#1585)
7e227b3 Convert merge errors to bad requests (#1555)
9b3ac8e Special warning before delete import branch (#1582)
5b31659 Upgrade to Go 1.16.2 (#1619)
c68123e batch hot paths for a very short duration (#1618)
e053cd3 gateway requests reaching ui handler to fail gracefully (#1578)
759baa8 lakectl validate action file (#1601)
v0.33.0
Main benefits
- Azure Blob Storage support Azure AD authentication
- Webhooks: actions mechanism for running hooks on pre-commit and pre-merge (CI/CD v1 milestone)
Changelog
- Protoc and swagger validation as part of CI using docker (#1498)
- Expose API metadata handlers (#1494)
- Fix diff throwing 500 instead of 404 when ref/repo not found (#1492)
- Make swagger.yml pass schema validation (#1495)
- Fixed filtering repositories in UI in case an exact repo name was passed (#1493)
- Enable delete repository action from the UI (#1372) thanks @shamikakumar