Iceberg Destination Connector which supports REST Catalog.

[cloudcheflabs]

Tell us about the new connector you’d like to have

• Which source and which destination?

Iceberg destination connector which supports REST catalog: https://github.com/apache/iceberg/tree/master/core/src/main/java/org/apache/iceberg/rest

• Do you need a specific version of the underlying data source e.g: you specifically need support for an older version of the API or DB?

No

Describe the context around this new connector

• Why do you need this integration? How does your team intend to use the data? This helps us understand the use case.

Iceberg REST catalog is simple and flexible to manage meta data of data catalog.

• How often do you want to run syncs?

Every an hour.

• If this is an API source connector, which entities/endpoints do you need supported?
• If the connector is for a paid service, can we name you as a mutual user when we subscribe for an account? Which company should we name?

Describe the alternative you are considering or using

What are you considering doing if you don’t have this integration through Airbyte?

It seems that airbyte is using spark to save data as iceberg using data catalogs like hive, jdbc and hadoop iceberg catalog. In similar approach, I need to write spark job which connects to Iceberg REST Catalog to save data as iceberg table in S3. Take a look at my spark codes: https://github.com/cloudcheflabs/iceberg-rest-catalog/blob/master/spark/src/test/java/com/cloudcheflabs/iceberg/catalog/rest/RunSparkWithIcebergRestCatalog.java

Are you willing to submit a PR?

No.