Implement full PG TAM

[dpxcc]

What feature are you requesting?

Implement full PG Table Access Method (TAM) for columnstore tables to enable reading/writing of columnstore tables within PG

Why are you requesting this feature?

In pg_mooncake v0.1.0, queries involving columnstore tables are fully pushed down to DuckDB. While this approach is often fine and performs better than executing the query in PG, there are limitations due to feature gaps. Although DuckDB is largely compatible with PG, it lacks support for several features, such as RCTEs, triggers, and others

What is your proposed implementation for this feature?

Develop a full PG TAM for columnstore tables, allowing PG to natively read and write these tables. Currently, most of the TAM routines for columnstore tables are not implemented

In this model, we build a storage layer with data files stored in an object store and metadata managed in PG with transactional support. Both PG and DuckDB can read from and write to this shared storage layer. At the planner stage, we use DuckDB as a speedy execution engine and push down as much computation to it as possible. Operators that cannot be executed in DuckDB seamlessly fall back to PostgreSQL, ensuring maximum compatibility

For the initial version, implement TAM in PG based on the already built shared storage layer, allowing queries to be executed entirely in either PostgreSQL or DuckDB. Fine-grained planning can be introduced as a follow-up task in the future

[ritwizsinha]

@dpxcc please assign

[ritwizsinha]

This might be a bigger change and might take some time, please inform me if my progress is slow and this needs to be released earlier, I'll try to expedite