Reading GeoParquet in QGIS

[rafapereirabr]

Feature description

GeoParquet is a new geospatial vector data format that is built on Parquet. It looks super efficient in terms of storage and loading/saving data files. It would be great if QGIS had the capability to read GeoParquet files without users having to develop plugins.

Here is an example of a GeoParquet file: example.parquet.

Additional context

No response

[DelazJ]

The best way to have QGIS read it is to have it read by GDAL, which seems to be the case if I understand https://gdal.org/drivers/vector/parquet.html. So the fix is to install a QGIS version that embarks GDAL 3.5 (maybe Windows nightlies have it already?)

[jef-n]

So the fix is to install a QGIS version that embarks GDAL 3.5 (maybe Windows nightlies have it already?)

https://lists.osgeo.org/pipermail/qgis-developer/2022-May/064730.html

[nmarchio]

Just to flag: it looks like the MacOS QGIS 3.26.3 currently ships with GDAL 3.3.2 which doesn't support GeoParquet files (Windows ships with GDAL 3.5.1).

I did some testing and was able to confirm this is the case. GeoParquet files load fine on Windows and in MacOS QGIS throws an invalid data source error.

Would it be possible to update the MacOS QGIS release to use GDAL 3.5.1?

[doublebyte1]

I am on QGIS 3.34.4-Prizren, which ships with GDAL 3.8.4. In theory, the support for geoparquet should be there from GDAL 3.5, but I am still unable to read geoparquet files.

Any updates or pointers to make it work, would be really appreciated!

Distributor ID: Ubuntu Description: Ubuntu 22.04.4 LTS Release: 22.04 Codename: jammy

[agiudiceandrea]

@doublebyte1 , please provide the full QGIS version info: "In the QGIS Help menu -> About, click in the table, Ctrl+A and then Ctrl+C. Finally paste here. Do not make a screenshot."

[doublebyte1]

@agiudiceandrea

<style type="text/css"> p, li { white-space: pre-wrap; } </style>

QGIS version	3.34.4-Prizren	QGIS code branch	Release 3.34
Qt version	5.15.3
Python version	3.10.12
GDAL/OGR version	3.8.4
PROJ version	9.3.1
EPSG Registry database version	v10.098 (2023-11-24)
GEOS version	3.12.1-CAPI-1.18.1
SQLite version	3.37.2
PostgreSQL client version	14.11 (Ubuntu 14.11-0ubuntu0.22.04.1)
SpatiaLite version	5.1.0
QWT version	6.1.4
QScintilla2 version	2.11.6
OS version	Ubuntu 22.04.4 LTS
Active Python plugins
SensorThingsAPI	1.1.12
quick_map_services	0.19.34
db_manager	0.1.20
MetaSearch	0.3.6
processing	2.12.99
grassprovider	2.12.99

[agiudiceandrea]

@doublebyte1 , how have you installed QGIS on your system? If using Flatpak, then the issue is due to an issue in the Flatpak build of the GDAL library flathub/org.qgis.qgis#272

[jef-n]

arrow is not yet in debian (and ubuntu), see https://bugs.debian.org/970021 (and https://lists.debian.org/debian-gis/2024/03/msg00002.html ff)

[agiudiceandrea]

arrow is not yet in debian (and ubuntu)

@jef-n, Ubuntu 22.04 ships GDAL 3.4.1 according to https://packages.ubuntu.com/search?keywords=gdal, so it looks like @doublebyte1 isn't using GDAL privided by his Ubuntu 22.04 system.

[jef-n]

@jef-n, Ubuntu 22.04 ships GDAL 3.4.1 according to https://packages.ubuntu.com/search?keywords=gdal, so it looks like @doublebyte1 isn't using GDAL privided by his Ubuntu 22.04 system.

My guess would be ubuntugis, which has GDAL 3.8.4, proj 9.3.1 and geos 3.12.1 - and no arrow.

[doublebyte1]

@jef-n @agiudiceandrea https://ppa.launchpadcontent.net/ubuntugis/ubuntugis-unstable/ubuntu jammy InRelease

So I guess my option now would be to build GDAL with arrow, and then build QGIS using that version of GDAL?

Thank you 🙏🏽

[prusswan]

@jef-n, Ubuntu 22.04 ships GDAL 3.4.1 according to https://packages.ubuntu.com/search?keywords=gdal, so it looks like @doublebyte1 isn't using GDAL privided by his Ubuntu 22.04 system.

My guess would be ubuntugis, which has GDAL 3.8.4, proj 9.3.1 and geos 3.12.1 - and no arrow.

yeah this is unfortunate, just found out this is the reason why ogr2ogr (bundled as part of QGIS) does not support parquet as a format, while I had no issue loading parquet in Anaconda (which is a separate Python environment from QGIS). There is also gpq if you just need to convert GeoJSON <-> GeoParquet

@jef-n issue should be open for Debian/Ubuntu. Right now it should be possible to install pyarrow into QGIS's python environment, as a partial solution, it does not help ogr2ogr usage however.

[arkanoid87]

Any viable solution for ubuntu 22.04? GDAL 3.8.4 from official deb repositories does not support parquet files, nor QGIS

[prusswan]

Any viable solution for ubuntu 22.04? GDAL 3.8.4 from official deb repositories does not support parquet files, nor QGIS

See flathub/org.qgis.qgis#272

[prusswan]

So there is a workaround for flatpak build (PR that builds with geoparquet support but yet to be merged).

The other option is to contact the UbuntuGIS team. https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable

I don't fully understand the process, but it appears to me that without a system package for arrow available, the maintainers of the qgis package for Debian/Ubuntu will need to build the arrow dependencies from source (which they may or may not want to do).

@agiudiceandrea can you reopen this?

[agiudiceandrea]

@prusswan, AFAIK the issue is not due to QGIS itself: it's a downstream issue and there is nothing in the QGIS source code that can be changed in order to fix such issue.
AFAIK QGIS distributed by Conda-forge is able to handle GeoParquet layers.

[prusswan]

@prusswan, AFAIK the issue is not due to QGIS itself: it's a downstream issue and there is nothing in the QGIS source code that can be changed in order to fix such issue. AFAIK QGIS distributed by Conda-forge is able to handle GeoParquet layers.

@agiudiceandrea conda-forge is an option but this is not always suitable if only the gdal command line tools is needed (conda is additional overhead). QGIS could do something by informing users there are additional dependencies needed for parquet driver to work (granted, this is useless for most users if they need to build the software). At this point, QGIS installers for Linux are listed on the website. QGIS also bundles gdal tools (without providing options for users to choose/use a different version with the drivers they need), so if gdal tools (as part of the larger QGIS bundle) are "broken" or do not meet user expectations, imho this is still partly a QGIS issue. Anyway I did contact UbuntuGIS team while relying on the unofficial flatpak build in the mean time..

[do-me]

@prusswan, AFAIK the issue is not due to QGIS itself: it's a downstream issue and there is nothing in the QGIS source code that can be changed in order to fix such issue. AFAIK QGIS distributed by Conda-forge is able to handle GeoParquet layers.

For anyone reading this issue, here is the code: qgis/QGIS-Mac-Packager#156 (comment)

[JohnnyBoy123123]

Yeah, really looking forward for this Geoparquet functionality straight out of the box on new Qgis installations. Currently running 3.40.1-Bratislava.