Support PMTiles in Editor

[ohrie]

PMTiles from Protomaps are increasingly popular and a lot of other services are supporting it now. It has some major advantages in comparison to MBTiles. Styling the PMTiles is, however not possible yet, using the data set directly.

MapLibre GL JS supports PMTiles. Therefore a switch to Maplibre GL JS (see #781) in addition with the support of pmtiles URLs link (like this) would be super cool, so we can use PMTiles in Maputnik!

[bogind]

MapLibre supports this feature with an external plugin that relies on the addprotocol method.

I think that adding this functionality might confuse users who do not know this,
and that may not know that the output of their style will only work under certain tile deployment and development conditions (for example it will likely not work in GeoServer or QGIS)

In my opinion, unless PMTiles are integrated as an official source in the MapLibre Style Sepcification this will likely cause confusion, even though it might be highly beneficial for certain users.

[lseelenbinder]

I agree with @bogind, I think it makes more sense to support this as a XYZ URL in Maputnik. It is very difficult to support plugins at the editor level, and it's not a significant workflow difference if you have to substitute one URL.

If and when it's integrated, then we can reconsider.

[zstadler]

Now that MapLibre is the rendering engine for Maputnik #812, it would make sense to me that the PMTiles plugin will be added by default to Maputnik, and enable the use of pmtiles:... sources.

[bogind]

Now that MapLibre is the rendering engine for Maputnik #812, it would make sense to me that the PMTiles plugin will be added by default to Maputnik, and enable the use of pmtiles:... sources.

I don't think the issue was with the rendering engine, but with the style specification itself.
PMTiles is still an external plugin for MapLibre.

In my opinion (currently as a user, not a contributor), this should not be added, unless with strict instructions and warnings about compatibility. This can cause credibility issues with non-proficient users who would try to use the exported pmtiles:... styles in rendering engines that do not support them.

[ThomasG77]

To bypass the issue in practice as I can understand the "why" of the policy, host locally Maputnik or serve pmtiles as x/y/z.mvt

Solution to use locally pmtiles with Maputnik

git clone https://github.com/maplibre/maputnik.git
cd maputnik
npm i
npm i pmtiles

echo -e "import { Protocol } from 'pmtiles';\n$(cat src/components/MapMaplibreGl.tsx)" >| src/components/MapMaplibreGl.tsx

sed -i 's#const map = new MapLibreGl.Map(mapOpts);#const map = new MapLibreGl.Map(mapOpts);\nlet protocol = new Protocol();\nMapLibreGl.addProtocol("pmtiles",protocol.tile);#g' src/components/MapMaplibreGl.tsx

mv dist maputnik
npm i -g http-server
http-server

Then open http://127.0.0.1:8080/maputnik/ and you can try using maputnik with pmtiles as source

Alternative if you have a public server where you can install stuff with your pmtiles

No need to change anything on Maputnik public side. Use simply https://maputnik.github.io/editor/ and consume {z}/{x}/{y}.mvt URLs

# If another OS, see https://github.com/protomaps/go-pmtiles/releases
wget https://github.com/protomaps/go-pmtiles/releases/download/v1.19.2/go-pmtiles_1.19.2_Linux_x86_64.tar.gz
tar -xvzf go-pmtiles_1.19.2_Linux_x86_64.tar.gz
chmod +x pmtiles
# Sample file. Use you own here
wget https://r2-public.protomaps.com/protomaps-sample-datasets/cb_2018_us_zcta510_500k.pmtiles
# See also https://github.com/protomaps/go-pmtiles?tab=readme-ov-file#serving-zxy-tiles for more
./pmtiles serve --cors=* --port=8080 cb_2018_us_zcta510_500k.pmtiles

Change now in the Maplibre JSON style the block starting with "pmtiles://..." to http://yourhost.domain:8080/cb_2018_us_zcta510_500k.pmtiles/{z}/{x}/{y}.mvt, do your styling then you can switch back to "pmtiles://..." when in production.

[HarelM]

I believe MBtiles and PMtiles are basically the same, you can't use them as is in maplibre, therefore you either need to serve them, even from local machine using a tile server, or host them somewhere.
There are great instructions that I have followed in order to serve a pmtiles file from cloudflare.
You can use martin for local development, or tileserver-gl.
All of those solutions will allow you to create a style that can easily work on any maplibre client.
I'm not sure adding the pmtiles plugin and supporting its protocol will benefit the users.
I think that a readme with a section about hosting local tile files such as PMtiles and MBtiles can solve most users' problems.

[diehl]

@HarelM Would you mind sharing the instructions for serving PMTiles from CloudFlare? I stumbled on this post while searching for information on designing a custom basemap style with PMTiles as the underlying data source. Sounds like you just provided me the pathway to a solution!

[HarelM]

Pm tiles has excellent instructions:
https://docs.protomaps.com/pmtiles/cloud-storage

[diehl]

Thank you @HarelM!

[josxha]

In the meantime Protomaps has a hosted version of Maputnik with support for PMTiles.
https://editor.protomaps.com/
This probably doesn't really solves this issue since it's a forked of Maputnik but should be helpful for everyone who wants to edit a protomaps compatible style. Only thing to care about is that the version of Maputnik hosted on Protomaps is slightly outdated.

[HarelM]

According to what I've seen there, you still need to host the PMTiles file locally.
For hosting a local file you can use a small tile server like Martin, and then you can use maputnik as is...

[bdon]

According to what I've seen there, you still need to host the PMTiles file locally.

http://editor.protomaps.com works with remote PMTiles archives, there's no limitation on localhost or local files etc. I have updated the fork with the most recent changes from maputnik/main.

(edit: since Maputnik supports this now, editor.protomaps.com redirects to http://maplibre.org/maputnik/)

[prusswan]

According to what I've seen there, you still need to host the PMTiles file locally.

http://editor.protomaps.com works with remote PMTiles archives, there's no limitation on localhost or local files etc. I have updated the fork with the most recent changes from maputnik/main.

I tried adding the inspect behavior used in the editor to the tiles viewer (pmtiles.io) just to see how it works with local files. Seems like something is still missing (internal behavior like map.style.sourceCaches[sourceId]._source.vectorLayerIds?). Maybe it will be easier to add drag n drop support to the editor instead (should work well for both small pmtiles packages and style.json)

[bdon]

Yeah this is a bug right now internally in the PMTiles MapLibre adapter with how it deals with fetching TileJSON for local file archives.

For Maputnik there is no UI element for drag-and-drop, and the user experience may not be ideal since it requires user interaction to open a local file. For the hosted maputnik you could communicate with localhost using a self-signed SSL certificate.

[prusswan]

Yeah this is a bug right now internally in the PMTiles MapLibre adapter with how it deals with fetching TileJSON for local file archives.

Turns out that you just need a way to pass metadata.vector_layers to map.style.sourceCaches["source"]._source.vectorLayerIds. I just store it under style["metadata"] but this hack is good as any.

The internal API that maplibre-gl-inspect is relying on: https://github.com/maplibre/maplibre-gl-inspect/blob/5e234935bba74e2870dae0df57098c48bc3be9ef/lib/MaplibreInspect.ts#L213

For Maputnik there is no UI element for drag-and-drop, and the user experience may not be ideal since it requires user interaction to open a local file. For the hosted maputnik you could communicate with localhost using a self-signed SSL certificate.

I guess you mean it is not easy to find a good spot for the UI element, not that it can't be done? It would be nice to be able to drop a local file onto https://editor.protomaps.com with the style pre-populated according to the metadata, probably the minimal interaction needed for those who prefer this option.

Update: https://github.com/prusswan/maputnik/tree/support-dropped-file (grafted from pmtiles.io)