mbtoolbox

⚠️ This repository is no longer maintained by Lukas Martinelli.

A MBTiles introspection tool for optimizing and verifying MBTiles files.

• Save space by removing redundant subpyramids
• Ensure size of tiles is below a threshold
• Verify that there are no missing or redundant tiles in subpyramids

Install

You need Python 2 or Python 3 installed on your system.

pip install git+https://github.com/lukasmartinelli/mbtoolbox.git

Usage

Remove large tiles from MBTiles

This is primarily thought for purging too large tiles (> 500KB) from a MBTiles file to upload it to Mapbox Studio for creating a Mapbox GL stylesheet.

mboptimize size <mbtiles_file> -s 500000

Verify Size of MBTiles

Check if a file contains any tiles larger than 500KB.

mbverify size <mbtiles_file> -s 500000

You get back a list of all tiles larger than 500KB.

14/8024/12095   506 KByte
14/8025/12095   1.1 Mbyte

Check MBTiles for Redundancy

If you are using tilelive-vector or a tile server like tileserver-php which supports maskLevel you can save a lot of redundant data in your MBTiles file by backfilling missing high zoom levels with data from low zoom levels. This approach is used at osm2vectortiles to decrease the size of the MBTiles downloads.

Check if a file contains any removable redundant subpyramids for a maskLevel of 8.

mboptimize check <mbtiles_file> -z 8

You get back a list of all optimizable subpyramids.

8/125/188   OPTIMIZABLE

Remove subpyramids

Once you know your file contains redundancy you can remove the unnecessary tiles. Removing subpyramids will gain a lot of space in big files since you no longer need to store all the references to the binary image data. If you render vector tiles of the entire wolrd to a file with 70GB this can decrease the size by over 12GB.

mboptimize remove <mbtiles_file> -z 8

Verify Subpyramid contains all Tiles

Given you have a MBTiles file you want to verify that that all tile data for the XYZ subpyramid 8/125/188 down to zoom level 14 is present.

mbverify missing <mbtiles_file> 125 188 -z 8 -Z 14

Verify Subpyramid has no additional Tiles

Given you have a MBTiles file you want to verify that only the exact tile data of the XYZ subpyramid 8/125/188 down to zoom level 14 is present. Any additional data is treated as redundant.

mbverify redundant <mbtiles_file> 125 188 -z 8 -Z 14

Mask Level

tilelive-vector supports the powerful concept of mask level.

To avoid requiring many duplicate or empty vector tiles to be generated at high zoom levels, the backend source can specify a maskLevel. If a vector tile is not initially found at some z > maskLevel, Vector will issue an additional request to the backend using the parent tile of of the request at maskLevel. This allows a lower zoom level to backfill high zoom levels.

Example

If each descendant tile in the entire subpyramid 8/100/101 has the same binary data as 8/100/101 we can remove all descendants. A request to 9/200/202 will then receive the data from 8/100/101 without the need of duplicating the data across multiple zoom levels.