Mapnik.VectorTile.painted() doesn't do what we want it to do

[mapsam]

TLDR

mapnik.VectorTile.painted() doesn't do exactly what we want any longer, which means our check here isn't as strong as expected.

Problem

When generating tiles and copying them (via tilelive), we have a check in the pyramid tiling scheme to ensure we should continue rendering and copying child tiles. This check is here.

There are some situations where we could have a datasource (geojson for this example) that has data in the z0 tile, but has no data in the z1 tile. This means tilelive-bridge checks vtile.painted() and sets the x-tilelive-contains-data header to false. If this is set to false, the rest of the tiling scheme is skipped, resulting in no tiles.

Node Mapnik essentially uses empty() to check for painted tiles via mapnik-vector-tile, which is exactly what we don't want. Right now this z1 tile returns false for empty(), which tells tilelive to stop the copy process and skip the rest of the tiles. We need to know if the source data is there (even if tile features are not) so we can continue to tile the children.

Why

Currently, I'm unable to figure out why a datasource in Node Mapnik would add data at z0 but not at z1.

Solution

A couple solutions come to mind:

• [time intensive] improve the painted(), solid(), and empty() architecture in Mapnik. cc @flippmoke
• [quick fix, potentially at CPU expense] don't check for painted/empty tiles since we already have a relatively solid pyramid built via mapnik-omnivore and spherical mercator. This either means we always set x-tilelive-contains-data to true or remove the if statement in tilelive.

cc @rclark @GretaCB @springmeyer @yhahn @flippmoke @captainbarbosa

[yhahn]

@mapsam thanks for digging into this -- it sounds like something we should look into further and try to fix.

Removing this check in the tiling pyramid scheme is definitely not an option -- it's used quite a bit, especially for very large/diverse datasets.

[mapsam]

This issue continues to come up. An immediate medium-term solution is to change our usage of the pyramid tiling scheme in mapbox-tile-copy to "scanline", which will ensure each tile has no geometry before being tossed, no matter the parent zoom. Some quick benchmarks on the NED 10m_lakes file:

"scanline": 25 seconds average copy time
"pyramid": 24.66 seconds average copy time

mapbox/mapbox-tile-copy#89