Repository navigation
Bump batch-machine to 10.6.0 for tippecanoe 2.79.0 - #711
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
batch-machine 10.6.0 upgrades tippecanoe from 2.31.0 to 2.79.0 (openaddresses/batch-machine#119, released in #120). The 2.31.0
tile-joinallocated memory proportional to its inputs up front and then wrote almost nothing, which is why the addresses and parcels fabric jobs were OOM-killed in the join step on Oct 11-13.On a 10% sample of the real parcels data (16 shards, 4 threads, Alpine/musl) 2.31.0 sat at 14-16GB of anonymous memory and had written a 28KB output after 40 minutes; 2.79.0 finished in 39 minutes with a 2.4GB peak. Details are in the batch-machine PR.
This also changes the tippecanoe used for the per-source previews in batch-machine; on a 300k-point test of that command the output matched 2.31.0 (same tiles per zoom, 486,985 vs 486,991 features).
The full-size fabric run is not tested yet; the next weekly fabric jobs after this deploys are the real test.