Too Much Recursion

[yourealwaysbe]

Hi,

I've been playing with the PBF parser for reading OSM data into a (experimental) web app. However, reading large files results in a "too much recursion error"

too much recursion osm-read-pbf.js:2269

This happens, for example, with the South Yorkshire PBF available here.

Is it possible to fix this, or is it an intrinsic problem with trying to read PBF in Javascript :)

[nrenner]

Can reproduce the error in Firefox. Chromium works with South Yorkshire (14,4 MB), but hangs with a larger file (older Berlin, 29,4 MB).

I'm having some troubles debugging this in Firefox, and updating dependencies needs some work. Will look further into this.

Don't know what your use case is, but for rendering on a map, splitting the data into vector tiles might help.

[yourealwaysbe]

Thanks for looking into it. This was just for an experiment which i've concluded successful enough (but continue to play with), so there's no rush on a fix :)

[nrenner]

The reason for this error is trying to execute Node.js-style callback code synchronously in the Browser. The recursive loop in visitNextBlock then leads the call stack to pile up.

As a fix I added an asynchronous call in the arrayBufferReader.

In addition, there is now a new reader for the HTML5 File API that uses async, sliced reading, see README and example/file.html on how to use it. Furthermore, you can already load a PBF file yourself with FileReader.readAsArrayBuffer() and pass the result as buffer option. Both ways of using the File API have the advantage that there is no restriction on accessing local files as there is when using XHR (implicitly by passing a filePath).

This was just for an experiment ...

Nice idea to tile into indexedDB! Would be great if this could be used with an existing renderer/map lib. My approach requires the Mapsplit commandline tool to generate PBF tiles.

[yourealwaysbe]

Cool, many thanks for this. I'll check it out :)