check for number of scanlines read from jpeg to prevent integer underflow

[the-shank]

jpeg_read_scanlines() can read less lines that requested in certain conditions (ref). When it returns without reading any rows, the output_scanline can be 0 (refer here). When this happens, it can lead to a integer underflow here.

This patch adds check for the return value of jpeg_read_scanlines() and in case the return value is 0, it just skips that loop and continues.

[Talinx]

I don't think that this problem can occur here. According to the libjpeg-turbo docs jpeg_read_scanlines() is guaranteed to read at least one scanline unless a suspending data source is used (which jp2a does not). The doc also implies that a buffer size of 1 (as used here) will always be fully filled with only one call to jpeg_read_scanlines() while larger buffer sizes necessitate checks/multiple calls.

Do you have an example that does not work currently but would with your code change?

[the-shank]

We found this as a part of our research study on Library API Misuse (handling of return values from library calls, etc.). We don't have a specific example available.

As stated by you, jp2a does not use a suspending data source, and so this is infeasible as of now. Would it be ok to add an assert to check the return value of jpeg_read_scanlines() so that if it ever changes, it could be flagged and attended to promptly.