Commit Graph

5938 Commits

Author SHA1 Message Date
John Cupitt
f472df4444 block /0 in unpremultiply 2019-08-04 14:01:42 +01:00
John Cupitt
64d42be091 better invert of gray signed int tiffs 2019-08-04 03:55:39 +01:00
John Cupitt
2841504fa0 fix a possible int overflow 2019-08-03 21:14:19 +01:00
John Cupitt
2a094995a3 fix warning in rect hash 2019-08-03 04:03:43 +01:00
John Cupitt
3c8a9815ba fix int overflow in shrinkv 2019-08-03 03:31:54 +01:00
John Cupitt
cfe0857ae1
Merge pull request #1379 from omira-sch/undefined-sanitizer
Add undefined sanitizer
2019-08-02 20:07:19 +01:00
Oscar Mira
78c0014708 add UBSAN sanitizer 2019-08-02 19:46:42 +02:00
Oscar Mira
cf655c6fbe fix fuzzing specific build macro 2019-08-02 19:46:42 +02:00
John Cupitt
4a05c67345
Merge pull request #1381 from omira-sch/fuzzing-large-images
skip large images in the fuzzers not in the lib
2019-08-02 18:41:46 +01:00
Oscar Mira
45de60e571 skip large images in the fuzzers not in the lib
This reverts commit 0accdf858b.
2019-08-02 17:56:33 +02:00
John Cupitt
4ce745dbee nudge travis 2019-08-02 11:05:49 +01:00
John Cupitt
0accdf858b band large image during fuzzing
Large images cause a lot of uninteresting OOMs and timeouts during
fuzzing. Ban them.
2019-08-02 10:53:32 +01:00
John Cupitt
0a225314b9 fix int overflow in flatten 2019-08-02 10:17:57 +01:00
John Cupitt
c361731693 better support for very long PNG comment names
pngs can have names comments -- improve support for very long comments.
2019-08-02 05:40:40 +01:00
John Cupitt
834acad825 fix << on signed int warnings
<< on a negative number is undefined behaviour in C, and will trigger
fuzzer warnings.
2019-08-02 05:35:18 +01:00
John Cupitt
0f1e278dd9 oop dropped a comment 2019-07-29 10:11:57 +01:00
John Cupitt
332e97272a Merge branch '8.8' 2019-07-29 10:07:29 +01:00
John Cupitt
5e77ab948b fix loop with malformed ppm
skip_line in ppm parse was not testing for EOF, so it could get stuck
with malformed files

thanks Kyle-Kyle

see https://github.com/libvips/libvips/issues/1377
2019-07-29 10:05:55 +01:00
John Cupitt
6a75776272 Merge branch 'loader-minimise-experiment' 2019-07-28 17:23:01 +01:00
John Cupitt
0e5447e537 final cleanup 2019-07-28 17:15:54 +01:00
John Cupitt
5ef14db544 put close-on-last-line back for jpg/tif/png
consider something like:

$ vips arrayjoin "$(echo *.jpg)" x.tif --across 10 --vips-progress

close on minimise won't close until the whole pipeline finishes, so
we'll need to keep every input file open

close on last line will shut down inputs as we are done with them, so we
save a lot of file descriptors

this patch puts close-on-last-line back for jpg/tif/png
2019-07-28 11:34:40 +01:00
John Cupitt
55ae22608f nope, revert
it's not possible to get inisert to trigger minimise reliably, since
threads can be out of order in the middle of the pipeline

we need to add minimise on Y (as before) to get reliable behaviour for
this case
2019-07-27 17:35:34 +01:00
John Cupitt
7f08e8fd28 remove stray flag 2019-07-27 16:13:54 +01:00
John Cupitt
01a82646a1 experiment with minimise in insert
try minimising sub after we've passed it
2019-07-27 13:40:18 +01:00
John Cupitt
67f3cc13ec remove old close-early comment from openexr load
no longer relevant in the world of minimise
2019-07-25 11:34:50 +01:00
John Cupitt
53e0bc6698 pdfload / pdfiumload use minimise to close early
see https://github.com/libvips/libvips/issues/1370#issuecomment-514949292
2019-07-25 11:31:21 +01:00
John Cupitt
9373d63d6e better early close for gif header read 2019-07-24 18:17:54 +01:00
John Cupitt
265cb8dffd close heif early for header read err
we were only closing early if there was an error during pixel read
2019-07-24 18:00:37 +01:00
John Cupitt
e11811a316 fix early close for corrupt gifs
we were not closing early on a read error during gif scan

see https://github.com/libvips/libvips/issues/1370#issuecomment-514172899
2019-07-24 17:43:14 +01:00
John Cupitt
264f41df1c Merge branch '8.8' 2019-07-24 16:07:39 +01:00
John Cupitt
3510e7abcf fetch HEIC metadata from the main image
since thumbs don't have it, see https://github.com/libvips/libvips/issues/1373
2019-07-24 16:06:49 +01:00
John Cupitt
28090d614f oops logic mixup
Kleis pointed out a suprious return in png load minimise.

see https://github.com/libvips/libvips/issues/1370#issuecomment-513706480
2019-07-23 12:06:35 +01:00
John Cupitt
7f3174c3e5 Merge branch 'master' into loader-minimise-experiment 2019-07-23 11:45:24 +01:00
John Cupitt
019ac9a14d Merge branch 'master' of github.com:libvips/libvips 2019-07-22 12:56:47 +01:00
John Cupitt
03bf5a05fa note fuzz in ChangeLog 2019-07-21 18:02:57 +01:00
John Cupitt
e1710edd39 ignore generated fuzzer binaries 2019-07-21 17:52:26 +01:00
John Cupitt
a6fbf1af36
Merge pull request #1264 from omira-sch/add-oss-fuzz
improve fuzz targets and oss-fuzz integration
2019-07-21 17:48:58 +01:00
John Cupitt
4f2f4b4577 add gif, heif, rad, webp early close 2019-07-21 12:29:25 +01:00
John Cupitt
2c654060f9 experiment with a different early-close strategy
We close loaders early in order to save file handles, and on Windows to
make sure that files can be deleted as soon as possible.

Currently loaders do this by watching the Y coordinate of requests and
freeing the fd when the final line of the file is fetched. This is messy
and does not always work, since there are cases when the final line is
not fetched.

Instead, this patch gets loaders to listen for "minimise" on their
output and close on that. This signal is emitted on all upstream images
whenever a threadpool finishes a scan of an image and is usually used to
trim caches after computation.

See https://github.com/libvips/libvips/issues/1370
2019-07-20 16:31:30 +01:00
John Cupitt
67ad303d43 add VIPS_PROFILE env var to enable profiling 2019-07-19 16:04:07 +01:00
John Cupitt
64a6a27326 Merge branch '8.8' 2019-07-19 11:59:38 +01:00
John Cupitt
8cf6a9f9ca don't attempt to save large XMP to JPG
jpeg_write_marker() with some libjpeg versions will throw a fatal
error with large chunks.

To write >64kb XMP it you need to parse the whole XMP object,
pull out the most important fields, code just them into the main
XMP block, then write any remaining XMP objects into a set of
extended XMP markers.

http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/XMPSpecificationPart3.pdf

Instead, just warn and skip large XMP blocks.

see https://github.com/libvips/libvips/issues/1372
2019-07-19 11:37:36 +01:00
John Cupitt
9063b8608e revise README 2019-07-18 08:43:41 +01:00
John Cupitt
db5671ecbb
Merge pull request #1366 from kleisauke/vector-array-int
Support std::vector within the image_get/set_array_int functions
2019-07-16 09:49:24 +01:00
Oscar Mira
4b1c49eb95 suppress memory leak warnings for python tests
Now that travis does not fail, the allow_failure flag for the
clang-compiler job can be removed.
2019-07-16 01:19:37 +02:00
Oscar Mira
50ce012520 vips_guess_prefix memory leak was fixed in 8.8 2019-07-16 01:19:37 +02:00
Kleis Auke Wolthuizen
42bb8f51b9 Support std::vector within the image_get/set_array_int functions 2019-07-15 15:29:19 +02:00
Oscar Mira
1dd0f4f6a7 add travis build with clang and address-sanitizer
Create a new build job to run fuzz regression testing in travis. Also include a
suppression file to ignore known memory leaks.
2019-07-15 14:02:27 +02:00
Oscar Mira
f2745e19c9 add check script for regression testing
Run the fuzz targets over the seed corpus with the standalone driver as part of
the test suite.
2019-07-15 14:02:27 +02:00
Oscar Mira
f3cdf25923 disable PNG CRC checks in fuzzing mode 2019-07-15 14:02:27 +02:00