John Cupitt
4a05c67345
Merge pull request #1381 from omira-sch/fuzzing-large-images
...
skip large images in the fuzzers not in the lib
2019-08-02 18:41:46 +01:00
Oscar Mira
45de60e571
skip large images in the fuzzers not in the lib
...
This reverts commit 0accdf858b
.
2019-08-02 17:56:33 +02:00
John Cupitt
4ce745dbee
nudge travis
2019-08-02 11:05:49 +01:00
John Cupitt
0accdf858b
band large image during fuzzing
...
Large images cause a lot of uninteresting OOMs and timeouts during
fuzzing. Ban them.
2019-08-02 10:53:32 +01:00
John Cupitt
0a225314b9
fix int overflow in flatten
2019-08-02 10:17:57 +01:00
John Cupitt
f0d1a5d82b
note PNG comment fix
2019-08-02 10:11:16 +01:00
John Cupitt
4ea50fe6f0
better support for very long PNG comment names
...
pngs can have names comments -- improve support for very long comments.
2019-08-02 10:08:45 +01:00
John Cupitt
c361731693
better support for very long PNG comment names
...
pngs can have names comments -- improve support for very long comments.
2019-08-02 05:40:40 +01:00
John Cupitt
834acad825
fix << on signed int warnings
...
<< on a negative number is undefined behaviour in C, and will trigger
fuzzer warnings.
2019-08-02 05:35:18 +01:00
John Cupitt
0a51ca96c9
mostly working
...
still fails to exit cleanly
2019-08-01 20:57:46 +01:00
John Cupitt
0f1e278dd9
oop dropped a comment
2019-07-29 10:11:57 +01:00
John Cupitt
332e97272a
Merge branch '8.8'
2019-07-29 10:07:29 +01:00
John Cupitt
5e77ab948b
fix loop with malformed ppm
...
skip_line in ppm parse was not testing for EOF, so it could get stuck
with malformed files
thanks Kyle-Kyle
see https://github.com/libvips/libvips/issues/1377
2019-07-29 10:05:55 +01:00
John Cupitt
6078ba744c
added more
2019-07-29 09:56:22 +01:00
John Cupitt
1220cc9667
start mapimage
2019-07-28 19:40:37 +01:00
John Cupitt
6a75776272
Merge branch 'loader-minimise-experiment'
2019-07-28 17:23:01 +01:00
John Cupitt
0e5447e537
final cleanup
2019-07-28 17:15:54 +01:00
John Cupitt
5ef14db544
put close-on-last-line back for jpg/tif/png
...
consider something like:
$ vips arrayjoin "$(echo *.jpg)" x.tif --across 10 --vips-progress
close on minimise won't close until the whole pipeline finishes, so
we'll need to keep every input file open
close on last line will shut down inputs as we are done with them, so we
save a lot of file descriptors
this patch puts close-on-last-line back for jpg/tif/png
2019-07-28 11:34:40 +01:00
John Cupitt
55ae22608f
nope, revert
...
it's not possible to get inisert to trigger minimise reliably, since
threads can be out of order in the middle of the pipeline
we need to add minimise on Y (as before) to get reliable behaviour for
this case
2019-07-27 17:35:34 +01:00
John Cupitt
7f08e8fd28
remove stray flag
2019-07-27 16:13:54 +01:00
John Cupitt
01a82646a1
experiment with minimise in insert
...
try minimising sub after we've passed it
2019-07-27 13:40:18 +01:00
John Cupitt
67f3cc13ec
remove old close-early comment from openexr load
...
no longer relevant in the world of minimise
2019-07-25 11:34:50 +01:00
John Cupitt
53e0bc6698
pdfload / pdfiumload use minimise to close early
...
see https://github.com/libvips/libvips/issues/1370#issuecomment-514949292
2019-07-25 11:31:21 +01:00
John Cupitt
9373d63d6e
better early close for gif header read
2019-07-24 18:17:54 +01:00
John Cupitt
265cb8dffd
close heif early for header read err
...
we were only closing early if there was an error during pixel read
2019-07-24 18:00:37 +01:00
John Cupitt
e11811a316
fix early close for corrupt gifs
...
we were not closing early on a read error during gif scan
see https://github.com/libvips/libvips/issues/1370#issuecomment-514172899
2019-07-24 17:43:14 +01:00
John Cupitt
264f41df1c
Merge branch '8.8'
2019-07-24 16:07:39 +01:00
John Cupitt
3510e7abcf
fetch HEIC metadata from the main image
...
since thumbs don't have it, see https://github.com/libvips/libvips/issues/1373
2019-07-24 16:06:49 +01:00
John Cupitt
28090d614f
oops logic mixup
...
Kleis pointed out a suprious return in png load minimise.
see https://github.com/libvips/libvips/issues/1370#issuecomment-513706480
2019-07-23 12:06:35 +01:00
John Cupitt
7f3174c3e5
Merge branch 'master' into loader-minimise-experiment
2019-07-23 11:45:24 +01:00
John Cupitt
019ac9a14d
Merge branch 'master' of github.com:libvips/libvips
2019-07-22 12:56:47 +01:00
John Cupitt
03bf5a05fa
note fuzz in ChangeLog
2019-07-21 18:02:57 +01:00
John Cupitt
e1710edd39
ignore generated fuzzer binaries
2019-07-21 17:52:26 +01:00
John Cupitt
a6fbf1af36
Merge pull request #1264 from omira-sch/add-oss-fuzz
...
improve fuzz targets and oss-fuzz integration
2019-07-21 17:48:58 +01:00
John Cupitt
4f2f4b4577
add gif, heif, rad, webp early close
2019-07-21 12:29:25 +01:00
John Cupitt
2c654060f9
experiment with a different early-close strategy
...
We close loaders early in order to save file handles, and on Windows to
make sure that files can be deleted as soon as possible.
Currently loaders do this by watching the Y coordinate of requests and
freeing the fd when the final line of the file is fetched. This is messy
and does not always work, since there are cases when the final line is
not fetched.
Instead, this patch gets loaders to listen for "minimise" on their
output and close on that. This signal is emitted on all upstream images
whenever a threadpool finishes a scan of an image and is usually used to
trim caches after computation.
See https://github.com/libvips/libvips/issues/1370
2019-07-20 16:31:30 +01:00
John Cupitt
67ad303d43
add VIPS_PROFILE env var to enable profiling
2019-07-19 16:04:07 +01:00
John Cupitt
64a6a27326
Merge branch '8.8'
2019-07-19 11:59:38 +01:00
John Cupitt
8cf6a9f9ca
don't attempt to save large XMP to JPG
...
jpeg_write_marker() with some libjpeg versions will throw a fatal
error with large chunks.
To write >64kb XMP it you need to parse the whole XMP object,
pull out the most important fields, code just them into the main
XMP block, then write any remaining XMP objects into a set of
extended XMP markers.
http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/XMPSpecificationPart3.pdf
Instead, just warn and skip large XMP blocks.
see https://github.com/libvips/libvips/issues/1372
2019-07-19 11:37:36 +01:00
John Cupitt
9063b8608e
revise README
2019-07-18 08:43:41 +01:00
John Cupitt
db5671ecbb
Merge pull request #1366 from kleisauke/vector-array-int
...
Support std::vector within the image_get/set_array_int functions
2019-07-16 09:49:24 +01:00
Oscar Mira
4b1c49eb95
suppress memory leak warnings for python tests
...
Now that travis does not fail, the allow_failure flag for the
clang-compiler job can be removed.
2019-07-16 01:19:37 +02:00
Oscar Mira
50ce012520
vips_guess_prefix memory leak was fixed in 8.8
2019-07-16 01:19:37 +02:00
Kleis Auke Wolthuizen
42bb8f51b9
Support std::vector within the image_get/set_array_int functions
2019-07-15 15:29:19 +02:00
Oscar Mira
1dd0f4f6a7
add travis build with clang and address-sanitizer
...
Create a new build job to run fuzz regression testing in travis. Also include a
suppression file to ignore known memory leaks.
2019-07-15 14:02:27 +02:00
Oscar Mira
f2745e19c9
add check script for regression testing
...
Run the fuzz targets over the seed corpus with the standalone driver as part of
the test suite.
2019-07-15 14:02:27 +02:00
Oscar Mira
f3cdf25923
disable PNG CRC checks in fuzzing mode
2019-07-15 14:02:27 +02:00
Oscar Mira
3ea72c286e
improve fuzz targets and oss-fuzz integration
...
It replaces current fuzz setup with 5 new fuzzers integrating them with
top-level Makefile. It now supports multiple fuzzing engines and Google's
oss-fuzz. By default, the fuzzers are linked against an standalone runner to
easily reproduce bugs.
2019-07-15 14:02:27 +02:00
John Cupitt
2da56b918d
note about libtiff error handlers
2019-07-13 14:28:50 +01:00
John Cupitt
044f8b74f6
Merge branch 'master' into add-tiff-webp-zstd
2019-07-11 13:50:47 +01:00