Commit Graph

5925 Commits

Author SHA1 Message Date
John Cupitt
c361731693 better support for very long PNG comment names
pngs can have names comments -- improve support for very long comments.
2019-08-02 05:40:40 +01:00
John Cupitt
834acad825 fix << on signed int warnings
<< on a negative number is undefined behaviour in C, and will trigger
fuzzer warnings.
2019-08-02 05:35:18 +01:00
John Cupitt
0f1e278dd9 oop dropped a comment 2019-07-29 10:11:57 +01:00
John Cupitt
332e97272a Merge branch '8.8' 2019-07-29 10:07:29 +01:00
John Cupitt
5e77ab948b fix loop with malformed ppm
skip_line in ppm parse was not testing for EOF, so it could get stuck
with malformed files

thanks Kyle-Kyle

see https://github.com/libvips/libvips/issues/1377
2019-07-29 10:05:55 +01:00
John Cupitt
6a75776272 Merge branch 'loader-minimise-experiment' 2019-07-28 17:23:01 +01:00
John Cupitt
0e5447e537 final cleanup 2019-07-28 17:15:54 +01:00
John Cupitt
5ef14db544 put close-on-last-line back for jpg/tif/png
consider something like:

$ vips arrayjoin "$(echo *.jpg)" x.tif --across 10 --vips-progress

close on minimise won't close until the whole pipeline finishes, so
we'll need to keep every input file open

close on last line will shut down inputs as we are done with them, so we
save a lot of file descriptors

this patch puts close-on-last-line back for jpg/tif/png
2019-07-28 11:34:40 +01:00
John Cupitt
55ae22608f nope, revert
it's not possible to get inisert to trigger minimise reliably, since
threads can be out of order in the middle of the pipeline

we need to add minimise on Y (as before) to get reliable behaviour for
this case
2019-07-27 17:35:34 +01:00
John Cupitt
7f08e8fd28 remove stray flag 2019-07-27 16:13:54 +01:00
John Cupitt
01a82646a1 experiment with minimise in insert
try minimising sub after we've passed it
2019-07-27 13:40:18 +01:00
John Cupitt
67f3cc13ec remove old close-early comment from openexr load
no longer relevant in the world of minimise
2019-07-25 11:34:50 +01:00
John Cupitt
53e0bc6698 pdfload / pdfiumload use minimise to close early
see https://github.com/libvips/libvips/issues/1370#issuecomment-514949292
2019-07-25 11:31:21 +01:00
John Cupitt
9373d63d6e better early close for gif header read 2019-07-24 18:17:54 +01:00
John Cupitt
265cb8dffd close heif early for header read err
we were only closing early if there was an error during pixel read
2019-07-24 18:00:37 +01:00
John Cupitt
e11811a316 fix early close for corrupt gifs
we were not closing early on a read error during gif scan

see https://github.com/libvips/libvips/issues/1370#issuecomment-514172899
2019-07-24 17:43:14 +01:00
John Cupitt
264f41df1c Merge branch '8.8' 2019-07-24 16:07:39 +01:00
John Cupitt
3510e7abcf fetch HEIC metadata from the main image
since thumbs don't have it, see https://github.com/libvips/libvips/issues/1373
2019-07-24 16:06:49 +01:00
John Cupitt
28090d614f oops logic mixup
Kleis pointed out a suprious return in png load minimise.

see https://github.com/libvips/libvips/issues/1370#issuecomment-513706480
2019-07-23 12:06:35 +01:00
John Cupitt
7f3174c3e5 Merge branch 'master' into loader-minimise-experiment 2019-07-23 11:45:24 +01:00
John Cupitt
019ac9a14d Merge branch 'master' of github.com:libvips/libvips 2019-07-22 12:56:47 +01:00
John Cupitt
03bf5a05fa note fuzz in ChangeLog 2019-07-21 18:02:57 +01:00
John Cupitt
e1710edd39 ignore generated fuzzer binaries 2019-07-21 17:52:26 +01:00
John Cupitt
a6fbf1af36
Merge pull request #1264 from omira-sch/add-oss-fuzz
improve fuzz targets and oss-fuzz integration
2019-07-21 17:48:58 +01:00
John Cupitt
4f2f4b4577 add gif, heif, rad, webp early close 2019-07-21 12:29:25 +01:00
John Cupitt
2c654060f9 experiment with a different early-close strategy
We close loaders early in order to save file handles, and on Windows to
make sure that files can be deleted as soon as possible.

Currently loaders do this by watching the Y coordinate of requests and
freeing the fd when the final line of the file is fetched. This is messy
and does not always work, since there are cases when the final line is
not fetched.

Instead, this patch gets loaders to listen for "minimise" on their
output and close on that. This signal is emitted on all upstream images
whenever a threadpool finishes a scan of an image and is usually used to
trim caches after computation.

See https://github.com/libvips/libvips/issues/1370
2019-07-20 16:31:30 +01:00
John Cupitt
67ad303d43 add VIPS_PROFILE env var to enable profiling 2019-07-19 16:04:07 +01:00
John Cupitt
64a6a27326 Merge branch '8.8' 2019-07-19 11:59:38 +01:00
John Cupitt
8cf6a9f9ca don't attempt to save large XMP to JPG
jpeg_write_marker() with some libjpeg versions will throw a fatal
error with large chunks.

To write >64kb XMP it you need to parse the whole XMP object,
pull out the most important fields, code just them into the main
XMP block, then write any remaining XMP objects into a set of
extended XMP markers.

http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/XMPSpecificationPart3.pdf

Instead, just warn and skip large XMP blocks.

see https://github.com/libvips/libvips/issues/1372
2019-07-19 11:37:36 +01:00
John Cupitt
9063b8608e revise README 2019-07-18 08:43:41 +01:00
John Cupitt
db5671ecbb
Merge pull request #1366 from kleisauke/vector-array-int
Support std::vector within the image_get/set_array_int functions
2019-07-16 09:49:24 +01:00
Oscar Mira
4b1c49eb95 suppress memory leak warnings for python tests
Now that travis does not fail, the allow_failure flag for the
clang-compiler job can be removed.
2019-07-16 01:19:37 +02:00
Oscar Mira
50ce012520 vips_guess_prefix memory leak was fixed in 8.8 2019-07-16 01:19:37 +02:00
Kleis Auke Wolthuizen
42bb8f51b9 Support std::vector within the image_get/set_array_int functions 2019-07-15 15:29:19 +02:00
Oscar Mira
1dd0f4f6a7 add travis build with clang and address-sanitizer
Create a new build job to run fuzz regression testing in travis. Also include a
suppression file to ignore known memory leaks.
2019-07-15 14:02:27 +02:00
Oscar Mira
f2745e19c9 add check script for regression testing
Run the fuzz targets over the seed corpus with the standalone driver as part of
the test suite.
2019-07-15 14:02:27 +02:00
Oscar Mira
f3cdf25923 disable PNG CRC checks in fuzzing mode 2019-07-15 14:02:27 +02:00
Oscar Mira
3ea72c286e improve fuzz targets and oss-fuzz integration
It replaces current fuzz setup with 5 new fuzzers integrating them with
top-level Makefile. It now supports multiple fuzzing engines and Google's
oss-fuzz. By default, the fuzzers are linked against an standalone runner to
easily reproduce bugs.
2019-07-15 14:02:27 +02:00
John Cupitt
2da56b918d note about libtiff error handlers 2019-07-13 14:28:50 +01:00
John Cupitt
044f8b74f6 Merge branch 'master' into add-tiff-webp-zstd 2019-07-11 13:50:47 +01:00
John Cupitt
c3e7504835 Merge branch '8.8' 2019-07-10 17:44:39 +01:00
John Cupitt
7f47acab56 tiny improvement 2019-07-10 17:43:17 +01:00
John Cupitt
5e2d66d14b better early shutdown behaviour in shrinkv
read the tail of the input to force early shutdown in seq readers

does reducev need something similar?

see https://github.com/kleisauke/net-vips/issues/12
2019-07-09 16:58:30 +01:00
John Cupitt
52ee3b083f add webp and zstd support to tiffsave
needs some tests still
2019-07-08 10:39:44 +01:00
John Cupitt
063234fde1 set webp loop even if we strip
Make sure that webp loop_count is set, even if strip is enabled.

See https://github.com/libvips/libvips/issues/1339
2019-07-08 08:48:06 +01:00
John Cupitt
bf9a2adf58 Merge branch '8.8' 2019-07-07 19:47:25 +01:00
John Cupitt
bad58598f6 fix "how it works" in docs
I tried loads of ways to fix the rendering of this page, but swapping
the sections headings is the only one that works :(
2019-07-07 17:18:48 +01:00
John Cupitt
37ef7ad231 tiny doc errors 2019-07-07 15:14:16 +01:00
John Cupitt
c7f98ad0a5 Merge branch '8.8' 2019-07-07 13:12:51 +01:00
John Cupitt
e4cff876a8 fix make dist
we'd renamed a file
2019-07-07 11:20:28 +01:00