* move tiff decompress outside lock
Most time in TIFF read is spent in decompression. If we move this
outside the lock, we can get a useful speedup.
This commit adds the machinery to move the lock to before decompress, so
jp2k decompression is now threaded.
Before:
```
$ vips copy wtc.jpg x.tif[tile,compression=jp2k]
$ time vips avg x.tif
117.249845
real 0m15.085s
user 0m16.155s
sys 0m0.109s
```
After:
```
$ time vips avg x.tif
117.249845
real 0m1.207s
user 0m18.384s
sys 0m0.369s
```
* start moving jpg decode outside the lock
* move jpeg decompress outside the lock
seems to work
* add some more tile size checks
double-check jpeg tile size before decode
* fix tiffload demand hinting
We were not setting the hint correctly in header load, and we were not
hinting smalltile for tiled TIFFs.
SOme combinations of high bitdepth images with low bitdepth PNG save could
produce incorrect images, for example saving a 16-bit fourier image as 1-bit.
We were avoiding /0 by testing for alpha==0, however, this will still
allow very small values of alpha to generate +/- Inf. Instead, check for
abs(alpha)<epsilon.
Fixes some artifacts after unpremul.
We were minimising at the end of threadpool_run, but this was being
called for each sinkscreen render pass, so we were throwing away the
display cache.
Instead, minimise after sink, sink_memory and sink_disc.
To disable DoS limits for JPEG loading. Adding API on a stable branch is
bad, but this fixes a regression, so I think it's necessary,
unfortunately.
See https://github.com/libvips/libvips/issues/2973
* flag to save bmp with bitdepth 1
* magicksave: add bitdepth option
* update comments
* update documentation
* revised based on review comments
Co-authored-by: Thomas Barton <thomasryanbarton@gmail.com>
* save GIF palette as metadata
... if there are no local colour tables.
See https://github.com/libvips/libvips/issues/2576
* cgifsave: reuse global palette, if possible
* add reuse_palette parameter
* add reoptimise parameter and reuse palette by default
* attach global palette even if local palettes are present
* add check for presence of use-lct
* Revert "add check for presence of use-lct"
This reverts commit cd0f14e45e8bed8f108ee9666e2569c6355f17eb.
* Revert "attach global palette even if local palettes are present"
This reverts commit 4085b9e14b73c8990e9653ccc6f90477c2a43032.
* move global palette quantization to cgif_build
* rename member variable gct
* update comments
* improve error handling
* update documentation
Co-authored-by: John Cupitt <jcupitt@gmail.com>
* start adding dzsave to a target
made a thing for gsf to write to a target
* fix stray tabs in dzsave
* fix dzsave write to "."
Early versions of libgsf did not support writing to ".", so we had an ugly
workaround, but this should now be OK.
Fixing this ought to make write to target simple
* dzsave_target compiles
no idea if it works though
* seems to work now
creates stray 0 length files though, very odd
* fix stray files from dzsave
next: save to buffer is returning null
* fix buffer flush in dzsave
all tests pass!
* update changelog
* compiles, but untested
* works, but libtiff needs seek for write
next: add seek methods to target
* add target seek and read
seem to work
next: disc temps for disc output
* add libnsgif COPYING
oops, we were missing the COPYING file
see https://github.com/libvips/libvips/issues/2800
thanks mika-fischer
* tiffsave uses a disc temp if it can
* revise temp target rules
only make a disc temp if we are writing to a filesystem target
* add new target methods to targetcustom
since finish did not return an error code
also make sure we don't call target_end from inside _dispose, since that
can't signal error either
see https://github.com/libvips/libvips/issues/2801
* quick proposal
warn on startup if untrusted operations might run
use vips_block_untrusted_set() to block untrusted operations, set an env
var or make a file to stop the warning
* mark fits, nifti and svg as untrusted
* remove the annoying "untrusted" warning message
better to warn on the download page
leave vips_block_untrusted_set() since it's obviously useful
* separate UNTRUSTED and BLOCKED
* typos
* add VIPS_BLOCK_UNTRUSTED env var
* move BLOCK_UNTRUSTED after plugin load
obviously, ooops
* add a test, disable *magick
although *magick is fuzzed, it's probably safer to disable it in
untrusted environments
* mark some more operations as untrusted
* quick hack
won't even compile
* works!
at a basic level, anyway
* start wiring up metadata
* metadata done
* add spng interlaced save
* start adding index support
* fix palette save
* Update libvips/foreign/spngsave.c
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
* start adding low bit depth write
* add a warning for low bitdepth write
* small fixes
* add 1/2/4 bit write
adapted from the vips2tiff bit packer
* small polish
* note spngsave in changelog
* Update libvips/foreign/spngsave.c
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
* Update libvips/foreign/spngsave.c
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
* revise 8 and 16-bit PNG write rules
* cleanup
* remove unnecessary palette filter optimisation
since libspng already does this
* add xres/yres rounding
thanks klies
he commit message for your changes. Lines starting
* look for spng.pc and libspng.pc
It can be called either, frustratingly.
* fix PACKAGES_USED for spng .pc changes
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
This patch makes "insert" issue minimise signals for inputs in sequential
mode. This can drop memory use in some programs, for example:
```python
import sys
import random
import pyvips
image = pyvips.Image.black(20000, 20000)
for filename in sys.argv[2:]:
tile = pyvips.Image.new_from_file(filename, access='sequential')
x = random.randint(0, image.width - tile.width)
y = random.randint(0, image.height - tile.height)
image = image.insert(tile, x, y)
image.write_to_file(sys.argv[1])
```
Before this patch:
```
$ for i in {0..1000}; do cp ~/pics/k2.jpg $i.jpg; done
$ /usr/bin/time -f %M:%e ../manyjoin.py ../x.jpg *.jpg
5456256:4.34
```
With this patch:
```
$ /usr/bin/time -f %M:%e ../manyjoin.py ../x.jpg *.jpg
2475324:3.38
```
tell buffer and target savers the file format
Currently, buffer and target savers are not told the format they should
write.
This is usually OK (the JPEG saver already knows it should write JPEG),
but some savers can write several formats, and these currently need an
extra parameter.
For example:
```ruby
buf = x.write_to_buffer ".bmp", format: "bmp"
```
The first ".bmp" gets libvips to pick magicksave, the second
`format:` param is necessary to tell magicksave to write BMP.
This patch adds stub subclasses so that the savers know the exact format. It also improves PPM save.
* Add jpeg restart_interval option.
This allows saving a jpeg with MCU restarts.
* Fix code style. Add description of restart_interval.
* Add a basic test based on output length.
* Update main change log.
We used to assume (in several places) that any source with a filename was
seekable. This patch adds a is_file test, and makes all the loaders use it.
see https://github.com/libvips/libvips/issues/2467
* fic gtk-doc typenames in cgif
* fix flatten clipping
flatten could produce out of range values if max_alpha was less than
the limit of the numeric range of the format
https://github.com/libvips/libvips/issues/2431
By making the sequential line cache non-persistent, and only minimising
when the read point is well past the image.
On large arrayjoin operations, this saves many GB of memory.
See https://github.com/kleisauke/net-vips/issues/135
arrayjoin with a sequential pipeline will now minimise inputs when they
are no longer being used. This drops the number of open file handles
needed to construct large arrays.
Sadly, memory savings are minimal.
see https://github.com/kleisauke/net-vips/issues/135
John Cupitt
Kleis Auke Wolthuizen
Lovell Fuller
Daniel Löbl
shado23
Heshy Roskes
David Manthey