* reimplement threadpool
just a set of threads that get recycled
"ninja test" passes and dzsave seems to work, though pytest fails for
some reason
* clean up threading code a bit
move base stuff into thread.c, so we have a simple
thread -> threadset -> threadpool
layering
* start trying to revert g_threadpool
based on the original commit
* working!
nice low systime again
still need to repply cnages to threadpool.c since 80e0cc3d1
* reapply fixes from master
so threadpool.c is now up to date
* rename VipsThread as VipsWorker
a bit less confusing
* use a semaphore to count workers in a pool
* tidy up
* formatting
* dynamic threadpool sizing
based on counting the number of blocked threads in each pool
it works, but the improvement is not great :(
* add "concurrency" metadata item
so operators can hint threadpool size (dzsave especially)
* don't use thinstrip for small images
* add RGB mode to openslide
since flatten was taking 20% of CPU time for dzsave
* fix up rgb mode
now actually works
* make the tile buffer per thread
in the new openslideload rgb mode
* fix dynamic pool downsize
* mild refactoring
* fix the buffer system
oops, turned it off by mistake
* all done!
* revise changelog
* Update libvips/iofuncs/threadset.c
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
* Update libvips/iofuncs/threadset.c
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
* LSan: add libMagickCore to suppression file
* Revert "Remove mutex lock for VipsThreadStartFn"
This reverts commit 41440491.
* add VIPS_MAX_THREADS
to set a hard limit on the threadset size
* Revert "Revert "Remove mutex lock for VipsThreadStartFn""
This reverts commit 77e8520966ba79194fff3b4e648bbd295cd5c260.
* remove sslock from sink.c
* move fixed threadpool build to init
not first use
* add some doc comments
* revert test suite threshold change
* add a test for MAX_THREADS
and move the test tmp/ area into the builddir
* limit VIPS_MAX_THREADS to sane values
* use tabs rather than spaces
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
We were minimising at the end of threadpool_run, but this was being
called for each sinkscreen render pass, so we were throwing away the
display cache.
Instead, minimise after sink, sink_memory and sink_disc.
Use GLib's i18n support instead of copying and pasting that
logic into its own header. This deprecates the vips/intl.h
header in favour of glib/gi18n.h.
Since threads on Windows can be terminated prior to the execution
of the atexit registered functions (i.e. vips_shutdown), causing
a surprising deadlock. This thread is never recycled, so it should
be fine.
Reverts commit d1b4d12388 partially.
vips_{avg,deviate,hough,max,min,stats} are the only arithmetic functions
that do not require a mutex on the _start and/or _stop function. All
other arithmetic functions still needs this, so move it to sink instead.
we always made N threads for a threadpool ... we could end up making
loads of threads we never used if the image was tiny
for very small images, limit the number of threads we make
but see the TODO in dzsave.c
- no way to pick a tile format, always jpeg
- no way to pick a container format, always zip
- needs testing
- examine memuse, how well do we work for large zips
- can we make a zip over 4gb?
John Cupitt
Kleis Auke Wolthuizen
Lovell Fuller
jtorresfabra
Alistair Thomas