libvips/libvips/foreign
John Cupitt 976db37f84
Revised threading system (#3105)
* reimplement threadpool

just a set of threads that get recycled

"ninja test" passes and dzsave seems to work, though pytest fails for
some reason

* clean up threading code a bit

move base stuff into thread.c, so we have a simple

	thread -> threadset -> threadpool

layering

* start trying to revert g_threadpool

based on the original commit

* working!

nice low systime again

still need to repply cnages to threadpool.c since 80e0cc3d1

* reapply fixes from master

so threadpool.c is now up to date

* rename VipsThread as VipsWorker

a bit less confusing

* use a semaphore to count workers in a pool

* tidy up

* formatting

* dynamic threadpool sizing

based on counting the number of blocked threads in each pool

it works, but the improvement is not great :(

* add "concurrency" metadata item

so operators can hint threadpool size (dzsave especially)

* don't use thinstrip for small images

* add RGB mode to openslide

since flatten was taking 20% of CPU time for dzsave

* fix up rgb mode

now actually works

* make the tile buffer per thread

in the new openslideload rgb mode

* fix dynamic pool downsize

* mild refactoring

* fix the buffer system

oops, turned it off by mistake

* all done!

* revise changelog

* Update libvips/iofuncs/threadset.c

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>

* Update libvips/iofuncs/threadset.c

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>

* LSan: add libMagickCore to suppression file

* Revert "Remove mutex lock for VipsThreadStartFn"

This reverts commit 41440491.

* add VIPS_MAX_THREADS

to set a hard limit on the threadset size

* Revert "Revert "Remove mutex lock for VipsThreadStartFn""

This reverts commit 77e8520966ba79194fff3b4e648bbd295cd5c260.

* remove sslock from sink.c

* move fixed threadpool build to init

not first use

* add some doc comments

* revert test suite threshold change

* add a test for MAX_THREADS

and move the test tmp/ area into the builddir

* limit VIPS_MAX_THREADS to sane values

* use tabs rather than spaces

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
2022-10-26 15:25:19 +01:00
..
libnsgif remove autotools (#2941) 2022-07-24 11:14:52 +01:00
analyze2vips.c revise loader demand hints 2022-09-03 15:31:37 +01:00
analyzeload.c Add a way to disable less well tested operations (#2636) 2022-04-11 11:32:32 +01:00
cairo.c fix background byte ordering in PDF 2022-05-16 14:28:53 +01:00
cgifsave.c oop debug left on 2022-09-08 10:21:19 +01:00
csvload.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
csvsave.c add vips_target_end (#2802) 2022-05-20 16:38:04 +01:00
dbh.h fix compiler warnings 2013-10-18 09:13:24 +01:00
dzsave.c Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
exif.c Merge branch '8.13' 2022-10-19 12:47:53 +01:00
fits.c Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
fitsload.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
fitssave.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
foreign.c Merge branch '8.13' 2022-09-20 10:04:22 +01:00
heifload.c fix an undefined shift 2022-09-15 12:57:36 +01:00
heifsave.c Merge branch '8.13' 2022-10-11 08:49:03 +01:00
jp2kload.c revise loader demand hints 2022-09-03 15:31:37 +01:00
jp2ksave.c jp2ksave saves as jp2 2022-08-11 16:49:40 +01:00
jpeg.h move tiff decompress outside lock (#2969) 2022-09-03 13:10:58 +01:00
jpeg2vips.c move tiff decompress outside lock (#2969) 2022-09-03 13:10:58 +01:00
jpegload.c add "unlimited" to jpegload 2022-08-02 13:50:09 +01:00
jpegsave.c add dzsave to a target (#2718) 2022-05-20 18:38:46 +01:00
jxlload.c fix jxl load rewind 2022-08-14 17:27:13 +01:00
jxlsave.c Merge branch '8.13' 2022-09-20 10:04:22 +01:00
magick.c magicksave: Add bitdepth option (#2819) 2022-05-27 11:41:01 +01:00
magick.h magicksave: Add bitdepth option (#2819) 2022-05-27 11:41:01 +01:00
magick2vips.c Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
magick6load.c revise help text for loaders 2022-08-10 12:50:53 +01:00
magick7load.c Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
magickload.c Cleanup gettext handling (#2695) 2022-02-27 15:27:28 +00:00
magicksave.c magicksave: Add bitdepth option (#2819) 2022-05-27 11:41:01 +01:00
matlab.c revise loader demand hints 2022-09-03 15:31:37 +01:00
matload.c Add a way to disable less well tested operations (#2636) 2022-04-11 11:32:32 +01:00
matrixload.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
matrixsave.c add vips_target_end (#2802) 2022-05-20 16:38:04 +01:00
meson.build webpsave: switch to sink_disc (#3018) 2022-09-08 14:32:13 +01:00
niftiload.c revise loader demand hints 2022-09-03 15:31:37 +01:00
niftisave.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
nsgifload.c revise formatting 2022-10-12 18:27:10 +01:00
openexr2vips.c Cleanup gettext handling (#2695) 2022-02-27 15:27:28 +00:00
openexrload.c Add a way to disable less well tested operations (#2636) 2022-04-11 11:32:32 +01:00
openslideload.c Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
pdfiumload.c Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
pforeign.h webpsave: switch to sink_disc (#3018) 2022-09-08 14:32:13 +01:00
pngload.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
pngsave.c pngsave: ensure 8-bit palette images can be created (#3031) 2022-09-08 09:10:02 +01:00
popplerload.c revise loader demand hints 2022-09-03 15:31:37 +01:00
ppmload.c add ".pnm" save 2022-08-30 12:31:47 +01:00
ppmsave.c add ".pnm" save 2022-08-30 12:31:47 +01:00
quantise.c try to make cgifsave easier to read (#2853) 2022-06-12 11:25:32 +01:00
quantise.h cgifsave: palette change POC (#2824) 2022-06-08 15:27:29 +01:00
radiance.c add vips_target_end (#2802) 2022-05-20 16:38:04 +01:00
radload.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
radsave.c Cleanup gettext handling (#2695) 2022-02-27 15:27:28 +00:00
rawload.c Add a way to disable less well tested operations (#2636) 2022-04-11 11:32:32 +01:00
rawsave.c Cleanup gettext handling (#2695) 2022-02-27 15:27:28 +00:00
spngload.c update spngload for new header test 2022-09-03 22:42:41 +01:00
spngsave.c spngsave: Don't shift indexed pixels (#3101) 2022-10-19 17:34:43 +01:00
svgload.c revise loader demand hints 2022-09-03 15:31:37 +01:00
tiff.c Add tiffsave target (#2798) 2022-05-20 18:38:17 +01:00
tiff.h Add tiffsave target (#2798) 2022-05-20 18:38:17 +01:00
tiff2vips.c tiff2vips: avoid `g_assert_not_reached` in the default clause (#3064) 2022-09-24 13:36:02 +01:00
tiffload.c revise help text for loaders 2022-08-10 12:50:53 +01:00
tiffsave.c Update C++ binding and function list (#2851) 2022-06-10 14:43:44 +01:00
vips2jpeg.c move tiff decompress outside lock (#2969) 2022-09-03 13:10:58 +01:00
vips2magick.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
vips2tiff.c Merge branch '8.13' 2022-09-22 14:11:31 +01:00
vipsload.c Prefer to use `flags |=` everywhere (#2854) 2022-06-11 15:40:39 +01:00
vipspng.c Merge branch '8.13' 2022-10-14 20:06:34 +01:00
vipssave.c add vips_target_end (#2802) 2022-05-20 16:38:04 +01:00
webp2vips.c Cleanup gettext handling (#2695) 2022-02-27 15:27:28 +00:00
webpload.c revise help text for loaders 2022-08-10 12:50:53 +01:00
webpsave.c Fix build with `-Dwebp=disabled` (#3106) 2022-10-20 10:34:08 +01:00