libvips/test/test_threading.sh

58 lines
1.3 KiB
Bash
Raw Normal View History

#!/bin/sh
# set -x
2014-11-20 15:36:04 +01:00
. ./variables.sh
exit_code=0
$vips im_benchmarkn || exit_code=$?
if [ $exit_code -ne 0 ]; then
echo "im_benchmark is not available, skipping test"
exit 0
fi
chain=1
# im_benchmark needs a labq
$vips colourspace $image $tmp/t3.v labq
for tile in 10 64 128 512; do
# benchmark includes a dither which will vary with tile size
$vips --vips-concurrency=1 \
--vips-tile-width=$tile --vips-tile-height=$tile \
im_benchmarkn $tmp/t3.v $tmp/t5.v $chain
for cpus in 2 3 4 5 6 7 8 99; do
echo trying cpus = $cpus, tile = $tile ...
$vips --vips-concurrency=$cpus \
--vips-tile-width=$tile --vips-tile-height=$tile \
im_benchmarkn $tmp/t3.v $tmp/t7.v $chain
$vips subtract $tmp/t5.v $tmp/t7.v $tmp/t8.v
$vips abs $tmp/t8.v $tmp/t9.v
max=$($vips max $tmp/t9.v)
if [ $(echo "$max > 0" | bc) -eq 1 ]; then
break
fi
done
if [ $(echo "$max > 0" | bc) -eq 1 ]; then
break
fi
done
if [ $(echo "$max > 0" | bc) -eq 1 ]; then
echo error, max == $max
exit 1
else
Revised threading system (#3105) * reimplement threadpool just a set of threads that get recycled "ninja test" passes and dzsave seems to work, though pytest fails for some reason * clean up threading code a bit move base stuff into thread.c, so we have a simple thread -> threadset -> threadpool layering * start trying to revert g_threadpool based on the original commit * working! nice low systime again still need to repply cnages to threadpool.c since 80e0cc3d1 * reapply fixes from master so threadpool.c is now up to date * rename VipsThread as VipsWorker a bit less confusing * use a semaphore to count workers in a pool * tidy up * formatting * dynamic threadpool sizing based on counting the number of blocked threads in each pool it works, but the improvement is not great :( * add "concurrency" metadata item so operators can hint threadpool size (dzsave especially) * don't use thinstrip for small images * add RGB mode to openslide since flatten was taking 20% of CPU time for dzsave * fix up rgb mode now actually works * make the tile buffer per thread in the new openslideload rgb mode * fix dynamic pool downsize * mild refactoring * fix the buffer system oops, turned it off by mistake * all done! * revise changelog * Update libvips/iofuncs/threadset.c Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl> * Update libvips/iofuncs/threadset.c Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl> * LSan: add libMagickCore to suppression file * Revert "Remove mutex lock for VipsThreadStartFn" This reverts commit 41440491. * add VIPS_MAX_THREADS to set a hard limit on the threadset size * Revert "Revert "Remove mutex lock for VipsThreadStartFn"" This reverts commit 77e8520966ba79194fff3b4e648bbd295cd5c260. * remove sslock from sink.c * move fixed threadpool build to init not first use * add some doc comments * revert test suite threshold change * add a test for MAX_THREADS and move the test tmp/ area into the builddir * limit VIPS_MAX_THREADS to sane values * use tabs rather than spaces Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
2022-10-26 16:25:19 +02:00
echo all benchmark threading tests passed
fi
Revised threading system (#3105) * reimplement threadpool just a set of threads that get recycled "ninja test" passes and dzsave seems to work, though pytest fails for some reason * clean up threading code a bit move base stuff into thread.c, so we have a simple thread -> threadset -> threadpool layering * start trying to revert g_threadpool based on the original commit * working! nice low systime again still need to repply cnages to threadpool.c since 80e0cc3d1 * reapply fixes from master so threadpool.c is now up to date * rename VipsThread as VipsWorker a bit less confusing * use a semaphore to count workers in a pool * tidy up * formatting * dynamic threadpool sizing based on counting the number of blocked threads in each pool it works, but the improvement is not great :( * add "concurrency" metadata item so operators can hint threadpool size (dzsave especially) * don't use thinstrip for small images * add RGB mode to openslide since flatten was taking 20% of CPU time for dzsave * fix up rgb mode now actually works * make the tile buffer per thread in the new openslideload rgb mode * fix dynamic pool downsize * mild refactoring * fix the buffer system oops, turned it off by mistake * all done! * revise changelog * Update libvips/iofuncs/threadset.c Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl> * Update libvips/iofuncs/threadset.c Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl> * LSan: add libMagickCore to suppression file * Revert "Remove mutex lock for VipsThreadStartFn" This reverts commit 41440491. * add VIPS_MAX_THREADS to set a hard limit on the threadset size * Revert "Revert "Remove mutex lock for VipsThreadStartFn"" This reverts commit 77e8520966ba79194fff3b4e648bbd295cd5c260. * remove sslock from sink.c * move fixed threadpool build to init not first use * add some doc comments * revert test suite threshold change * add a test for MAX_THREADS and move the test tmp/ area into the builddir * limit VIPS_MAX_THREADS to sane values * use tabs rather than spaces Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
2022-10-26 16:25:19 +02:00
# setting VIPS_MAX_THREADS low should force a small thread limit
echo -n "checking threadset size limit ... "
VIPS_MAX_THREADS=5 VIPS_CONCURRENCY=3 $vips copy $image x.v || exit_code=$?
Revised threading system (#3105) * reimplement threadpool just a set of threads that get recycled "ninja test" passes and dzsave seems to work, though pytest fails for some reason * clean up threading code a bit move base stuff into thread.c, so we have a simple thread -> threadset -> threadpool layering * start trying to revert g_threadpool based on the original commit * working! nice low systime again still need to repply cnages to threadpool.c since 80e0cc3d1 * reapply fixes from master so threadpool.c is now up to date * rename VipsThread as VipsWorker a bit less confusing * use a semaphore to count workers in a pool * tidy up * formatting * dynamic threadpool sizing based on counting the number of blocked threads in each pool it works, but the improvement is not great :( * add "concurrency" metadata item so operators can hint threadpool size (dzsave especially) * don't use thinstrip for small images * add RGB mode to openslide since flatten was taking 20% of CPU time for dzsave * fix up rgb mode now actually works * make the tile buffer per thread in the new openslideload rgb mode * fix dynamic pool downsize * mild refactoring * fix the buffer system oops, turned it off by mistake * all done! * revise changelog * Update libvips/iofuncs/threadset.c Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl> * Update libvips/iofuncs/threadset.c Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl> * LSan: add libMagickCore to suppression file * Revert "Remove mutex lock for VipsThreadStartFn" This reverts commit 41440491. * add VIPS_MAX_THREADS to set a hard limit on the threadset size * Revert "Revert "Remove mutex lock for VipsThreadStartFn"" This reverts commit 77e8520966ba79194fff3b4e648bbd295cd5c260. * remove sslock from sink.c * move fixed threadpool build to init not first use * add some doc comments * revert test suite threshold change * add a test for MAX_THREADS and move the test tmp/ area into the builddir * limit VIPS_MAX_THREADS to sane values * use tabs rather than spaces Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
2022-10-26 16:25:19 +02:00
if [ $exit_code -ne 0 ]; then
echo FAILED
exit 1
fi
echo ok