Go to file
John Cupitt 976db37f84
Revised threading system (#3105)
* reimplement threadpool

just a set of threads that get recycled

"ninja test" passes and dzsave seems to work, though pytest fails for
some reason

* clean up threading code a bit

move base stuff into thread.c, so we have a simple

	thread -> threadset -> threadpool

layering

* start trying to revert g_threadpool

based on the original commit

* working!

nice low systime again

still need to repply cnages to threadpool.c since 80e0cc3d1

* reapply fixes from master

so threadpool.c is now up to date

* rename VipsThread as VipsWorker

a bit less confusing

* use a semaphore to count workers in a pool

* tidy up

* formatting

* dynamic threadpool sizing

based on counting the number of blocked threads in each pool

it works, but the improvement is not great :(

* add "concurrency" metadata item

so operators can hint threadpool size (dzsave especially)

* don't use thinstrip for small images

* add RGB mode to openslide

since flatten was taking 20% of CPU time for dzsave

* fix up rgb mode

now actually works

* make the tile buffer per thread

in the new openslideload rgb mode

* fix dynamic pool downsize

* mild refactoring

* fix the buffer system

oops, turned it off by mistake

* all done!

* revise changelog

* Update libvips/iofuncs/threadset.c

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>

* Update libvips/iofuncs/threadset.c

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>

* LSan: add libMagickCore to suppression file

* Revert "Remove mutex lock for VipsThreadStartFn"

This reverts commit 41440491.

* add VIPS_MAX_THREADS

to set a hard limit on the threadset size

* Revert "Revert "Remove mutex lock for VipsThreadStartFn""

This reverts commit 77e8520966ba79194fff3b4e648bbd295cd5c260.

* remove sslock from sink.c

* move fixed threadpool build to init

not first use

* add some doc comments

* revert test suite threshold change

* add a test for MAX_THREADS

and move the test tmp/ area into the builddir

* limit VIPS_MAX_THREADS to sane values

* use tabs rather than spaces

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
2022-10-26 15:25:19 +01:00
.github CI: various improvements (#3096) 2022-10-17 19:08:35 +01:00
cplusplus webpsave: switch to sink_disc (#3018) 2022-09-08 14:32:13 +01:00
doc webpsave: switch to sink_disc (#3018) 2022-09-08 14:32:13 +01:00
examples copied invert example into examples (#3069) 2022-09-29 11:48:24 +01:00
fuzz remove autotools (#2941) 2022-07-24 11:14:52 +01:00
libvips Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
man Remove remnant of Autotools (#2950) 2022-07-27 10:56:56 +01:00
po Regenerate POTFILES.in (#2951) 2022-07-27 13:34:20 +01:00
suppressions Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
test Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
tools Remove cruft (#2942) 2022-07-24 11:18:30 +01:00
.editorconfig Remove remnant of Autotools (#2950) 2022-07-27 10:56:56 +01:00
.gitattributes Remove remnant of Autotools (#2950) 2022-07-27 10:56:56 +01:00
.gitignore Remove remnant of Autotools (#2950) 2022-07-27 10:56:56 +01:00
ChangeLog Revised threading system (#3105) 2022-10-26 15:25:19 +01:00
CODE_OF_CONDUCT.md tiny reformat of CoC 2020-04-09 12:23:48 +01:00
LICENSE Remove remnant of Autotools (#2950) 2022-07-27 10:56:56 +01:00
meson_options.txt add some more examples 2022-09-09 10:14:59 +01:00
meson.build Merge branch '8.13' 2022-10-11 08:49:03 +01:00
README.md clarify readme a little 2022-10-19 09:12:12 +01:00

libvips : an image processing library

CI Fuzzing Status Coverity Status Gitter

Introduction

libvips is a demand-driven, horizontally threaded image processing library. Compared to similar libraries, libvips runs quickly and uses little memory. libvips is licensed under the LGPL 2.1+.

It has around 300 operations covering arithmetic, histograms, convolution, morphological operations, frequency filtering, colour, resampling, statistics and others. It supports a large range of numeric types, from 8-bit int to 128-bit complex. Images can have any number of bands. It supports a good range of image formats, including JPEG, JPEG2000, JPEG-XL, TIFF, PNG, WebP, HEIC, AVIF, FITS, Matlab, OpenEXR, PDF, SVG, HDR, PPM / PGM / PFM, CSV, GIF, Analyze, NIfTI, DeepZoom, and OpenSlide. It can also load images via ImageMagick or GraphicsMagick, letting it work with formats like DICOM.

It comes with bindings for C, C++, and the command-line. Full bindings are available for :

Language Binding
Ruby ruby-vips
Python pyvips
PHP php-vips
C# / .NET NetVips
Go govips
Lua lua-vips
Crystal crystal-vips
Elixir vix

libvips is used as an image processing engine by:

sharp (on node.js)
bimg
sharp for Go
Ruby on Rails
carrierwave-vips
mediawiki
PhotoFlow

and others. The official libvips GUI is nip2, a strange combination of a spreadsheet and a photo editor.

Install

There are packages for most Unix-like operating systems, including macOS. Check your package manager.

There are binaries for Windows in releases.

The libvips website has detailed install notes.

Building from source

libvips uses the Meson build system, version 0.56 or later. Meson can use ninja, Visual Studio or XCode as a backend, so you'll also need one of them.

libvips must have build-essential, pkg-config, libglib2.0-dev, libexpat1-dev. See the Dependencies section below for a full list of the libvips optional dependencies.

Cheatsheet

cd libvips-x.y.x
meson build --prefix /my/install/prefix
cd build
meson compile
meson test
meson install

Check the output of meson setup carefully and make sure it found everything you wanted it to find. Add arguments to meson setup to change the build configuration.

  • Add flags like -Dnsgif=false to turn libvips options on and off, see meson_options.txt for a list of all the build options libvips supports.

  • Add flags like -Dmagick=disabled to turn libvips dependencies on and off, see meson_options.txt and the list below for a summary of all the libvips dependencies.

  • Meson will do a debug build by default. Add --buildtype release for a release (optimised) build.

  • You might need to add --libdir lib on Debian if you don't want the arch name in the library path.

  • Add --default-library static for a static build.

  • Use eg. CC=clang CXX=clang++ meson setup ... to change compiler.

  • You can have many build-dir, pick whatever names you like, for example one for release and one for debug.

There's a more comprehensive test suite you can run once libvips has been installed. Use pytest in the libvips base directory.

Optional dependencies

If suitable versions are found, libvips will add support for the following libraries automatically. Packages are generally found with pkg-config, so make sure that is working.

libjpeg

Anything that is compatible with the IJG JPEG library. Use mozjpeg if you can. Another option is libjpeg-turbo.

libexif

If available, libvips adds support for EXIF metadata in JPEG files.

librsvg

The usual SVG loader. If this is not present, vips will try to load SVGs via imagemagick instead.

PDFium

If present, libvips will attempt to load PDFs with PDFium. Download the prebuilt pdfium binary from:

https://github.com/bblanchon/pdfium-binaries

Untar to the libvips install prefix, for example:

cd ~/vips
tar xf ~/pdfium-linux.tgz

Create a pdfium.pc like this (update the version number):

VIPSHOME=/home/john/vips
cat > $VIPSHOME/lib/pkgconfig/pdfium.pc << EOF
     prefix=$VIPSHOME
     exec_prefix=\${prefix}
     libdir=\${exec_prefix}/lib
     includedir=\${prefix}/include
     Name: pdfium
     Description: pdfium
     Version: 4290
     Requires:
     Libs: -L\${libdir} -lpdfium
     Cflags: -I\${includedir}
EOF

If PDFium is not detected, libvips will look for poppler-glib instead.

poppler-glib

The Poppler PDF renderer, with a glib API. If this is not present, vips will try to load PDFs via imagemagick.

cgif

If available, libvips will save GIFs with cgif. If this is not present, vips will try to save gifs via imagemagick instead.

libgsf-1

If available, libvips adds support for creating image pyramids with dzsave.

libtiff

The TIFF library. It needs to be built with support for JPEG and ZIP compression. 3.4b037 and later are known to be OK.

fftw3

If libvips finds this library, it uses it for fourier transforms.

lcms2

If present, vips_icc_import(), vips_icc_export() and vips_icc_transform() can be used to manipulate images with ICC profiles.

libspng

If present, libvips will load and save PNG files using libspng. If not, it will look for the standard libpng package.

libimagequant, quantizr

If one of these quantisation packages is present, libvips can write 8-bit palette-ised PNGs and GIFs.

ImageMagick, or optionally GraphicsMagick

If available, libvips adds support for loading and saving all libMagick-supported image file types. You can enable and disable load and save separately.

Imagemagick 6.9+ needs to have been built with --with-modules. Most packaged IMs are, I think.

If you are going to be using libvips with untrusted images, perhaps in a web server, for example, you should consider the security implications of enabling a package with such a large attack surface.

pangocairo

If available, libvips adds support for text rendering. You need the package pangocairo in pkg-config --list-all.

orc-0.4

If available, vips will accelerate some operations with this run-time compiler.

matio

If available, vips can load images from Matlab save files.

cfitsio

If available, vips can load FITS images.

libwebp

If available, vips can load and save WebP images.

libniftiio

If available, vips can load and save NIfTI images.

OpenEXR

If available, libvips will directly read (but not write, sadly) OpenEXR images.

OpenJPEG

If available, libvips will read and write JPEG2000 images.

libjxl

If available, libvips will read and write JPEG-XL images.

OpenSlide

If available, libvips can load OpenSlide-supported virtual slide files: Aperio, Hamamatsu, Leica, MIRAX, Sakura, Trestle, and Ventana.

libheif

If available, libvips can load and save HEIC and AVIF images. Your libheif (in turn) needs to be built with the correct decoders and encoders. You can check with eg.:

$ pkg-config libheif --print-variables
builtin_avif_decoder
builtin_avif_encoder
builtin_h265_decoder
builtin_h265_encoder
exec_prefix
includedir
libdir
pcfiledir
prefix

Contributors

Code Contributors

This project exists thanks to all the people who contribute.

Organizations

Support this project with your organization. Your logo will show up here with a link to your website.