diff --git a/TODO b/TODO index 48d6e63c..36f83c6e 100644 --- a/TODO +++ b/TODO @@ -1,5 +1,3 @@ -- move "how libvips opens files" into docs? - - not sure about utf8 error messages on win - strange: diff --git a/doc/How-it-opens-files.md b/doc/How-it-opens-files.md index 4e960202..3e4e4165 100644 --- a/doc/How-it-opens-files.md +++ b/doc/How-it-opens-files.md @@ -1,22 +1,33 @@ -libvips now has at least four different ways of opening image files, each best -for different file types, file sizes and image use cases. libvips -tries hard to pick the best strategy in each case and mostly you don't need -to know what it is doing behind the scenes, except unfortunately when you do. + + Opening files + 3 + libvips + -This post tries to explain what the different strategies are and when each is + + Opening + How libvips opens files + + +libvips now has at least four different ways of opening image files, each +best for different file types, file sizes and image use cases. libvips tries +hard to pick the best strategy in each case and mostly you don't need to +know what it is doing behind the scenes, except unfortunately when you do. + +This page tries to explain what the different strategies are and when each is used. If you are running into unexpected memory, disc or CPU use, this might -be helpful. `vips_image_new_from_file()` has the official documentation. +be helpful. `vips_image_new_from_file()` has the official documentation. # Direct access -This is the fastest and simplest one. The file is mapped directly into -the process's address space and can be read with ordinary pointer -access. Small files are completely mapped; large files are mapped in a -series of small windows that are shared and which scroll about as pixels -are read. Files which are accessed like this can be read by many threads -at once, making them especially quick. They also interact well with the -computer's operating system: your OS will use spare memory to cache -recently used chunks of the file, very handy. +This is the fastest and simplest one. The file is mapped directly into the +process's address space and can be read with ordinary pointer access. Small +files are completely mapped; large files are mapped in a series of small +windows that are shared and which scroll about as pixels are read. Files +which are accessed like this can be read by many threads at once, making +them especially quick. They also interact well with the computer's operating +system: your OS will use spare memory to cache recently used chunks of the +file. For this to be possible, the file format needs to be a simple dump of a memory array. libvips supports direct access for vips, 8-bit binary ppm/pbm/pnm, diff --git a/doc/How-it-opens-files.xml b/doc/How-it-opens-files.xml index 1941bd97..64e213d7 100644 --- a/doc/How-it-opens-files.xml +++ b/doc/How-it-opens-files.xml @@ -3,28 +3,23 @@ "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> - - How-it-opens-files.md - 3 - libvips - - - - libvips - How-it-opens-files.md - - + + Opening files 3 libvips + + + Opening How libvips opens files + libvips now has at least four different ways of opening image files, each best for different file types, file sizes and image use cases. libvips tries hard to pick the best strategy in each case and mostly you don’t need to know what it is doing behind the scenes, except unfortunately when you do. - This post tries to explain what the different strategies are and when each is used. If you are running into unexpected memory, disc or CPU use, this might be helpful. vips_image_new_from_file() has the official documentation. + This page tries to explain what the different strategies are and when each is used. If you are running into unexpected memory, disc or CPU use, this might be helpful. vips_image_new_from_file() has the official documentation. Direct access - This is the fastest and simplest one. The file is mapped directly into the process’s address space and can be read with ordinary pointer access. Small files are completely mapped; large files are mapped in a series of small windows that are shared and which scroll about as pixels are read. Files which are accessed like this can be read by many threads at once, making them especially quick. They also interact well with the computer’s operating system: your OS will use spare memory to cache recently used chunks of the file, very handy. + This is the fastest and simplest one. The file is mapped directly into the process’s address space and can be read with ordinary pointer access. Small files are completely mapped; large files are mapped in a series of small windows that are shared and which scroll about as pixels are read. Files which are accessed like this can be read by many threads at once, making them especially quick. They also interact well with the computer’s operating system: your OS will use spare memory to cache recently used chunks of the file. For this to be possible, the file format needs to be a simple dump of a memory array. libvips supports direct access for vips, 8-bit binary ppm/pbm/pnm, analyse and raw. diff --git a/doc/How-it-works.md b/doc/How-it-works.md index 683e678a..fc74394a 100644 --- a/doc/How-it-works.md +++ b/doc/How-it-works.md @@ -1,3 +1,14 @@ + + How libvips works + 3 + libvips + + + + Internals + A high-level technical overview of libvips's evaluation system + + Compared to most image processing libraries, VIPS needs little RAM and runs quickly, especially on machines with more than one CPU. VIPS achieves this improvement by only keeping the pixels currently being processed in RAM @@ -152,8 +163,8 @@ from FITS images and from tiled OpenEXR images. VIPS will automatically unpack other formats to temporary disc files for you but this can obviously generate a lot of disc traffic. It also has a special sequential mode for streaming operations on non-random-access -formats. A post on the libvips blog [explains how libvips opens a -file](http://libvips.blogspot.co.uk/2012/06/how-libvips-opens-file.html). One +formats. Another section in these docs explains [how libvips opens a +file](How-it-opens-files.html). One of the sources uses the [ImageMagick](http://www.imagemagick.org) (or optionally [GraphicsMagick](http://www.graphicsmagick.org)) library, so VIPS can read any image format that these libraries can read. diff --git a/doc/How-it-works.xml b/doc/How-it-works.xml index 5b4064dc..8cdf0606 100644 --- a/doc/How-it-works.xml +++ b/doc/How-it-works.xml @@ -3,18 +3,13 @@ "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> - - How-it-works.md - 3 - libvips - - - - libvips - How-it-works.md - - + + How libvips works 3 libvips + + + Internals A high-level technical overview of libvips’s evaluation system + Compared to most image processing libraries, VIPS needs little RAM and runs quickly, especially on machines with more than one CPU. VIPS achieves this improvement by only keeping the pixels currently being processed in RAM and by having an efficient, threaded image IO system. This page explains how these features are implemented. diff --git a/doc/Making-image-pyramids.md b/doc/Making-image-pyramids.md index 399fb4a2..4eab76ad 100644 --- a/doc/Making-image-pyramids.md +++ b/doc/Making-image-pyramids.md @@ -1,3 +1,14 @@ + + Image pyramids + 3 + libvips + + + + Pyramids + How to use libvips to make image pyramids + + libvips includes `vips_dzsave()`, an operation that can build image pyramids compatible with [DeepZoom](http://en.wikipedia.org/wiki/Deep_Zoom), Zoomify and [Google Maps](https://developers.google.com/maps/) image viewers. It's diff --git a/doc/Making-image-pyramids.xml b/doc/Making-image-pyramids.xml index a43754ce..6be7b4f7 100644 --- a/doc/Making-image-pyramids.xml +++ b/doc/Making-image-pyramids.xml @@ -3,18 +3,13 @@ "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> - - Making-image-pyramids.md - 3 - libvips - - - - libvips - Making-image-pyramids.md - - + + Image pyramids 3 libvips + + + Pyramids How to use libvips to make image pyramids + libvips includes vips_dzsave(), an operation that can build image pyramids compatible with DeepZoom, Zoomify and Google Maps image viewers. It’s fast and can generate pyramids for large images using only a small amount of memory. diff --git a/doc/Using-vipsthumbnail.md b/doc/Using-vipsthumbnail.md index c4fd6ac4..09428ea6 100644 --- a/doc/Using-vipsthumbnail.md +++ b/doc/Using-vipsthumbnail.md @@ -1,10 +1,27 @@ -libvips ships with a handy command-line image thumbnailer, `vipsthumbnail`. -This page introduces it with examples. + + Using `vipsthumbnail` + 3 + libvips + -The thumbnailing functionality is implemeted by -`vips_thumbnail()` and -`vips_thumbnail_buffer()`, see the docs for details. You can use these -functions from any language with a libvips binding. + + `vipsthumbnail` + Introduction to `vipsthumbnail`, with examples + + +libvips ships with a handy command-line image thumbnailer, `vipsthumbnail`. +This page introduces it, with some examples. + +The thumbnailing functionality is implemented by `vips_thumbnail()` and +`vips_thumbnail_buffer()` (which thumbnails an image held as a string), +see the docs for details. You can use these functions from any language +with a libvips binding. For example, from PHP you could write: + +```php +$filename = ...; +$image = Vips\Image::thumbnail($filename, 200, ["height" => 200]); +$image.writeToFile("my-thumbnail.jpg"); +``` # libvips options @@ -25,15 +42,15 @@ is running.  # Looping -vipsthumbnail can process many images in one operation. For example: +`vipsthumbnail` can process many images in one command. For example: ``` $ vipsthumbnail *.jpg ``` will make a thumbnail for every jpeg in the current directory.  See the -**Output directory** section below to see how to change where thumbnails -are written. +[Output directory](#output-directory) section below to see how to change +where thumbnails are written. `vipsthumbnail` will process images one after the other. You can get a good speedup by running several `vipsthumbnail`s in parallel, depending on how @@ -93,7 +110,7 @@ for details. Shrinking images involves combining many pixels into one. Arithmetic averaging really ought to be in terms of the number of photons, but (for historical reasons) the values stored in image files are usually related -to the voltage that should be applied to a CRT electron gun. +to the voltage that should be applied to the electron gun in a CRT display. `vipsthumbnail` has an option to perform image shrinking in linear space, that is, a colourspace where values are proportional to photon numbers. For example: @@ -103,10 +120,22 @@ $ vipsthumbnail fred.jpg --linear ``` The downside is that in linear mode, none of the very fast shrink-on-load -tricks that `vipsthumbnail` normally uses are possible, since the shrinking -done by the image libraries is done at encode time, and done in -terms of CRT voltage, not light. This can make linear light thumbnailing of -large images extremely slow. +tricks that `vipsthumbnail` normally uses are possible, since the shrinking is +done at encode time, not decode time, and is done in terms of CRT voltage, not +photons. This can make linear light thumbnailing of large images extremely slow. + +For example, for a 10,000 x 10,000 pixel JPEG I see: + +``` +$ time vipsthumbnail wtc.jpg +real 0m0.317s +user 0m0.292s +sys 0m0.016s +$ time vipsthumbnail wtc.jpg --linear +real 0m4.660s +user 0m4.640s +sys 0m0.016s +``` # Output directory @@ -118,27 +147,24 @@ produce the output filename. For example: $ vipsthumbnail fred.jpg jim.tif -o tn_%s.jpg ``` -For each of the files to be thumbnailed, `vipsthumbnail` -will drop the extension (`.jpg` and `.tif` -in this case) and then substitute the name into the `-o` -option, replacing the `%s` -So this example will write thumbnails to `tn_fred.jpg` and `tn_jim.jpg`. +For each of the files to be thumbnailed, `vipsthumbnail` will drop the +extension (`.jpg` and `.tif` in this case) and then substitute the name into +the `-o` option, replacing the `%s` So this example will write thumbnails to +`tn_fred.jpg` and `tn_jim.jpg`. -If the pattern given to `-o` -is an absolute path, any path components are dropped from the input -filenames. This lets you write all of your thumbnails to a specific -directory, if you want. For example: +If the pattern given to `-o` is an absolute path, any path components are +dropped from the input filenames. This lets you write all of your thumbnails +to a specific directory, if you want. For example: ``` $ vipsthumbnail fred.jpg ../jim.tif -o /mythumbs/tn_%s.jpg ``` -Now both thumbnails will be written to `/mythumbs`, -even though the source images are in different directories. +Now both thumbnails will be written to `/mythumbs`, even though the source +images are in different directories. -Conversely, if `-o` -is set to a relative path, any path component from the input file is -prepended. For example: +Conversely, if `-o` is set to a relative path, any path component from the +input file is prepended. For example: ``` $ vipsthumbnail fred.jpg ../jim.tif -o mythumbs/tn_%s.jpg @@ -149,8 +175,7 @@ their current directory. # Output format and options -You can use `-o` -to specify the thumbnail image format too. For example:  +You can use `-o` to specify the thumbnail image format too. For example:  ``` $ vipsthumbnail fred.jpg ../jim.tif -o tn_%s.png @@ -158,18 +183,17 @@ $ vipsthumbnail fred.jpg ../jim.tif -o tn_%s.png Will write thumbnails in PNG format. -You can give options to the image write operation as a list of -comma-separated arguments in square brackets. For example: +You can give options to the image write operation as a list of comma-separated +arguments in square brackets. For example: ``` $ vipsthumbnail fred.jpg ../jim.tif -o > tn_%s.jpg[Q=90,optimize_coding] ``` -will write jpeg images with quality 90, and will turn on the libjpeg -coding optimizer. +will write jpeg images with quality 90, and will turn on the libjpeg coding +optimizer. -Check the image write operations to see all the possible options. For -example: +Check the image write operations to see all the possible options. For example: ``` $ vips jpegsave diff --git a/doc/Using-vipsthumbnail.xml b/doc/Using-vipsthumbnail.xml index 525b125b..71d62f62 100644 --- a/doc/Using-vipsthumbnail.xml +++ b/doc/Using-vipsthumbnail.xml @@ -3,25 +3,25 @@ "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> - - Using-vipsthumbnail.md - 3 - libvips - - - - libvips - Using-vipsthumbnail.md - - - libvips ships with a handy command-line image thumbnailer, vipsthumbnail. This page introduces it with examples. + Using vipsthumbnail 3 libvips - The thumbnailing functionality is implemeted by vips_thumbnail() and vips_thumbnail_buffer(), see the docs for details. You can use these functions from any language with a libvips binding. + vipsthumbnail Introduction to vipsthumbnail, with examples - + + libvips ships with a handy command-line image thumbnailer, vipsthumbnail. This page introduces it, with some examples. + + + The thumbnailing functionality is implemented by vips_thumbnail() and vips_thumbnail_buffer() (which thumbnails an image held as a string), see the docs for details. You can use these functions from any language with a libvips binding. For example, from PHP you could write: + + +$filename = ...; +$image = Vips\Image::thumbnail($filename, 200, ["height" => 200]); +$image.writeToFile("my-thumbnail.jpg"); + + libvips options vipsthumbnail supports the usual range of vips command-line options. A few of them are useful: @@ -38,17 +38,17 @@ --vips-info shows a higher level view of the operations that vipsthumbnail is running.  - - + + Looping - vipsthumbnail can process many images in one operation. For example: + vipsthumbnail can process many images in one command. For example: $ vipsthumbnail *.jpg - will make a thumbnail for every jpeg in the current directory.  See the Output directory section below to see how to change where thumbnails are written. + will make a thumbnail for every jpeg in the current directory.  See the Output directory section below to see how to change where thumbnails are written. vipsthumbnail will process images one after the other. You can get a good speedup by running several vipsthumbnails in parallel, depending on how much load you want to put on your system. For example: @@ -56,8 +56,8 @@ $ vipsthumbnail *.jpg $ parallel vipsthumbnail ::: *.jpg - - + + Thumbnail size You can set the bounding box of the generated thumbnail with the --size option. For example: @@ -77,8 +77,8 @@ $ vipsthumbnail shark.jpg --size 200x You can append < or > to mean only resize if the image is smaller or larger than the target. - - + + Cropping vipsthumbnail normally shrinks images to fit within the box set by --size. You can use the --smartcrop option to crop to fill the box instead. Excess pixels are trimmed away using the strategy you set. For example: @@ -111,8 +111,8 @@ $ vipsthumbnail owl.jpg --smartcrop attention -s 128 First it shrinks the image to get the vertical axis to 128 pixels, then crops down to 128 pixels across using the attention strategy. This one searches the image for features which might catch a human eye, see vips_smartcrop() for details. - - + + Linear light Shrinking images involves combining many pixels into one. Arithmetic averaging really ought to be in terms of the number of photons, but (for historical reasons) the values stored in image files are usually related to the voltage that should be applied to a CRT electron gun. @@ -124,10 +124,23 @@ $ vipsthumbnail owl.jpg --smartcrop attention -s 128 $ vipsthumbnail fred.jpg --linear - The downside is that in linear mode, none of the very fast shrink-on-load tricks that vipsthumbnail normally uses are possible, since the shrinking done by the image libraries is done at encode time, and done in terms of CRT voltage, not light. This can make linear light thumbnailing of large images extremely slow. + The downside is that in linear mode, none of the very fast shrink-on-load tricks that vipsthumbnail normally uses are possible, since the shrinking is done at encode time, not decode time, and is done in terms of CRT voltage, not photons. This can make linear light thumbnailing of large images extremely slow. - - + + For example, for a 10,000 x 10,000 pixel JPEG I see: + + +$ time vipsthumbnail wtc.jpg +real 0m0.317s +user 0m0.292s +sys 0m0.016s +$ time vipsthumbnail wtc.jpg --linear +real 0m4.660s +user 0m4.640s +sys 0m0.016s + + + Output directory You set the thumbnail write parameters with the -o option. This is a pattern which the input filename is pasted into to produce the output filename. For example: @@ -156,8 +169,8 @@ $ vipsthumbnail fred.jpg ../jim.tif -o mythumbs/tn_%s.jpg Now both input files will have thumbnails written to a subdirectory of their current directory. - - + + Output format and options You can use -o to specify the thumbnail image format too. For example:  @@ -231,8 +244,8 @@ $ vipsthumbnail 42-32157534.jpg -o x.jpg[optimize_coding,strip] $ ls -l x.jpg -rw-r–r– 1 john john 3600 Nov 12 21:27 x.jpg - - + + Colour management vipsthumbnail will optionally put images through LittleCMS for you. You can use this to move all thumbnails to the same colour space. All web browsers assume that images without an ICC profile are in sRGB colourspace, so if you move your thumbnails to sRGB, you can strip all the embedded profiles. This can save several kb per thumbnail. @@ -259,8 +272,8 @@ $ ls -l tn_shark.jpg You can also specify a fallback input profile to use if the image has no embedded one, but this is less useful. - - + + Auto-rotate Many JPEG files have a hint set in the header giving the image orientation. If you strip out the metadata, this hint will be lost, and the image will appear to be rotated. @@ -268,8 +281,8 @@ $ ls -l tn_shark.jpg If you use the --rotate option, vipsthumbnail examines the image header and if there’s an orientation tag, applies and removes it. - - + + Final suggestion Putting all this together, I suggest this as a sensible set of options: @@ -281,7 +294,7 @@ $ vipsthumbnail fred.jpg \ --eprofile /usr/share/color/icc/sRGB.icc \ --rotate - + diff --git a/doc/pandoc-docbook-template.docbook b/doc/pandoc-docbook-template.docbook index bef16217..ef420a8d 100644 --- a/doc/pandoc-docbook-template.docbook +++ b/doc/pandoc-docbook-template.docbook @@ -8,17 +8,6 @@ $else$ $endif$ - - $title$ - 3 - libvips - - - - libvips - $title$ - - $for(include-before)$ $include-before$ $endfor$