libvips/doc/extending.xml

<?xml version="1.0"?>
<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
               "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
]>
<refentry id="extending">
  <refmeta>
    <refentrytitle>Extending VIPS</refentrytitle>
    <manvolnum>3</manvolnum>
    <refmiscinfo>VIPS Library</refmiscinfo>
  </refmeta>

  <refnamediv>
    <refname>Extending</refname>
    <refpurpose>How to add operations to VIPS</refpurpose>
  </refnamediv>

  <refsect3 id="extending-intro">
    <title>Introduction</title>
    <para>
			This section runs quickly through adding a simple operator to VIPS. 
			For more information, see #VipsOperation and #VipsRegion. A good 
			starting point for a new operation is a similar one in the VIPS library. 
    </para>

    <para>
      All VIPS operations are subclasses of #VipsOperation, which in turn
			subclasses #VipsObject and then %GObject. You add an operation to VIPS
			by defining a new subclass of #VipsOperation and arranging for its 
			<code>class_init()</code> to be called, perhaps by calling its get_type()
			function.
    </para>
  </refsect3>

  <refsect3 id="extending-classstruct">
    <title>The class and object structures</title>

    <para>
      First you need to define a new
      object struct and a new class struct.

<programlisting language="C">
typedef struct _Negative {
  VipsOperation parent_instance;

  VipsImage *in;
  VipsImage *out;

  int image_max;

} Negative;

typedef struct _NegativeClass {
  VipsOperationClass parent_class;

  /* No new class members needed for this op.
   */

} NegativeClass;
</programlisting>
    </para>

    <para>
      This operation will find the photographic negative of an unsigned
      8-bit image, optionally letting you specify the value which the pixels
      "pivot" about. It doesn't need any class members (ie. values common
      to all operations of this type), so the second struct is empty. See
			the source to vips_invert() for a more complete version of this 
			operation that's actually in the library.
    </para>

    <para>
      %GObject has a handy macro to write some of the boilerplate for you.

<programlisting language="C">
G_DEFINE_TYPE( Negative, negative, VIPS_TYPE_OPERATION );
</programlisting>

      G_DEFINE_TYPE() defines a function called negative_get_type(),
      which registers this new class and returns its %GType (a
      pointer-sized integer). negative_get_type() in turn needs two
      functions, negative_init(), to initialise a new instance, and
      negative_class_init(), to initialise a new class.
    </para>
  </refsect3>

  <refsect3 id="extending-init">
    <title>Class and object initialisation</title>
    <para>
      negative_init() is very simple, it just sets the default value for
      our optional parameter.

<programlisting language="C">
static void
negative_init( Negative *negative )
{
  negative->image_max = 255;
}
</programlisting>
    </para>

    <para>
      negative_class_init() is more complicated: it has to set various
      fields in various superclasses and define the operation's parameters. 

<programlisting language="C">
static void
negative_class_init( NegativeClass *class )
{
  GObjectClass *gobject_class = G_OBJECT_CLASS( class );
  VipsObjectClass *object_class = VIPS_OBJECT_CLASS( class );

  gobject_class->set_property = vips_object_set_property;
  gobject_class->get_property = vips_object_get_property;

  object_class->nickname = "negative";
  object_class->description = "photographic negative";
  object_class->build = negative_build;

  VIPS_ARG_IMAGE( class, "in", 1, 
    "Input", 
    "Input image",
    VIPS_ARGUMENT_REQUIRED_INPUT,
    G_STRUCT_OFFSET( Negative, in ) );

  VIPS_ARG_IMAGE( class, "out", 2, 
    "Output", 
    "Output image",
    VIPS_ARGUMENT_REQUIRED_OUTPUT, 
    G_STRUCT_OFFSET( Negative, out ) );

  VIPS_ARG_INT( class, "image_max", 4, 
    "Image maximum", 
    "Maximum value in image: pivot about this",
    VIPS_ARGUMENT_OPTIONAL_INPUT,
    G_STRUCT_OFFSET( Negative, image_max ),
    0, 255, 255 );
}
</programlisting>
    </para>

    <para>
      In %GObject, it needs to set the getters and setters for this class. vips
      has a generic get/set system, so any subclass of #VipsObject needs to
      use the vips ones.
    </para>

    <para>
      In #VipsObject, it needs to set the operation @nickname and @description,
      and set a build function (see below). @nickname is used to refer to
      this operation in the API, @description is used to explain this
      operation to users and will be translated into their language.
    </para>

    <para>
      Finally, it needs to define the arguments the constructor for this class
			takes. There are a set of handy macros for doing this, see 
			VIPS_ARG_INT() and friends.
    </para>

    <para>
			The first few
      parameters are always the same and mean: class pointer for argument,
      argument name, argument priority (bindings expect required arguments in
      order of priority), long argument name (this one is internationalised
      and displayed to users), description (again, users can see this),
      some flags describing the argument, and finally the position of the
      member in the struct.
    </para>

    <para>
      Integer arguments take three more values: the minimum, maximum and
      default value for the argument.
    </para>
  </refsect3>

  <refsect3 id="extending-build">
    <title>The build() function</title>
    <para>
			The build function is the thing #VipsObject calls during object 
			construction, after all arguments have been supplied and before the 
			object is used. It has two roles: to verify that arguments are correct, 
			and then to construct the object.  After build(), the object is expected
      to be ready for use.

<programlisting language="C">
static int
negative_build( VipsObject *object )
{
  VipsObjectClass *class = VIPS_OBJECT_GET_CLASS( object );
  Negative *negative = (Negative *) object;

  if( VIPS_OBJECT_CLASS( negative_parent_class )->build( object ) )
    return( -1 );

  if( vips_check_uncoded( class->nickname, negative->in ) ||
    vips_check_format( class->nickname, negative->in, VIPS_FORMAT_UCHAR ) )
    return( -1 );

  g_object_set( object, "out", vips_image_new(), NULL ); 

  if( vips_image_pipelinev( negative->out, 
    VIPS_DEMAND_STYLE_THINSTRIP, negative->in, NULL ) )
    return( -1 );

  if( vips_image_generate( negative->out, 
    vips_start_one, 
    negative_generate, 
    vips_stop_one, 
    negative->in, negative ) )
    return( -1 );

  return( 0 );
}
</programlisting>
    </para>

    <para>
      negative_build() first chains up to the superclass: this will check
      that all input arguments have been supplied and are sane.
    </para>

    <para>
      Next, it adds its own checks. This is a demo operation, so we just
			work for uncoded, unsigned 8-bit images. There are a lot of 
			convenience functions like vips_check_format(), see the docs. 
    </para>

    <para>
      Next, it creates the output image. This needs to be set with
      g_object_set() so that vips can see that it has been assigned. vips
      will also handle the reference counting for you.
    </para>

    <para>
      vips_image_pipelinev() links our new image onto the input image and
      notes that this operation prefers to work in lines. You can request 
      other input geometries, see #VipsDemandStyle. 
    </para>

    <para>
			The geometry hint is just a hint, an operation needs to be able to 
			supply any size 
      #VipsRegion on request. If you must have a certain size request, you can 
      put a cache in the pipeline after your operation, see vips_linecache() 
			and vips_tilecache(). You can also make requests to your operation 
			ordered, see vips_sequential(). 
    </para>

    <para>
      Finally, vips_image_generate() attaches a set of callbacks to the
      output image to generate chunks of it on request. vips_start_one()
      and vips_stop_one() are convenience functions that make the input
      region for you, see below. 
    </para>
  </refsect3>

  <refsect3 id="extending-gen">
    <title>The generate() function</title>

    <para>
			The generate() function does the actual image processing. 
			negative_generate() (of type #VipsGenerateFn, supplied to 
			vips_image_generate() above) is 
			called whenever some pixels of our output image are required. 

<programlisting language="C">
static int
negative_generate( VipsRegion *or, 
  void *vseq, void *a, void *b, gboolean *stop )
{
  /* The area of the output region we have been asked to make.
   */
  VipsRect *r = &amp;or-&gt;valid;

  /* The sequence value ... the thing returned by vips_start_one().
   */
  VipsRegion *ir = (VipsRegion *) vseq;

  VipsImage *in = (VipsImage *) a;
  Negative *negative = (Negative *) b;
  int line_size = r-&gt;width * negative-&gt;in-&gt;Bands; 

  int x, y;

  /* Request matching part of input region.
   */
  if( vips_region_prepare( ir, r ) )
    return( -1 );

  for( y = 0; y &lt; r-&gt;height; y++ ) {
    unsigned char *p = (unsigned char *)
      VIPS_REGION_ADDR( ir, r-&gt;left, r-&gt;top + y ); 
    unsigned char *q = (unsigned char *)
      VIPS_REGION_ADDR( or, r-&gt;left, r-&gt;top + y ); 

    for( x = 0; x &lt; line_size; x++ ) 
      q[x] = negative-&gt;image_max - p[x];
  }

  return( 0 );
}
</programlisting>
    </para>

    <para>
      This has to calculate a section of the output image. The output
      #VipsRegion, @or, contains a #VipsRect called @valid which is the
			area needing calculation. This call to negative_generate() must 
			somehow make this part of @or contain pixel data. 
    </para>

    <para>
			@vseq is the sequence value. This is the 
			per-thread state for this generate, created (in this example) by 
			vips_start_one(). In this simple case it's just a #VipsRegion defined on 
			the input image. If you need more per-thread state you can write your 
			own start and stop functions and have a struct you create and pass as a 
			sequence value. There are plenty of examples in the VIPS source code, 
			see vips_rank(). 
    </para>

    <para>
			@a and @b are the last two arguments to vips_image_generate() above. 
			@stop is a bool pointer you can set to stop computation early. vips_min()
			on an unsigned int image, for example, will set @stop as soon as it sees 
			a zero, and will not scan the entire image.
    </para>

    <para>
			The first thing negative_generate() does is 
			use vips_region_prepare() to 
			ask for the corresponding pixels from the input image. Operations which 
			do coordinate transforms or which need an area of input for each output 
			point will need to calculate a new rect before calling 
			vips_region_prepare(). 
    </para>

    <para>
			Finally, it can calculate some pixels. negative_generate() loops
			over the valid area of the output and calls VIPS_REGION_ADDR() for each 
			line. This macro is reasonaly quick, but it's best not to call it for 
			each pixel. Once per line is fine though. 
    </para>
  </refsect3>

  <refsect3 id="extending-add">
    <title>Adding to VIPS</title>
    <para>
			To add the operation to vips, just call negative_get_type(). You can 
			include the source in your program, or use %GModule to make a binary 
			plugin that will be loaded by libvips at startup. There are some <ulink 
				role="online-location"
				url="https://github.com/libvips/vips-gmic">example
			plugins available</ulink>.
    </para>

    <para>
			You
      can then use @negative from any of the vips interfaces. For example,
      in Python you'd use it like this:

<programlisting language="python">
out = in.negative(image_max = 128)
</programlisting>
    </para>

    <para>
      From the command-line it'd look like this:

<programlisting language="bash">
$ vips negative in.png out.tif --image-max 128
</programlisting>
    </para>

    <para>
      And from C like this:

<programlisting language="C">
VipsImage *in;
VipsImage *out;
if( vips_call( "negative", in, &amp;out, "image_max", 128, NULL ) )
  ... error
</programlisting>
    </para>

    <para>
      Unfortunately that will do almost no compile-time type checking,
      so all vips operations have a tiny extra wrapper to add a bit of
      safety. For example:

<programlisting language="C">
static int 
negative( VipsImage *in, VipsImage **out, ... )
{
  va_list ap;
  int result;

  va_start( ap, out );
  result = vips_call_split( "negative", ap, in, out );
  va_end( ap );

  return( result );
}
</programlisting>
    </para>

    <para>
      And now you can write:

<programlisting language="C">
if( negative( in, &amp;out, "image_max", 128, NULL ) )
  ... error
</programlisting>

      and it's at least a bit safer.
    </para>
  </refsect3>

  <refsect3 id="extending-othertypes">
    <title>Other types of operation</title>
    <para>
      Change the _build() function to make other types of operation. 
    </para>
      
    <para>
      Use vips_image_generate() with vips_start_many() to make operations 
      which demand pixels from more than one image at once, such as image 
      plus image. 
    </para>
      
    <para>
      Use vips_sink() instead of vips_image_generate() to loop over an image 
      and calculate a value. vips uses this for the statistics operations, 
      like vips_avg().
    </para>

    <para>
      Use vips_image_wio_input() to get an entire image into memory so you
      can read it with a pointer. This will obviously not scale well to
      very large images, but some operations, like FFTs or flood-fill, need 
      the whole image to be available at once.
    </para>

    <para>
      Make area operations, like filters, by enlarging the #VipsRect that
      _generate() is given before calling vips_region_prepare(). You can
      enlarge the input image, so that the output image is the same size as
      the original input, by using vips_embed() within the _build() function.
    </para>

    <para>
      Make things like flips and rotates by making larger changes to the
      #VipsRect in _generate().
    </para>

    <para>
      Make zero-copy operations, like vips_insert(), with vips_region_region().
    </para>

  </refsect3>

</refentry>