You can use <literal>pyvips.Image.new_from_memory()</literal> to make a vips image from an area of memory. The memory array needs to be laid out band-interleaved, as a set of scanlines, with no padding between lines.
This makes a 100,000 x 100,000 black image, then inserts all the images you pass on the command-line into it at random positions. libvips is able to run this program in sequential mode: it’ll open all the input images at the same time, and stream pixels from them as it needs them to generate the output.
</para>
<para>
To test it, first make a large 1-bit image. This command will take the green channel and write as a 1-bit fax image. <literal>wtc.jpg</literal> is a test 10,000 x 10,000 jpeg:
Now make 1,000 copies of that image in a subdirectory:
</para>
<programlisting>
$ mkdir test
$ for i in {1..1000}; do cp x.tif test/$i.tif; done
</programlisting>
<para>
And run this Python program on them:
</para>
<programlisting>
$ time ./try255.py x.tif[squash,compression=ccittfax4,strip,bigtif] test/*
real 1m59.924s
user 4m5.388s
sys 0m8.936s
</programlisting>
<para>
It completes in just under two minutes on this laptop, and needs about 7gb of RAM to run. It would need about the same amount of memory for a full-colour RGB image, I was just keen to keep disc usage down.
</para>
<para>
If you wanted to handle transparency, or if you wanted mixed CMYK and RGB images, you’d need to do some more work to convert them all into the same colourspace before inserting them.