First step of making possibility to have offline snapshots of
'termux-packages' repo. Useful for making full repo snapshots.
That will allow to preinstall tools like Cmake, Go, Meson to
directory stored within packages tree and will not be wiped when
using ./clean.sh.
Next steps will also implement offlining of the package sources
and NDK standalone toolchain.
Release or Release.gpg files may be absent if a new .deb file was uploaded
to apt repository recently. We may need to do a multiple attempts in a loop
to retry downloading of these files.
Needed after 99e9ab67b634a89d0be9eca12c4f1cdc091ee6e8, where I have disabled
looping in termux_download but curl doesn't do retries on 404 error. We
want immediate failure only for 404's on source and build tool URLs.
As termux_download.sh retries only on transient errors now, we need a
loop when downloading Release{,.gpg} as they may be temporarily unavailable
when metadata is being rebuilt in repository.
For for example libandroid-support we now have
TERMUX_PKG_SRCDIR/License.txt
and
TERMUX_PKG_SRCDIR/wcwidth-$version/License.txt.
Previously the latter would have overwritten the first one in the
install step as the file names are the same.
When no source URL defined, the following error occurs:
/home/builder/termux-packages/scripts/build/get_source/termux_step_get_source.sh: line 2: TERMUX_PKG_SRCURL: unbound variable
Calls termux_git_clone_src if TERMUX_PKG_SRCURL ends with .git, and
termux_download_src_archive and termux_extract_src_archive otherwise.
termux_step_extract_package has been split up into the latter two
functions.
termux_step_post_extract_package has been renamed to
termux_step_post_get_source to reflect these changes.
* termux-licenses: remove licenses that contain copyright information
These licenses cannot be taken from the common termux-licenses package as the copyright information needs to be distributed with the package as well.
271 packages in the main repository needs to be rebuilt after this change.
* termux_step_install_license: remove duplicated comment
* Bump packages that has a license with copyright information
We need to include this copyright information in the package so need
to rebuild them now that termux_step_install_license has been
modified.
The affected licenses are: MIT, ISC, PythonPL, Openfont-1.1, ZLIB,
Libpng, BSD, BSD 2-Clause, BSD 3-Clause.
* zlib: extract license file from zlib.h
* xorgproto: include all individual license files in package
* libgcrypt: include LICENSE file in package
It contains some copyright information so needs to be included in package
* libcrypt: add a dummy BSD 2-Clause license
According to homepage it is suppose to be BSD 2-Clause, but original
project did not include a license file
* libandroid-spawn: add LICENSE file to package
* libandroid-glob: add license file to package
* licenses: allow for comma separated TERMUX_PKG_LICENSE_FILE
%ci:no-build
* libicu: fix so that termux_step_install_license finds license file
%ci:no-build
* termux_step_install_license: check for license file as "License"
%ci:no-build
* libtiff: specify TERMUX_PKG_LICENSE_FILE
%ci:no-build
* asciidoctor: install license to standard location
* krb5: specify license file
* libdb: specify path to license file
%ci:no-build
* libprotobuf: specify path to license file
* darkhttpd: extract license file from darkhttpd.c
* w3m: ensure host libraries are not picked up
* w3m: extract LICENSE file from dock/README
* liblua{,52}: extract license files from doc/readme.html
* termux_step_install_license: look for License.txt as well
* dart: specify path to LICENSE file
* nim: specify license file
* runit: specify license file
* termux_step_install_license: look for LICENSE.TXT as well
* ossp-uuid: extract LICENSE from README
* pigz: extract LICENSE from pigz.c
* ttyrec: extract LICENSE from ttyrec.c
%ci:no-build
* bmon: specify license files
Most of the source files are licensed under MIT, it is not clear (to me) which
parts that are licensed under BSD 2-Clause.
%ci:no-build
* wordgrinder: specify license files
* fdupes: add LICENSE
* autossh: add LICENSE patch
* cmake: specify LICENSE file
* dos2unix: specify LICENSE file
* gnuplot: specify LICENSE file
* termux_step_install_license: look for variants of "copyright" also
* picolisp: specify LICENSE file
* lua-lpeg: extract license file
* tidy: specify license file
* gflags: specify LICENSE file
* timewarrior: fix src url
It seems to have been changed.
* postgresql: specify license file
* ttyrec: fix license patch header
* fossil: specify license file
* fmt: specify license file
* i2pd: specify license file
* leptonica: specify license file
* imgflo: add license file patch
* ired: add license file from ired github repo
* libzen: specify license file
* libmediainfo: specify license file
* mediainfo: specify license file
* tty-clock: add license patch
* samefile: add license patch
* quickjs: add license patch
* docbook-xml: add LICENSE patch
* docbook-xsl: specify LICENSE files
* gnuplot: configure with --without-latex
* timewarrior: add libshared to SRCURL as well
The non-release archives does not contain gitsubmodules so we need to
download it manually.
* restic: switch back to using 0.9.6 archive
* dart: revert revision bump
dart's SDK does not fully support python3 yet, and trying to solve
that is not entirely trivial. We save the dart bump (and upgrade to
2.8.4) for another day.
%ci:no-build
Specifying Termux org members is more correct as we don't have repo-specific
maintainers for termux-packages.
DEB control field specification doesn't allow to specify multiple people as
package maintainers. Value "Termux members @termux" will be used as default
for that field which can be overridden on per-package basis if needed.
%ci:no-build
This new variable is extremely useful when iterating on creating a large package,
as otherwise you have to wipe the source and rebuild each time you make a mistake
with the patches or build.sh script.
Simply set TERMUX_PKG_QUICK_REBUILD=true in build.sh if a build fails and then the
TERMUX_PKG_SRCDIR and TERMUX_PKG_BUILDDIR will not be touched when you rebuild,
including that the patches will not be applied again. When you're done iterating,
diff for any new patches, save them, and remove this variable before rebuilding
from scratch, hopefully for the last time. ;)
An example is shown for the giant libllvm package, where other modifications are
also excluded if this variable is set.
Unlike the NDK clang, the on-device clang has already been patched to add an rpath
to the Termux-prefixed library path, so only have the build script add the rpath
flag for NDK builds.
termux_step_install_service_scripts is run after
termux_step_post_make_install and loops over the new array
TERMUX_PKG_SERVICE_SCRIPT to add service scripts for termux-services.
The service scripts are usually only a one-liner so we might just as
well define it in a variable like TERMUX_PKG_SERVICE_SCRIPT.
TERMUX_PKG_SERVICE_SCRIPT should be an array on the format
("daemon-name" 'script to execute' "another daemon" 'multi\n line\n script'),
i.e. it should be of even length with name + script where the script
part preferably is within single quotes (to avoid accidental expansion
of for example $HOME).
By defining __TERMUX__ and __TERMUX_PREFIX__ in <sys/cdefs.h> (which is
basically always included) one can more easily target Termux
specifically, both when cross-compiling packages and when code is built
on-device.
posix_spawn is not available on Android devices.
While this commit is not necessary for cross-compiling, it is required
when building on-device since there posix_spawn is found somehow. This
produces following errors:
./spawn.h:514:17: error: use of undeclared identifier 'POSIX_SPAWN_RESETIDS'
[(((POSIX_SPAWN_RESETIDS | POSIX_SPAWN_SETPGROUP
Variables
TERMUX_PKG_PLATFORM_INDEPENDENT
TERMUX_DEBUG
TERMUX_PKG_HAS_DEBUG
TERMUX_PKG_ESSENTIAL
TERMUX_SUBPKG_ESSENTIAL
TERMUX_PKG_NO_STATICSPLIT
TERMUX_PKG_BUILD_IN_SRC
TERMUX_PKG_FORCE_CMAKE
TERMUX_PKG_HOSTBUILD
should not accept arbitrary values for marking them "enabled". Instead
they should accept boolean values which makes them easier to handle and
also makes their meaning clear.
build-package.sh should make decision based on variable's value but not on
whether it is set or empty.
%ci:no-build
Partial compatibility for on-device builds.
There is no guarantee that it will be possible to build all available
packages and built packages will have same reliability that cross-compiled
but should solve "self-hosting" problems as much as possible.
Do not use pipes for building & uploading packages because:
* They spawn more jobs on Bintray for metadata generation - each upload is
done in a separate session.
* Bintray doesn't handle well concurrent uploads and metadata can be corrupted.
This happened in all recent builds of platform independent packages - in
each upload task file had a same name and that triggered problem with metadata.
* Using delays for workaround for a problem described above works, but isn't
safe.
CI periodically shows error like
Failed to decrypt some environment variables
so it will be better to stop immediately when environment is inconsistent
for unknown reason (e.g. due to Github's problems).
All platform independent packages affected by this issue. Results in
persistent apt error like:
Writing more data than expected (22610650 > 22610408)
Hashes of expected file:
- SHA256:08abc5cffb9e1967fab487a471bb547f5ae706c48bf8e59d1443b2ad4af9caaf
- SHA1:36ded1e207135bdeb8b7301bc03309ebced3cd32 [weak]
- Filesize:22610408 [weak]
In certain cases Bintray may return errors like `invalid version a.b.c` for
specific tasks. This probably happens because we now running package uploading in
parallel.
Retrying upload one more time should fix the problem.
* Move package building/uploading implementation from config to a separate script.
* Implement new tag '%ci:no-build' to allow immediately stop build with status 'passed'.
%ci:no-build
