Commit Graph

126 Commits

Author SHA1 Message Date
Yaksh Bariya
4c532c0de5
ci: fix "Free Additional disk space" step
Hardcode so tbat it greps built_termux-main_packages.txt instead of
built_packages.txt
2022-04-21 16:59:53 +05:30
Yaksh Bariya
d366fcab14
fixup(ci): Fix duplication of package names in
`built_${repo}_packages.txt`

Caused by 56dbb9cf80
2022-04-21 16:04:16 +05:30
Henrik Grimler
56dbb9cf80
CI: specify distribution in repo.json as well
x11-repo uses distribution x11, and root-repo distribution root.
Store this information in the json file as well, and parse it to set
both REPOSITORY_NAME and REPOSITORY_DISTRIBUTION.

Also remove unnecessary `< repo.json` from jq command, jq accepts the
file as an argument.

This fixes upload issues to x11-repo and root-repo.
2022-04-20 16:37:36 +02:00
Yaksh Bariya
04e7833ede
fixup(ci): do not process '%ci:no-build' on 'workflow_dispatch' 2022-04-19 20:55:54 +05:30
Yaksh Bariya
e7431f5aea
fixup(ci): do not lint all packages when %ci:no-build is added to HEAD commit 2022-04-19 20:51:26 +05:30
Yaksh Bariya
d41ad97af5
ci: fix uploads 2022-04-19 07:19:39 +05:30
Yaksh Bariya
98fff5b520
Fix generation of built_$repo_packages.txt 2022-04-19 07:19:28 +05:30
Yaksh Bariya
7669ea0e3b
ci: fix variables 2022-04-18 15:29:38 +05:30
Yaksh Bariya
1b8ab728e0
ci: fix generate build artifact step 2022-04-18 15:08:31 +05:30
Yaksh Bariya
681d108453
ci: do not use local keyword for declaring variables 2022-04-18 14:57:35 +05:30
Yaksh Bariya
f0ea9b922b
monorepo: apply changes suggested by buttaface 2022-04-18 13:57:19 +05:30
Yaksh Bariya
52c383b4d1
chore: switch to monorepo 2022-04-18 13:57:13 +05:30
Yaksh Bariya
dcaf44f95b
ci: sleep only when aptly_add_to_repo returns with status code 000
It is just a waste of time to sleep for 3 minutes when the packages are
successfully added and aptly returns 200 http code
2022-04-16 14:07:17 +05:30
Yaksh Bariya
cac3799e1d
ci: further split jobs to improve log browsing experience 2022-04-16 14:07:12 +05:30
Yaksh Bariya
c4ab7bcfd0
attempt2: Fix repository upload problems
Earlier in 5b8b15e7aa, I had configured
our upload scripts to treat failure of `aptly_add_to_repo` as failure to publish
packages at all. However it won't work, since even if all debs are
uploaded, and aptly_add_to_repo is called, debs are moved from temporary
directory to the repository. The proxy may drop connection during this
moment due to stale connection. And just after that we call
aptly_delete_dir which will delete the temporary directory. On the
server side, aptly will however continue to move debs from temporary
directory. Repository is published, but since the temporary directory is
removed during the transaction, the transaction never completely
succeeds. So only some of the debs are uploaded practically. This is now
fixed by adding a sleep of 180s (should be more than enough for all
stuff)
2022-04-14 17:26:22 +05:30
Henrik Grimler
369db121c1
CI: packages: fix if statement syntax .. again 2022-04-14 09:06:08 +02:00
Henrik Grimler
1c71b0db0d
CI: packages: fix if statement syntax
The script uses bash, so need a closing fi statement.
2022-04-14 08:42:07 +02:00
Yaksh Bariya
5b8b15e7aa
fixup(ci:upload): Treat failure to add debs to repository as error 2022-04-14 11:45:17 +05:30
Yaksh Bariya
8b8cd21cd9
respect '%ci:no-build' also for PRs
Helpful when a large dependency rebuild is being done and changes are tested locally or on fork.
2022-04-13 17:38:13 +05:30
Yaksh Bariya
50a98af858
CI: Upload build artifacts even on build failure
Helpful when CI times out or building a large number of packages for testing and it fails
2022-03-30 08:52:01 +05:30
Henrik Grimler
e585e63454
CI: upload to packages.termux.org again
The cloudflare variant, packages-cf.termux.org, doesn't like large
uploads (> 100M?).
2022-03-25 17:43:25 +01:00
Henrik Grimler
829368cac9
CI: set GPG_PASSHRASE as well for packages.termux.org uploads 2022-03-13 13:44:05 +01:00
Henrik Grimler
3997a41ff0
CI: remove duplicate info message
It's already printed in aptly_publish_repo.
2022-03-13 13:28:22 +01:00
Henrik Grimler
e7be93609b
CI: extract deb archive only once
No need to extract it again during grimler.se upload.
2022-03-13 13:15:16 +01:00
Henrik Grimler
ccf1b2d111
CI: packages.yml: upload to grimler.se as well 2022-03-13 11:12:42 +01:00
Henrik Grimler
e7f3a436ae
CI: packages.yml: refactor, move aptly api functions to scripts/ 2022-03-13 11:12:39 +01:00
Tee KOBAYASHI
8a5fe78e76 github actions: Properly build custom Docker image
when `setup-android-sdk.sh` is modified.
2022-03-13 08:30:29 +09:00
Henrik Grimler
a429f53d6f
CI: use cloudflare url for uploading packages
Since the fosshost proxy does not work at the moment.
2022-03-07 11:15:48 +01:00
Henrik Grimler
ca5438c1ef
CI: fix occasional dependency upload issue
Previously we were outputting built debs into ./debs instead of
standard folder ./output.  We were also sorting built_packages.txt so
that $(cat ./built_packages.txt) returned an alphabetically sorted
list.

This caused issues for pushes that updated both a package and one or
more of its dependencies, if the dependencies start with a letter that
is later in the alphabet.

Latest example was libforestdb and libsnappy.  Libforestdb depends on
libsnappy, but the buildorder as given to build-package.sh was
"libforestdb libsnappy".  During the libforestdb build libsnappy was
built first, but the deb put into the standard ./output folder (that's
how our build system currently handles built dependency debs).  When
build-package.sh then later tries building libsnappy it skips it
since it has already been built.  We then end up with only libforestdb
in the ./debs folder.

Fix this issue by putting all debs in ./output/, and then moving the
once mentioned in built_packages.txt to ./debs/.

Fixes https://github.com/termux/termux-packages/issues/8773.
2022-01-30 17:06:27 +01:00
Leonid Pliushch
88b2e3efd4
github actions: set --max-time option for curl calls where it can hang for long time 2022-01-15 13:12:39 +02:00
Leonid Pliushch
750a529467
github actions: filter http codes in other places, not only when publishing changes 2021-12-26 22:41:08 +02:00
Leonid Pliushch
d332ec887e
github actions: change regexp for packages
Allow dot in package name.
2021-12-20 17:21:55 +02:00
Leonid Pliushch
7aa03f39d8
github actions: filter http codes when publishing changes, handle 504
Aptly may process request for long time enough to make CloudFlare or
server drop connection. Treat this as false failure and don't stop
GitHub Actions with error status.
2021-12-05 12:03:00 +02:00
Yaksh Bariya
99468e7fa8
Fix typo in packages.yml 2021-11-04 19:59:01 +05:30
Yaksh Bariya
f3eae9f980
CI: Also package debs for subpackages when workflow_dispatch 2021-11-04 19:38:17 +05:30
Leonid Pliushch
3194a85278
github actions: introduce workflow_dispatch trigger for rebuilding packages 2021-10-26 20:19:10 +03:00
Lucinda May Phipps
80a69bf5e0
packages.yml: allow packages with + in name 2021-10-23 13:40:37 +01:00
Leonid Pliushch
8bde94f6a3
github actions: don't use --retry-all-errors curl option as unavailable in this version 2021-10-09 00:24:54 +03:00
Leonid Pliushch
23ba08388b
github actions: get rid of retry loops 2021-10-08 23:48:58 +03:00
Leonid Pliushch
160e810be8
github actions: use cloudflare endpoint in all problematic places
Should finally resolve issues with uploads.
2021-10-08 23:36:57 +03:00
Leonid Pliushch
b8b89471f5
github actions: upload delays 30s, retries 3
Do less retries but with higher delay. Previous config didn't work.
2021-10-08 23:24:30 +03:00
Leonid Pliushch
5272d228e5
github actions: specify own user agent for all curl commands 2021-10-08 14:05:25 +03:00
Leonid Pliushch
df3a651aa8
github actions: package upload delays 10s and 10 retry attempts
Also noticed that planned initial delay of 3s hasn't been implemented at
all.
2021-10-08 14:01:17 +03:00
Leonid Pliushch
938f0e4da6
github actions: don't exit with failure status on publishing
Revert to old behavior: if connection has been dropped, do not exit
with failure status.
2021-10-07 16:43:29 +03:00
Leonid Pliushch
8c7b2298f5
github actions: retry problematic steps up to 5 times 2021-10-07 16:41:21 +03:00
Leonid Pliushch
49e2fe69cd
github actions: fix curl command for publishing step 2021-10-07 15:52:53 +03:00
Leonid Pliushch
9fc7764e09
github actions: mask curl exit code 2021-10-07 15:36:30 +03:00
Leonid Pliushch
c4ac6712bc
github actions: manually handle curl exit code 52 2021-10-07 15:08:36 +03:00
Leonid Pliushch
19acaed7cd
github actions: add time tracking to upload logs 2021-10-07 15:02:11 +03:00
Leonid Pliushch
150479d560
github actions: retry up to 3 times when connection has been dropped 2021-10-07 14:49:01 +03:00