When building on device (aarch64), the lib directory is set to $PREFIX/lib64 which is wrong and should be overridden.