1. Update all CMakeLists.txt to adapt to new layout
2. Fix cmake build break
3. Update all new file license
4. Fully compatible with current compilation environment(use configure.sh or cmake as you choose)
------------------
How to test
From within nuttx/. Configure:
cmake -B build -DBOARD_CONFIG=sim/nsh -GNinja
cmake -B build -DBOARD_CONFIG=sim:nsh -GNinja
cmake -B build -DBOARD_CONFIG=sabre-6quad/smp -GNinja
cmake -B build -DBOARD_CONFIG=lm3s6965-ek/qemu-flat -GNinja
(or full path in custom board) :
cmake -B build -DBOARD_CONFIG=$PWD/boards/sim/sim/sim/configs/nsh -GNinja
This uses ninja generator (install with sudo apt install ninja-build). To build:
$ cmake --build build
menuconfig:
$ cmake --build build -t menuconfig
--------------------------
2. cmake/build: reformat the cmake style by cmake-format
https://github.com/cheshirekow/cmake_format
$ pip install cmakelang
$ for i in `find -name CMakeLists.txt`;do cmake-format $i -o $i;done
$ for i in `find -name *\.cmake`;do cmake-format $i -o $i;done
Co-authored-by: Matias N <matias@protobits.dev>
Signed-off-by: chao an <anchao@xiaomi.com>
This change allows boards to define an additional kconfig option, which specifies the final link format of application executables.
By selecting `CONFIG_BINFMT_ELF_RELOCATABLE`, and providing an appropriate linker script, applications can be fully linked, removing the need to process relocations.
This is preparation to use kernel stack for everything when the user
process enters the kernel. Now the user stack is in use when the user
process runs a system call, which might not be the safest option.
The following changes make it possible for Nuttx to load binaries in ELF format which are fully linked.
The change does not include the necessary modifications to produce such binaries. In order to build an applicable binary:
- The userspace applications linker script (`gnu-elf.ld`) needs to modified so the data and text section origin's match those setup by the address environment.
- The makefile used, in `apps/import/Make.defs` needs to remove the `-r` LDELFFLAG.
The spawn proxy thread is a special existence in NuttX, usually some developers
spend a lot of time on stack overflow of spawn proxy thread:
https://github.com/apache/nuttx/issues/9046https://github.com/apache/nuttx/pull/9081
In order to avoid similar issues, this PR will remove spawn proxy thread to simplify
the process of task/posix_spawn().
1. Postpone the related processing of spawn file actions until after task_init()
2. Delete the temporary thread of spawn proxy and related global variables
Signed-off-by: chao an <anchao@xiaomi.com>
1. stack_alloc_ptr should be freed in kernel mode
2. replace nxsched_release_tcb() to nxtask_uninit() to remove tcb from inactive list
Signed-off-by: chao an <anchao@xiaomi.com>
Store the old environment in a local context so another temporary address
environment can be selected. This can happen especially when a process
is being loaded (the new process's mappings are temporarily instantiated)
and and interrupt occurs.
Instead of using a volatile storage for the address environment in the
binfmt / loadinfo structures, always allocate the address environment
from kheap.
This serves two purposes:
- If the task creation fails, any kernel thread that depends on the
address environment created during task creation will not lose their
mappings (because they hold a reference to it)
- The current address environment variable (g_addrenv) will NEVER contain
a stale / incorrect value
- Releasing the address environment is simplified as any pointer given
to addrenv_drop() can be assumed to be heap memory
- Makes the kludge function addrenv_clear_current irrelevant, as the
system will NEVER have invalid mappings any more
- Remove the temporary "saved" variable when temporarily changing MMU
mappings to access another process's memory. The fact that it has an
address environment is enough to make the choice
- Restore nxflat_addrenv_restore-macro. It was accidentally lost when
the address environment handling was re-factored.
The function is not relevant any longer, remove it. Also remove
save_addrenv_t, the parameter taken by up_addrenv_restore.
Implement addrenv_select() / addrenv_restore() to handle the temporary
instantiation of address environments, e.g. when a process is being
created.
Detach the address environment handling from the group structure to the
tcb. This is preparation to fix rare cases where the system (MMU) is left
without a valid page directory, e.g. when a process exits.
Same as with group_free(), there is no need to instantiate the address
environment to destroy it.
The only problem was the ARM implementation modified the L1 mappings
in up_addrenv_destroy(), which it no longer does.
- Also remove the nuttx private shm.h file nuttx/mm/shm.h, which became redundant
- Also remove the gran allocator initialization/release in binfmt since common
vpage allocator is initialized in group_create/group_leave
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
The dependency should be vice versa; the MM_SHM should depend on the
existence of the virtual memory range allocator.
Create a new CONFIG flag CONFIG_ARCH_VMA_MAPPING, which will define that
there is a virtual memory range allocator. Make MM_SHM select that flag
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
In the past, header file paths were generated by the incdir command
Now they are generated by concatenating environment variables
In this way, when executing makefile, no shell command will be executed,
it will improve the speed of executing makfile
Signed-off-by: yinshengkai <yinshengkai@xiaomi.com>
1.Don't check the return value of nxsem_init or nxmutex_init
2.Fix some style issue
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Fixes minor issue with a trace (build failure with -Werror):
libelf/libelf_symbols.c:310:41: error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]
310 | (uintptr_t)(sym->st_value + symbol->sym_value));
Summary:
- I noticed that the hello (ELF) application causes a memory leak.
- Finally, I found that the data section is not deallocated.
- This commit fixes this issue.
Impact:
- ELF loader with CONFIG_ARCH_ADDRENV=n
Testing:
- Tested with the following configs
- sprensense:elf, esp32-devkitc:elf, sabre-6quad:elf
- spresense:wifi_smp, rv-virt:nsh64, sabre-6quad:netnsh
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Summary:
- I noticed that the hello (ELF) application causes a memory leak.
- Finally, I found that the data section is not deallocated.
- This commit fixes this issue.
Impact:
- None
Testing:
- Tested with spresense:wifi_smp, rv-virt:nsh64, sabre-6quad:netnsh
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Make use of XCHG/BSWAP on x86, REV16 and REV on ARMv6-m and above,
and whatever other optimized instructions on other platforms.
Defines extra CONFIG variables, and removes the unused functions for
endian-swapping. Fixes some oversights in using the macros.
Implement a generic access rights modification procedure instead
of the procedures that only do one thing (enable/disable write)
to one section (text).
When the .elf file is loaded from disk, the kernel must be given write
access to the allocated .text section in the task's address environment.
The access is removed after the elf is loaded and relocations are done.
NOTE:
The reason this works for the ARM implementation, is that the ARM MMU
can be configured to give write access for the privileged mode, but
revoke write access for the user mode.
Regardless, it would be smart to revoke write access even for the
kernel, when the kernel does not need it. This framework allows doing
that, if someone wishes to take up the task.
Using the Kconfig macro does not work for RISC-V target, as there the
user heap follows .data/.bss and does not obey any Kconfig provided
boundary.
Added stubs for ARM and Z80 also.
This keeps backwards compatibility with apps that do not provide
envp. The old implementation passes NULL always and this change fixes
any regression caused by it.
If address environments are in use, it is not possible to simply
memcpy from from one process to another. The current implementation
of env_dup does precisely this and thus, it fails at once when it is
attempted between two user processes.
The solution is to use the kernel's heap as an intermediate buffer.
This is a simple, effective and common way to do a fork().
Obviously this is not needed for kernel processes.
Summary:
- I noticed that the user heap is not initialized correctly
if BUILD_KERNEL=y and ADDRENV=y
- This commit fixes this issue
Impact:
- None
Testing:
- Tested with sabre-6quad:netknsh (not merged yet)
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Basically, mirror the following two commits from modlib.
It's shame we have two copies of elf loaders.
```
commit 51490bad55
Author: YAMAMOTO Takashi <yamamoto@midokura.com>
Date: Wed Apr 14 17:07:39 2021 +0900
modlib: Implement sh_addralign handling
I've seen a module with 16 bytes .rodata alignment for xmm operations.
It was getting SEGV on sim/Linux because of the alignment issue.
The same module binary seems working fine after applying this patch.
Also, tested on sim/macOS and esp32 on qemu,
using a module with an artificially large alignment. (64 bytes)
```
```
commit 418e11b8b3
Author: YAMAMOTO Takashi <yamamoto@midokura.com>
Date: Thu Apr 15 11:33:48 2021 +0900
modlib: Always use separate allocation for text and data
Pros:
* Reduce code differences
* Smaller allocations for !CONFIG_ARCH_USE_MODULE_TEXT
Cons:
* Likely to use more memory for !CONFIG_ARCH_USE_MODULE_TEXT in total
Tested with:
* sim:module on macOS
* esp32-devkit:nsh + CONFIG_MODULE on qemu
* lm3s6965-ek:qemu-protected + CONFIG_EXAMPLES_SOTEST on qemu
```
since the standard require the caller pass the name explicitly
https://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html:
The argument argv is an array of character pointers to null-terminated strings.
The last member of this array shall be a null pointer and is not counted in argc.
These strings constitute the argument list available to the new process image.
The value in argv[0] should point to a filename that is associated with the
process image being started by the posix_spawn() or posix_spawnp() function.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
since the standard require the caller pass the name explicitly
https://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html:
The argument argv is an array of character pointers to null-terminated strings.
The last member of this array shall be a null pointer and is not counted in argc.
These strings constitute the argument list available to the new process image.
The value in argv[0] should point to a filename that is associated with the
process image being started by the posix_spawn() or posix_spawnp() function.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: Id79ffcc501ae9552dc4e908418ff555f498be7f1
Summary:
- I noticed that up_module_text_free() is not called
if CONFIG_ARCH_USE_MODULE_TEXT=y
- This commit fixes this issue
Impact:
- None
Testing:
- Tested with spresense
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Summary:
- This commit introduces a separate text memory for ELF
- The logic is similar to modlib
Impact:
- None
Testing:
- Tested with spresense:elf
- NOTE: needs separate commits
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
to avoid close the same handle twice because
the caller also call elf_uninit in this case
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: Iddcbae9587d11d3b5b06e876d4a037ac0d11992e
elf_init open a elf file , if both enabled
CONFIG_NSH_BUIlTAPPS and CONFIG_NSH_FILE_APPS ,
elf_init will read elf file in /bin directory, but
that length is zero , elf_read failed return and NOT
close elf fd, so this line MUST be return to errout_with_init
When CONFIG_SYMTAB_ORDEREDBYNAME is selected most code will use
the ordered search function. When it is not selected no code will
use the ordered search function. This change merges the two
functions and varies its behaviour based on the config setting,
such that all callers can simply call the one search function
and get the best behaviour.
An additional configuration option allows leading underscores to
be stripped from symbols being relocated in loaded objects. This
allows toolchains which prefix C symbol with underscores to make
loadable ELF objects.
