Most tools used for compliance and SBOM generation use SPDX identifiers
This change brings us a step closer to an easy SBOM generation.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
The file descriptors of kernel threads should be clean and should
not be generated based on user threads. Instead, an idle thread
hould be chosen.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
Kthreads can share the group data so that to reduce overheads.
This implements shared kthread group via:
- use `tcb_s` instead of `task_tcb_s` for kthreads
- use `g_kthread_group` when creating kthreads
- use stackargs to start tasks and kthreads
see pull/12320 for test logs.
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
1. add support to join main task
| static pthread_t self;
|
| static void *join_task(void *arg)
| {
| int ret;
| ret = pthread_join(self, NULL); <--- /* Fix Task could not be joined */
| return NULL;
| }
|
| int main(int argc, char *argv[])
| {
| pthread_t thread;
|
| self = pthread_self();
|
| pthread_create(&thread, NULL, join_task, NULL);
| sleep(1);
|
| pthread_exit(NULL);
| return 0;
| }
2. Detach active thread will not alloc for additional join, just update the task flag.
3. Remove the return value waiting lock logic (data_sem),
the return value will be stored in the waiting tcb.
4. Revise the return value of pthread_join(), consistent with linux
e.g:
Joining a detached and canceled thread should return EINVAL, not ESRCH
https://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_join.html
[EINVAL]
The value specified by thread does not refer to a joinable thread.
NOTE:
This PR will not increase stack usage, but struct tcb_s will increase 32 bytes.
Signed-off-by: chao an <anchao@lixiang.com>
The delete flag is not synchronized with the life cycle of the group,
if the flag set before waitpid(), the tcb will be mistakenly deleted
by group_del_waiter(), use-after-free will happen.
Regression by:
| commit 29e50ffa73 (origin/master, origin/HEAD)
| Author: chao an <anchao@lixiang.com>
| Date: Mon Mar 4 09:19:27 2024 +0800
|
| sched/group: move task group into task_tcb_s to improve performance
|
| move task group into task_tcb_s to avoid access allocator to improve performance
|
| for Task Termination, the time consumption will be reduced ~2us (Tricore TC397 300MHZ):
| 15.97(us) -> 13.55(us)
|
| Signed-off-by: chao an <anchao@lixiang.com>
Signed-off-by: chao an <anchao@lixiang.com>
move task group into task_tcb_s to avoid access allocator to improve performance
for Task Termination, the time consumption will be reduced ~2us (Tricore TC397 300MHZ):
15.97(us) -> 13.55(us)
Signed-off-by: chao an <anchao@lixiang.com>
Skip the child wait if here is only self in member list,
since the members of the task group should be 1 if task exit.
Fix Regression issue that the task could not terminate normally.
Signed-off-by: chao an <anchao@lixiang.com>
Change the type of task group member to single list chain to
avoid accessing the memory allocator to improve the performance
Signed-off-by: chao an <anchao@lixiang.com>
The maximum startup parameters have been checked accordingly in nxtask_setup_stackargs(),
let us save argument counter to avoid limit check.
Signed-off-by: chao an <anchao@lixiang.com>
Task group could find from process id, replace group_findbypid to
task_getgroup to simplify the search logic
Signed-off-by: chao an <anchao@lixiang.com>
The task files should consult the "spawn action" and "O_CLOEXEC flags"
to determine further whether the file should be duplicated.
This PR will further optimize file list duplicating to avoid the performance
regression caused by additional file operations.
Signed-off-by: chao an <anchao@xiaomi.com>
This moves task / thread cancel point logic from the NuttX kernel into
libc, while the data needed by the cancel point logic is moved to TLS.
The change is an enabler to move user-space APIs to libc as well, for
a coherent user/kernel separation.
reproduce:
static void *pthread(void *arg)
{
system(arg);
}
void test (int argc, char *argv[])
{
pthread_create(&pthread0, &attr, pthread, argv[1]);
pthread_create(&pthread1, &attr, pthread, argv[2]);
}
only one pthread system() returnd, othres hanged
rootcause:
As we known, system() will create a new task called:
system -c XX
The example:
parent group child groups
pthread0 -> waitpid() -> system -c ps -> exit() -> nxtask_signalparent()
pthread1 -> waitpid() -> system -c ls -> exit() -> nxtask_signalparent()
Each child group exit with function nxtask_signalparent(),
As we expect:
system -c ps will signal pthread0
system -c ls will signal pthread1
But actually:
system -c ps will signal pthread0/1
system -c ls will signal pthread0/1
As the spec, we know, this behavior is normal:
https://man7.org/linux/man-pages/man2/sigwaitinfo.2.html
So for this situation, when the signo is SIGCHLD, we broadcast.
Signed-off-by: ligd <liguiding1@xiaomi.com>
1. Update all CMakeLists.txt to adapt to new layout
2. Fix cmake build break
3. Update all new file license
4. Fully compatible with current compilation environment(use configure.sh or cmake as you choose)
------------------
How to test
From within nuttx/. Configure:
cmake -B build -DBOARD_CONFIG=sim/nsh -GNinja
cmake -B build -DBOARD_CONFIG=sim:nsh -GNinja
cmake -B build -DBOARD_CONFIG=sabre-6quad/smp -GNinja
cmake -B build -DBOARD_CONFIG=lm3s6965-ek/qemu-flat -GNinja
(or full path in custom board) :
cmake -B build -DBOARD_CONFIG=$PWD/boards/sim/sim/sim/configs/nsh -GNinja
This uses ninja generator (install with sudo apt install ninja-build). To build:
$ cmake --build build
menuconfig:
$ cmake --build build -t menuconfig
--------------------------
2. cmake/build: reformat the cmake style by cmake-format
https://github.com/cheshirekow/cmake_format
$ pip install cmakelang
$ for i in `find -name CMakeLists.txt`;do cmake-format $i -o $i;done
$ for i in `find -name *\.cmake`;do cmake-format $i -o $i;done
Co-authored-by: Matias N <matias@protobits.dev>
Signed-off-by: chao an <anchao@xiaomi.com>
Fixed ltp_stress_mqueues_multi_send_rev_1 test issue:
In SMP mode, tg_members will operate on different cores.
Adding interrupt locking operations ensures that the operation
of tg_members will not be interrupted by other cores.
Signed-off-by: zhangyuan21 <zhangyuan21@xiaomi.com>
Store the old environment in a local context so another temporary address
environment can be selected. This can happen especially when a process
is being loaded (the new process's mappings are temporarily instantiated)
and and interrupt occurs.
- Remove the temporary "saved" variable when temporarily changing MMU
mappings to access another process's memory. The fact that it has an
address environment is enough to make the choice
- Restore nxflat_addrenv_restore-macro. It was accidentally lost when
the address environment handling was re-factored.
Do not allow a deferred cancellation if the group is exiting, it is too
dangerous to allow the threads to execute any user space code after the
exit has started.
If the cancelled thread is not inside a cancellation point, just kill it
immediately via asynchronous cancellation. This will create far less
problems than allowing it to continue running user code.
Detach the address environment handling from the group structure to the
tcb. This is preparation to fix rare cases where the system (MMU) is left
without a valid page directory, e.g. when a process exits.
Implement a function for dropping references to the group structure and
finally freeing the allocated memory, if the group has been marked for
destruction
This is just unnecessary, a process cannot be destroyed by another
process in any case, every time this is executed the active address
environment is the process getting destroyed.
Even in the hypothetical case this was possible, the system would
crash at once if a context switch happens between "select()" and
"restore()", which is possible as the granule allocator is protected by
a semaphore (which is a synchronization point).
- Also remove the nuttx private shm.h file nuttx/mm/shm.h, which became redundant
- Also remove the gran allocator initialization/release in binfmt since common
vpage allocator is initialized in group_create/group_leave
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
tg_info is still in use after task_uninit_info(), unifies
lib_stream_* with life cycle of task info to avoid this issue.
| ==1940861==ERROR: AddressSanitizer: heap-use-after-free on address 0xf47032e0 at pc 0x5676dc4f bp 0xf2f38c68 sp 0xf2f38c58
|
|#10 0xf7abec89 in __asan::__asan_report_load2 (addr=4100993760) at ../../../../src/libsanitizer/asan/asan_rtl.cpp:119
|#11 0x5677356a in nxsem_destroy (sem=0xf47032e0) at semaphore/sem_destroy.c:73
|#12 0x56773695 in sem_destroy (sem=0xf47032e0) at semaphore/sem_destroy.c:120
|#13 0x5676faa2 in nxmutex_destroy (mutex=0xf47032e0) at include/nuttx/mutex.h:126
|#14 0x567a3430 in lib_stream_release (group=0xf4901ba0) at stdio/lib_libstream.c:98
|#15 0x5676da75 in group_release (group=0xf4901ba0) at group/group_leave.c:162
|#16 0x5676e51c in group_leave (tcb=0xf5377740) at group/group_leave.c:360
|#17 0x569fe79b in nxtask_exithook (tcb=0xf5377740, status=0) at task/task_exithook.c:455
|#18 0x569f90b9 in _exit (status=0) at task/exit.c:82
|#19 0x56742680 in exit (status=0) at stdlib/lib_exit.c:61
|#20 0x56a69c78 in iperf_showusage (progname=0xf2f28838 "iperf", exitcode=0) at iperf_main.c:91
|#21 0x56a6a6ec in iperf_main (argc=1, argv=0xf2f28830) at iperf_main.c:140
|#22 0x5679c148 in nxtask_startup (entrypt=0x56a69c78 <iperf_main>, argc=1, argv=0xf2f28830) at sched/task_startup.c:70
|#23 0x56767f58 in nxtask_start () at task/task_start.c:134
Signed-off-by: chao an <anchao@xiaomi.com>
The task_group specific list can be used to store information about
mmappings.
For a driver or filesystem performing mmap can also enable munmap by
adding an item to this list using mm_map_add(). The item is then
returned in the corresponding munmap call.
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
Summary:
- I noticed that nxplayer (HTTP audio streaming) + command execution
via telnet sometimes causes memory corruption.
See https://github.com/apache/nuttx/pull/7947 for the detail.
- This commit fixes this issue by calling lib_stream_release() before
lib_stream_release() in group_leave.c
Impact:
- Should be none
Testing:
- Tested with spresense:wifi_smp
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
This is preparation for flushing streams from user space, like it should
be done.
- Move tg_streamlist (group, kernel space) ->
ta_streamlist (TLS, user space)
- Access stream list via tg_info in kernel
- Access stream list via TLS in user space
- Remove / rename nxsched_get_streams -> lib_getstreams
- Remove system call for nxsched_get_streams
It takes about 10 cycles to obtain the task list according to the task
status. In most cases, we know the task status, so we can directly
delete the task from the specified task list to reduce time consuming.
