Resolution of Issue 619 will require multiple steps, this part of the first step in that resolution: Every call to nxsem_wait_uninterruptible() must handle the return value from nxsem_wait_uninterruptible properly. This commit is only for those files under fs/tmpfs and fs/spiffs. Still do do: The rest of fs/ and all of drivers/ and arch/.
Resolution of Issue 619 will require multiple steps, this part of the first step in that resolution: Every call to nxsem_wait_uninterruptible() must handle the return value from nxsem_wait_uninterruptible properly. This commit is only for those files under graphics/, mm/, net/, sched/, wireless/bluetooth.
Still to do: Files under fs/, drivers/, and arch. The last is 116 files and will take some effort.
This reverts commit b9ace36fcc.
This change was added by PR 625 but has a serious logic flaw. It removes all occurrences of INCDIROPT and replaces it with a definition in tools/Config.mk:
else ifeq ($(WINTOOL),y)
DEFINE = "$(TOPDIR)/tools/define.sh"
INCDIR = "$(TOPDIR)/tools/incdir.sh" -w
This logic flaw is the Config.mk is included in all Make.defs files BEFORE WINTOOL is defined. As a result, the definition is wrong in many places when building under Cygwin with a Windows native toolchain.
Eliminate the MOVEOBJ definition from tools/Config.mk, Makefiles, and other locations. The MOVEOBJ definition was used to work around the case where the object output of a compile or assemble is in a different directory than the source file. The ZDS-II tools have no command line option to perform this move; MOVEOBJ handled the move as a post-compiler operation.
MOVEOBJ is no longer needed because this the ez80 COMPILE and ASSEMBLE definitions now handle all of the object file movement cases.
It seems that "ld -r" on macOS doesn't include objects from
libraries for common symbols. Because of that, sim build
ends up with undefined references to globals like g_binfmts
and g_mmheap.
@(#)PROGRAM:ld PROJECT:ld64-530
BUILD 18:57:17 Dec 13 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
mm/Makefile: Same change should be applied to mm/Makefile since the mm/ build is identical to the libc/ build.
libs/libnx/Makefile: Same change should be applied to libnx/Makefile since the mm/ build is identical to the libc/ build.
Author: chao.an <anchao@xiaomi.com>
libs/libc/Makefile: Correct the dependents path
Here is the email loop talk about why it is better to remove the option:
https://groups.google.com/forum/#!topic/nuttx/AaNkS7oU6R0
Change-Id: Ib66c037752149ad4b2787ef447f966c77aa12aad
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
* Simplify EINTR/ECANCEL error handling
1. Add semaphore uninterruptible wait function
2 .Replace semaphore wait loop with a single uninterruptible wait
3. Replace all sem_xxx to nxsem_xxx
* Unify the void cast usage
1. Remove void cast for function because many place ignore the returned value witout cast
2. Replace void cast for variable with UNUSED macro
Iobinstrumentation
* mm/iob: Introduces producer/consumer id to every iob call. This is so that the calls can be instrumented to monitor the IOB resources.
* iob instrumentation - Merges producer/consumer enumeration for simpler IOB user.
* fs/procfs: Starts adding support for /proc/iobinfo
* fs/procfs: Finishes first pass of simple IOB user stastics and /proc/iobinfo entry
Approved-by: Gregory Nutt <gnutt@nuttx.org>
arch/mips/src: Previous commit used CP0 register definitions that were not defined in the cp0.h header file. Probably these were from the Microchip hacked up GCC toolchain but are not generally available. Fix: Add definitions to NuttX cp0.h header file.
mm/iob: Eliminate some warnings about testing the value of an undefined pre-processor variable.
- Logic that generates the low lever framebuffer renderers for the per-window framebuffers,
- The logic that picks off the per-window framebuffer updates from normal graphics device updates. This logic must update both the per-window framebuffer and the graphics device (from the framebuffer).
Squashed commit of the following:
graphics: Add logic to allocate the per-window framebuffer.
graphics: A few fragmentary thoughts on how a per-window framebuffer could be represented and allocated.
sched/init/nx_bringup.c: Fix a naming collision.
sched/init: Rename os_start() to nx_start()
sched/init: Rename os_smp* to nx_smp*
sched/init: Rename os_bringup to nx_bringup
sched/init: rename all internal static functions to begin with nx_ vs os_
This commit repartitions the logic by moving some of the changes from mm_sem.c into task_getpid.c. The logic is equivalent for the case of mm_trysemaphore(), but no has wider impact since it potentially affects all callers of getpid(). Hence, this change may also introduce some other issues that will need to be addressed.
This change adds a check to mm_trysemaphore() (the root implementation of both kmm_trysemaphore() and umm_trysemaphore()). It checks if the that task that is apparently executing is marked as RUNNING. If not, how could the non-running task be trying to get the MM semaphore? I think only in the exact scenario that Eunbong Song has described.
So I think the solution should provide the same protection as 91aa26774b but without the horrific consequences to memory usage.
Pullreq libc libnx updates
* NuttX: make strerror() return 'Success' for 0
* NuttX: fix strrchr() so that it considers null terminator as part of string
From strrchr(3) man page:
"The terminating null byte is considered part of the string, so that if c
is specified as '\0', these functions return a pointer to the terminator."
* NuttX: mm_free(): Add DEBUGASSERT()'s to catch memory corruption early.
It's easier to find the source when asserts fail already when freeing
an overflowed buffer, than if the corruption is only detected on next
malloc().
* MM_FILL_ALLOCATIONS: Add debug option to fill all mallocs()
This is helpful for detecting uninitialized variables,
especially in C++ code. I seem to be forgetting to initialize
member variables and then they just get random values..
* NuttX: nxtk_bitmapwindow: Fix warning message when bitmap is fully off-screen.
* nxfonts_getfont: Avoid unnecessary warnings for other whitespace chars also.
* NuttX: Fix kerning of 'I' in Sans17x22 font
The I character was running together with some other
characters, e.g. in sequence "IMI".
* NXMU: Revalidate window pointer for mouse events.
NXMU caches the previous window pointer so that further mouse
events can be sent to the same window. However, if the window
is destroyed while mouse button is held down, the pointer may
become invalid and cause a crash. This patch revalidates the
pointer before using it.
Approved-by: GregoryN <gnutt@nuttx.org>
Squashed commit of the following:
Fix up some final compile isses.
net/netdev: Convert the network down notification logic to use the new wqueue-based notification factility.
net/udp: Convert the UDP readahead notification logic to use the new wqueue-based notification factility.
net/tcp: Convert the TCP readahead notification logic to use the new wqueue-based notification factility.
mm/iob: Convert the IOB notification logic to use the new wqueue-based notification factility.
sched/wqueue: Signals are not good IPCs to support the target poll functionality for several reasons including the amount of data that can be passed with a signal and in the fact that in protected and kernel modes, user threads executing signal handlers in protected, kernel memory is problematic. Instead, convert the same logic to perform the notifications via function callback on the high priority work queue.
Squashed commit of the following:
net/tcp: Add signal notification for the case when UDP read-ahead data is buffered. This is basically of clone of the TCP notification logic with naming adapted for UDP.
net/tcp: Add signal notification for the case when TCP read-ahead data is buffered.
Squashed commit of the following:
mm/iob: The IOB available notifier is now just a wrapper around the common signal notifier.
sched/signal: Add a generic signal notification facility.
sched/signal/sig_evthread.c: More trivial naming changes.
sched/signal: Rename nxsig_notification() to nxsig_evthread() to make forthcoming naming additions more consistent.
Fix SMP related bugs
* sched/sched: Fix a deadlock in SMP mode
Two months ago, I introduced sched_tasklist_lock() and
sched_tasklist_unlock() to protect tasklists in SMP mode.
Actually, this change works pretty well for HTTP audio
streaming aging test with lc823450-xgevk.
However, I found a deadlock in the scheduler when I tried
similar aging tests with DVFS autonomous mode where CPU
clock speed changed based on cpu load. In this case, call
sequences were as follows;
cpu1: sched_unlock()->sched_mergepending()->sched_addreadytorun()->up_cpu_pause()
cpu0: sched_lock()->sched_mergepending()
To avoid this deadlock, I added sched_tasklist_unlock() when calling
up_cpu_pause() and sched_addreadytorun(). Also, added
sched_tasklist_lock() after the call.
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
* libc: Add critical section in lib_filesem.c for SMP
To set my_pid into fs_folder atomically in SMP mode,
critical section API must be used.
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
* mm: Add critical section in mm_sem.c for SMP
To set my_pid into mm_folder atomically in SMP mode,
critical section API must be used.
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
* net: Add critical section in net_lock.c for SMP
To set my pid (me) into fs_folder atomically in SMP mode,
critical section API must be used.
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Approved-by: Gregory Nutt <gnutt@nuttx.org>
binfmt/, configs/, grahics/, libc/, mm/, net/, sched/: OS references to the errno variable should always use the set_errno(), get_errno() macros
arch/arm/src/stm32 and stm32f7: Architecture-specific code is not permitted to modify the errno variable. drivers/ and libc/: OS references to the errno variable should always use the set_errno(), get_errno() macros
Squashed commit of the following:
net/udp: Address most of the issues with UDP write buffering. There is a remaining issue with handling one network going down in a multi-network environment. None of this has been test but it is certainly ready for test. Hence, the feature is marked EXPERIMENTAL.
net/udp: Some baby steps toward a corrected write buffering design.
net/udp: Remove pesky write buffer macros.
Eliminate trailing space at the end of lines.
net/udp: A little more UDP write buffering logic. Still at least on big gaping hole in the design.
net/udp: Undefined CONFIG_NET_SENDTO_TIMEOUT.
net/udp: Crude, naive port of the TCP write buffering logic into UDP. This commit is certainly non-functional and is simply a starting point for the implementatin of UDP write buffering.
net/udp: Rename udp/udp_psock_sendto.c udp/udp_psock_sendto_unbuffered.c.
Replace all usage kill() in the OS proper with nxsig_kill().
sched/signal: Add nxsig_kill() which is functionally equivalent to kill() except that it does not modify the errno variable.
This commit backs out most of commit b4747286b1. That change was added because sem_wait() would sometimes cause cancellation points inappropriated. But with these recent changes, nxsem_wait() is used instead and it is not a cancellation point.
In the OS, all calls to sem_wait() changed to nxsem_wait(). nxsem_wait() does not return errors via errno so each place where nxsem_wait() is now called must not examine the errno variable.
In all OS functions (not libraries), change sem_wait() to nxsem_wait(). This will prevent the OS from creating bogus cancellation points and from modifying the per-task errno variable.
sched/semaphore: Add the function nxsem_wait(). This is a new internal OS interface. It is functionally equivalent to sem_wait() except that (1) it is not a cancellation point, and (2) it does not set the per-thread errno value on return.
sched/semaphore: Add nxsem_post() which is identical to sem_post() except that it never modifies the errno variable. Changed all references to sem_post in the OS to nxsem_post().
sched/semaphore: Add nxsem_destroy() which is identical to sem_destroy() except that it never modifies the errno variable. Changed all references to sem_destroy() in the OS to nxsem_destroy().
libc/semaphore and sched/semaphore: Add nxsem_getprotocol() and nxsem_setprotocola which are identical to sem_getprotocol() and set_setprotocol() except that they never modifies the errno variable. Changed all references to sem_setprotocol in the OS to nxsem_setprotocol(). sem_getprotocol() was not used in the OS
libc/semaphore: Add nxsem_getvalue() which is identical to sem_getvalue() except that it never modifies the errno variable. Changed all references to sem_getvalue in the OS to nxsem_getvalue().
sched/semaphore: Rename all internal private functions from sem_xyz to nxsem_xyz. The sem_ prefix is (will be) reserved only for the application semaphore interfaces.
libc/semaphore: Add nxsem_init() which is identical to sem_init() except that it never modifies the errno variable. Changed all references to sem_init in the OS to nxsem_init().
sched/semaphore: Rename sem_tickwait() to nxsem_tickwait() so that it is clear this is an internal OS function.
sched/semaphoate: Rename sem_reset() to nxsem_reset() so that it is clear this is an internal OS function.
Task A holds an IOB. There are no further IOBs. The value of semcount is zero.
Task B calls iob_alloc(). Since there are not IOBs, it calls sem_wait(). The v
alue of semcount is now -1.
Task A frees the IOB. iob_free() adds the IOB to the free list and calls sem_post() this makes Task B ready to run and sets semcount to zero NOT 1. There is one IOB in the free list and semcount is zero. When Task B wakes up it would increment the sem_count back to the correct value.
But an interrupt or another task runs occurs before Task B executes. The interrupt or other tak takes the IOB off of the free list and decrements the semcount. But since semcount is then < 0, this causes the assertion because that is an invalid state in the interrupt handler.
So I think that the root cause is that there the asynchrony between incrementing the semcount. This change separates the list of IOBs: Currently there is only a free list of IOBs. The problem, I believe, is because of asynchronies due sem_post() post cause the semcount and the list content to become out of sync. This change adds a new 'committed' list: When there is a task waiting for an IOB, it will go into the committed list rather than the free list before the semaphore is posted. On the waiting side, when awakened from the semaphore wait, it will expect to find its IOB in the committed list, rather than free list.
In this way, the content of the free list and the value of the semaphore count always remain in sync.
The solution is to remove all of the memory management function calls from the interface. Instead, the interface exports the userspace heap structure and then kernel size implementations of those memory management functions will operate on the userspace heap structure. This avoids the unnecessary system calls and, more importantly, failures do to freeing memory when a test exits.
