This adds functionality to map pages dynamically into kernel virtual
memory. This allows implementing I/O remap for example, which is a useful
(future) feature.
Now, the first target is to support mapping user pages for the kernel.
Why? There are some userspace structures that might be needed when the
userspace process is not running. Semaphores are one such example. Signals
and the WDT timeout both need access to the user semaphore to work
properly. Even though for this only obtaining the kernel addressable
page pool virtual address is needed, for completeness a procedure is
provided to map several pages.
Add a new field to record the global on the basis of mm_backtrace.
When using alloc, the field is incremented by 1,
so that the memory usage can be dumped within the range
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
This change introduce 2 items:
1. If the size of the space requested is 0, the behavior is implementation-defined:
either a null pointer shall be returned, or the behavior shall be as if the size
were some non-zero value, except that the behavior is undefined if the returned
pointer is used to access an object.
Change the behavior to be similar to Linux and Android and allocates an object
of a minimum size instead of returning null pointer.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/malloc.htmlhttps://pubs.opengroup.org/onlinepubs/9699919799/functions/calloc.htmlhttps://pubs.opengroup.org/onlinepubs/9699919799/functions/realloc.html
2. The description of realloc() has been modified from previous versions of this
standard to align with the ISO/IEC 9899:1999 standard. Previous versions explicitly
permitted a call to realloc (p, 0) to free the space pointed to by p and return
a null pointer. While this behavior could be interpreted as permitted by this
version of the standard, the C language committee have indicated that this
interpretation is incorrect. Applications should assume that if realloc() returns
a null pointer, the space pointed to by p has not been freed. Since this could lead
to double-frees, implementations should also set errno if a null pointer actually
indicates a failure, and applications should only free the space if errno was changed.
Do not free memory of zero-length reallocation is requested
https://pubs.opengroup.org/onlinepubs/9699919799/functions/realloc.html
Co-authored-by: fangxinyong <fangxinyong@xiaomi.com>
Signed-off-by: Petro Karashchenko <petro.karashchenko@gmail.com>
Use double delim to fix windows native build and give an error:
makefile:132: *** target mode do not include“%”. stop.
In Windows environment DELIM := $(strip \) but \ has two role:
first: \ as directory, and second \ as Escape character, Reference:
https://github.com/apache/nuttx/pull/7572#discussion_r1028219229
Signed-off-by: chao an <anchao@xiaomi.com>
We don't want to get a NULL pointer after iob_pack on an IOB chain with
several iobs with length 0, it should return one IOB with length 0.
Otherwise each place calls iob_pack needs to check the result.
Signed-off-by: Zhe Weng <wengzhe@xiaomi.com>
This way the mappings can be modified for any vm area, not only the
process that is running.
Why? This allows mapping pages to kernel dynamically, this functionality
will be presented later.
caculate blk address when mempool_multiple_free
have a bug. need a real blocksize to caulate the
memory address.
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
D:\archer\code\nuttx\mm\mempool\mempool_multiple.c(180,72): warning C4098: "mempool_multiple_free_callback":"void" void function returning a value
Compiler Warning C4098:
A function declared with return type void has a return statement that returns a value. The compiler assumes the function returns a value of type int.
Reference:
https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-1-c4098?view=msvc-170
Signed-off-by: chao an <anchao@xiaomi.com>
NuttX kernel should not use the syscall functions, especially after
enabling CONFIG_SCHED_INSTRUMENTATION_SYSCALL, all system functions
will be traced to backend, which will impact system performance.
Signed-off-by: chao an <anchao@xiaomi.com>
Original code assumes the previous node in physical must be
ALLOCED, but other thread may free the previous node between
mm_malloc() and mm_lock(), and the original code didn't condsider
this, which will cause two adjacent free nodes situation and this
should not be happened.
This commit will merge the previous node and node if the previous
node is free to avoid the situation discribed above.
Signed-off-by: wangbowen6 <wangbowen6@xiaomi.com>
To avoid lto alias with override __asan_* symbols
undefined reference to `__asan_load4_noabort'
undefined reference to `__asan_load1_noabort'
undefined reference to `__asan_store1_noabort'
undefined reference to `__asan_load1_noabort'
undefined reference to `__asan_store1_noabort'
undefined reference to `__asan_load4_noabort'
undefined reference to `__asan_store4_noabort'
Signed-off-by: chao.an <anchao@xiaomi.com>
When adding more heap memory, the total heap size was not updated. This
results in a crash in mm_mallinfo:
DEBUGASSERT((size_t)info->uordblks + info->fordblks == heap->mm_heapsize);
This commit fixes this issue
Cast substraction arguments to FAR char *, which gives the same result as the
gcc extension on the original void * arithmetic.
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
There are many small memory block in NuttX system, eg: struct tcb_s,
struct inode, etc, and several disadvantages about them:
1.Their frequent allocate and free cause the system memory fragmentation.
2.Since each memory block has an overhead, the utilization of small memory
blocks is relatively low, which will cause memory waste.
So we can use mempool to alloc smallo block, to improve alloc speed
and utilization, to reduce fragmentation.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
do simple copy to instead of memset and memcpy operation because
they have been instrumented, if you access the posion area,
the system will crash.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
The memalign is special to multiple mempool because multiple mempool
doesn't support split and shrink chunk operate. So When you alloc a
memory block and find an aligned address in this block, you need to
occupy 8 bytes before the address to save the address of the padding
size and pool to ensure correct use in realloc and free operations.
So we will use bit1 in the previous address of the address to represent
that it is applied by memalign.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
kasan_report (addr=0xf3c68618, size=1, is_write=false) at
kasan/kasan.c:106
0x56585fbf in __asan_loadN_noabort (addr=0xf3c68618, size=1) at
kasan/kasan.c:300
0x565860ac in __asan_load1_noabort (addr=0xf3c68618) at
kasan/kasan.c:354
0x565843af in memcpy (dest=0xf3de9d6c, src=0xf3c685cc, n=3) at
string/lib_memcpy.c:44
0x56587ae8 in mempool_multiple_realloc (mpool=0xf3c670fc,
oldblk=0xf3c685cc, size=416) at mempool/mempool_multiple.c:218
0x5658707a in mm_realloc (heap=0xf3c67000, oldmem=0xf3c685cc,
size=416) at mm_heap/mm_realloc.c:98
0x5658524e in realloc (oldmem=0xf3c685cc, size=416) at
umm_heap/umm_realloc.c:97
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
This delta describes the relationship between the block size of each
mempool in multiple mempool by user initialized. It is automatically
detected by the mempool_multiple_init function. If the delta is not
equal to 0, the block size of the pool in the multiple mempool is an
arithmetic progressions, otherwise it is an increasing progressions.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
Let's specify size instead of number, so that we can unify the size of
expansion memory in the multiple mempool.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
the crash backtrace:
kasan_report (addr=0xf3d02fd4, size=4, is_write=false) at
kasan/kasan.c:106
0x5658518d in __asan_loadN_noabort (addr=0xf3d02fd4, size=4) at
kasan/kasan.c:300
0x565851ee in __asan_load4_noabort (addr=0xf3d02fd4) at
kasan/kasan.c:334
0x56580b02 in sq_remfirst (queue=0xf3d02b08) at
queue/sq_remfirst.c:45
0x565e0e0b in mempool_alloc (pool=0xf3d02aec) at
mempool/mempool.c:161
0x566033d2 in mempool_multiple_alloc (mpool=0xf3d02a30, size=16) at
mempool/mempool_multiple.c:147
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
- Also remove the nuttx private shm.h file nuttx/mm/shm.h, which became redundant
- Also remove the gran allocator initialization/release in binfmt since common
vpage allocator is initialized in group_create/group_leave
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
The dependency should be vice versa; the MM_SHM should depend on the
existence of the virtual memory range allocator.
Create a new CONFIG flag CONFIG_ARCH_VMA_MAPPING, which will define that
there is a virtual memory range allocator. Make MM_SHM select that flag
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
Replace static gs_vaddr with a new dynamic mapping list. Collecting all
this kind of virtual memory mappings into a single structure makes
things more consistent.
This still leaves the task group specific granule alloocator, gs_handle,
in the task group
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
The task_group specific list can be used to store information about
mmappings.
For a driver or filesystem performing mmap can also enable munmap by
adding an item to this list using mm_map_add(). The item is then
returned in the corresponding munmap call.
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
ubsan/ubsan.c: In function ‘get_signed_val’:
ubsan/ubsan.c:162:22: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
162 | uint64_t ret = (uint64_t)val & mask;
|
Signed-off-by: chao an <anchao@xiaomi.com>
This will allow changing the underlying backend, i.e. granule allocator,
to something more efficient, if this is needed later.
Also, it gives easier access to the common SHM book keeping.
User can ask for specific granules to be allocated.
This is useful for one thing only: when mmap() is called for a specific
vaddr. The parameter itself is non-sensical, this is just to satisfy
the POSIX standard.
In the past, header file paths were generated by the incdir command
Now they are generated by concatenating environment variables
In this way, when executing makefile, no shell command will be executed,
it will improve the speed of executing makfile
Signed-off-by: yinshengkai <yinshengkai@xiaomi.com>
In the past, predefined macros were generated by define.sh scripts
Now they are generated by concatenating environment variables
In this way, when executing makefile, no shell command will be executed,
it will improve the speed of executing makfile
Signed-off-by: yinshengkai <yinshengkai@xiaomi.com>
The return value of function mm_takesemaphore will never below
than zero, DEBUGVERIFY make no effect to check it, use DEBUGASSERT
instead.
Signed-off-by: xiangdong6 <xiangdong6@xiaomi.com>
Summary:
- In the case of CONFIG_BUILD_KERNEL=y, showing Kmem and Page
info is enough for free command
Impact:
- CONFIG_BUILD_KERNEL=y only
Testing:
- Tested with sabre-6quad:netknsh
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
It is ok to call kmm_free with a NULL pointer. Thus adopt the
DEBUGASSERT statement to cover this case.
Signed-off-by: Michael Jung <michael.jung@secore.ly>
so the user could disable the full image instrumentation,
but enable the instrumentation by files or directories.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
without UBSan
```
text data bss dec hex filename
85612 208 142258 228078 37aee nuttx
```
with UBSan:
```
text data bss dec hex filename
194290 98164 208634 501088 7a560 nuttx
```
```c
int main(int argc, FAR char *argv[])
{
uint32_t ptr[32];
printf("Hello, World!! %lu\n", ptr[64]);
return 0;
}
```
Try to run this sample:
```
nsh> hello
ubsan_prologue: ================================================================================
ubsan_prologue: UBSAN: array-index-out-of-bounds in hello_main.c:39:37
__ubsan_handle_out_of_bounds: index 64 is out of range for type 'uint32_t [32]'
ubsan_epilogue: ================================================================================
Hello, World!! 1070182368
nsh>
```
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
since it is impossible to track producer and consumer
correctly if TCP/IP stack pass IOB directly to netdev
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
The free node is still in use after kasan_poison(), the node member
access will cause the assert report by kasan.
| (gdb) bt
| #0 kasan_report (addr=1743265406637584896, size=140737337053680, is_write=46) at kasan/kasan.c:97
| #1 0x0000555555607bdd in __asan_loadN_noabort (addr=140737272831420, size=4) at kasan/kasan.c:289
| #2 0x0000555555607cd7 in __asan_load4_noabort (addr=140737272831420) at kasan/kasan.c:323
| #3 0x00005555556061ef in gmtime_r (timep=0x7ffff3275dbc, result=0x7ffff3275e10) at time/lib_gmtimer.c:301
| #4 0x000055555560e507 in sim_rtc_rdtime (lower=0x55555576b780 <g_sim_rtc>, rtctime=0x7ffff3275e10) at sim/up_rtc.c:77
| #5 0x00005555555fcbdb in up_rtc_gettime (tp=0x7ffff3275ef0) at timers/arch_rtc.c:128
| #6 0x00005555555f08b4 in clock_systime_timespec (ts=0x7ffff3275ef0) at clock/clock_systime_timespec.c:72
| #7 0x00005555555ecc77 in note_common (tcb=0x7ffff31d2180, note=0x7ffff3275f80, length=21 '\025', type=18 '\022') at sched/sched_note.c:144
| #8 0x00005555555ed706 in sched_note_syscall_enter (nr=1, argc=0) at sched/sched_note.c:765
| #9 0x000055555560eb37 in __wrap_getpid () at wraps/WRAP_getpid.c:26
| #10 0x0000555555608d1c in mm_takesemaphore (heap=0x7ffff30ae000) at mm_heap/mm_sem.c:127
| #11 0x0000555555609477 in mm_free (heap=0x7ffff30ae000, mem=0x7ffff3265b80) at mm_heap/mm_free.c:89
| #12 0x00005555556070c5 in free (mem=0x7ffff3265b80) at umm_heap/umm_free.c:49
| #13 0x000055555560c3b0 in up_release_stack (dtcb=0x7ffff31e4b00, ttype=0 '\000') at sim/up_releasestack.c:67
| #14 0x00005555555f2515 in nxsched_release_tcb (tcb=0x7ffff31e4b00, ttype=0 '\000') at sched/sched_releasetcb.c:134
| #15 0x00005555556bdf0c in nxtask_terminate (pid=4, nonblocking=true) at task/task_terminate.c:184
| #16 0x00005555556bdb0f in nxtask_exit () at task/task_exit.c:168
| #17 0x000055555566e05f in up_exit (status=0) at sim/up_exit.c:64
| #18 0x000055555564f454 in _exit (status=0) at task/exit.c:78
| #19 0x000055555560ea89 in __wrap__exit (parm1=0) at wraps/WRAP__exit.c:27
| #20 0x00005555555eb288 in exit (status=0) at stdlib/lib_exit.c:54
| #21 0x00005555555fe2cc in nxtask_startup (entrypt=0x555555670c34 <critmon_start_main>, argc=1, argv=0x7ffff3265bb0) at sched/task_startup.c:70
| #22 0x00005555555f02a0 in nxtask_start () at task/task_start.c:134
| #23 0x0000000000000000 in ?? ()
Signed-off-by: chao.an <anchao@xiaomi.com>
Targets build during the kernel phase did not have their dependencies
specified and thus they were not rebuilt after their dependencies have
changed, for example by changing options in menuconfig.
Signed-off-by: Abdelatif Guettouche <abdelatif.guettouche@espressif.com>
When allocation failed, it isn't too uncommon for the caller
to fall back to other allocation method.
(eg. esp32 textheap code tries iram heap when an allocation from rtc heap
failed.)
DEBUGASSERT(false) is too much in that case.
This commit removes the DEBUGASSERT, and also makes the heap dump
a separate option.
malloc() should return aligned (with MM_MIN_CHUNK) pointer, but
pr #5906 destroy that, this pr find a better method to solve
these questions.
Signed-off-by: YAMAMOTO Takashi <yamamoto@midokura.com>
and
Signed-off-by: wangbowen6 <wangbowen6@xiaomi.com>
