Commit Graph

53 Commits

Author SHA1 Message Date
hujun5
8fe8417ffb libc/fdcheck: add fdcheck module
In embedded development environments, due to the lack of address isolation between processes,
fd may be passed between processes and lead to misuse,

We have designed an fd cross-process automatic detection tool,
fdcheck_protect returns the fd containing the pid information,
indicating that the ownership of the current fd belongs to the pid and is not allowed to be used by other processes.
fdcheck_restore will obtain the true fd and check if the ownership of the fd is legal

For ease of understanding, let's give an example where
the following information is represented in 32-bit binary format

fd        00000000 00000000 00000000 10001010
pid       00000000 00000000 00000011 01010101
ret       00000000 00000011 01010101 10001010

Signed-off-by: hujun5 <hujun5@xiaomi.com>
2023-06-10 02:19:58 +08:00
hujun5
02628deb55 libc/fdsan: keep f_tag unchanged
When the file pointer is reopened, keep f_tag unchanged as fd remains unchanged
fix issue https://github.com/apache/nuttx/issues/9494

Signed-off-by: hujun5 <hujun5@xiaomi.com>
2023-06-09 21:06:48 +08:00
hujun5
90387a5b41 libc/misc: add fdsan module
FD (file descriptor) is widely used in system software development,
and almost all implementations of posix os (including nuttx) use FD as an index.
the value of fd needs to be allocated starting from the minimum available value of 3, and each process has a copy,
so the same fd value is very easy to reuse in the program.

In multi threaded or multi process environments without address isolation,
If the ownership, global variables, and competition relationships of fd are not properly handled,
there may be issues with fd duplication or accidental closure.
Further leading to the following issues, which are difficult to troubleshoot.

1. Security vulnerability: the fd we wrote is not the expected fd and will be accessed by hackers to obtain data
2. Program exceptions or crashes: write or read fd failures, and program logic errors
3. The structured file XML or database is damaged: the data format written to the database is not the expected format.

The implementation principle of fdsan is based on the implementation of Android
https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md

Signed-off-by: hujun5 <hujun5@xiaomi.com>
2023-05-17 10:24:42 +08:00
chao an
507c8145a9 sched/spawn: remove spawn proxy thread to simplify task/posix_spawn()
The spawn proxy thread is a special existence in NuttX, usually some developers
spend a lot of time on stack overflow of spawn proxy thread:

https://github.com/apache/nuttx/issues/9046
https://github.com/apache/nuttx/pull/9081

In order to avoid similar issues, this PR will remove spawn proxy thread to simplify
the process of task/posix_spawn().

1. Postpone the related processing of spawn file actions until after task_init()
2. Delete the temporary thread of spawn proxy and related global variables

Signed-off-by: chao an <anchao@xiaomi.com>
2023-04-27 17:35:58 +08:00
Xiang Xiao
a05f9aaa85 fs: Don't guard fsync with CONFIG_DISABLE_MOUNTPOINT
since the driver can also support fsync by implementing BIOC_FLUSH

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2023-02-14 11:24:37 +08:00
dongjiuzhu1
63e6f80fb0 sync: add sync api
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
2023-02-07 12:15:05 +02:00
Alan Carvalho de Assis
c3a34a5956 fs: Avoid accessing filep fields if it is NULL & add DEBUGASSERTs 2023-01-20 22:31:00 +08:00
Xiang Xiao
488a56280a fs: Correct the comment of file_allocate
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2022-10-28 10:12:08 +02:00
Xiang Xiao
604eea453b fs/vfs: Let caller control whether add the reference count of inode in file_allocate
to simplify the caller in some special case

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2022-10-28 10:12:08 +02:00
Xiang Xiao
4af98af376 fs/vfs: Rename files_allocate to file_allocate
align with the intention better

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2022-10-28 10:12:08 +02:00
Xiang Xiao
6b31918b42 Remove the unnecessary cast for main_t, NULL and argv
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2022-10-18 08:51:45 +02:00
anjiahao
d1d46335df Replace nxsem API when used as a lock with nxmutex API
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2022-10-17 15:59:46 +09:00
chao an
42388f16e9 fs/dup2: fix potential deadlock on usrsock
apps/examples/usrsocktest/usrsocktest_basic_daemon.c:

321 static void basic_daemon_dup2(FAR struct usrsocktest_daemon_conf_s *dconf)
322 {
...
335   ret = dup2(sd2, sd);
352 }

Usrsocktest Task hold the file group lock and send the close request to usrsock deamon :

| #0  net_lockedwait_uninterruptible (sem=0x5555555f8ba2 <g_usrsockdev+34>) at utils/net_lock.c:427
| #1  0x000055555557489c in usrsockdev_do_request (conn=0x5555555f8800 <g_usrsock_connections>, iov=0x7ffff3f36040, iovcnt=1) at usrsock/usrsock_dev.c:1185
|                           --> send close request to usrsock deamon
|
| #2  0x00005555555d0439 in do_close_request (conn=0x5555555f8800 <g_usrsock_connections>) at usrsock/usrsock_close.c:109
| #3  0x00005555555d04f5 in usrsock_close (conn=0x5555555f8800 <g_usrsock_connections>) at usrsock/usrsock_close.c:157
| #4  0x00005555555cf100 in usrsock_sockif_close (psock=0x7ffff3ea4a60) at usrsock/usrsock_sockif.c:234
| #5  0x00005555555c7b2f in psock_close (psock=0x7ffff3ea4a60) at socket/net_close.c:102
| #6  0x000055555557a518 in sock_file_close (filep=0x7ffff3f253d0) at socket/socket.c:115
| #7  0x000055555557678f in file_close (filep=0x7ffff3f253d0) at vfs/fs_close.c:74
| #8  0x000055555557694c in file_dup2 (filep1=0x7ffff3f253e8, filep2=0x7ffff3f253d0) at vfs/fs_dup2.c:129
|                           --->  hold group file list lock  ( _files_semtake(list) )
|
| #9  0x0000555555575aab in nx_dup2 (fd1=7, fd2=6) at inode/fs_files.c:451
| #10 0x0000555555575af3 in dup2 (fd1=7, fd2=6) at inode/fs_files.c:473
| #11 0x000055555559d937 in basic_daemon_dup2 (dconf=0x5555555f8d80 <usrsocktest_daemon_config>) at usrsocktest_basic_daemon.c:335
| #12 0x000055555559ed80 in usrsocktest_test_basic_daemon_basic_daemon_dup2 () at usrsocktest_basic_daemon.c:612
| #13 0x000055555559f18d in usrsocktest_group_basic_daemon_run () at usrsocktest_basic_daemon.c:666
| #14 0x0000555555599f8d in run_tests (name=0x5555555dc8c3 "basic_daemon", test_fn=0x55555559ef50 <usrsocktest_group_basic_daemon_run>) at usrsocktest_main.c:117
| #15 0x000055555559a06c in run_all_tests () at usrsocktest_main.c:154
| #16 0x000055555559a3d1 in usrsocktest_main (argc=1, argv=0x7ffff3f25450) at usrsocktest_main.c:248
| #17 0x000055555555cad8 in nxtask_startup (entrypt=0x55555559a357 <usrsocktest_main>, argc=1, argv=0x7ffff3f25450) at sched/task_startup.c:70
| #18 0x0000555555559938 in nxtask_start () at task/task_start.c:134

Usrsock Deamon weakup and setup the poll want to perform close request, but locked on fs_getfilep():

| #0  _files_semtake (list=0x7ffff3f250b8) at inode/fs_files.c:51
|                           --> Request group lock but which hold by close request, deadlock
| #1  0x00005555555758b1 in fs_getfilep (fd=5, filep=0x7ffff3f47190) at inode/fs_files.c:375
| #2  0x00005555555d3064 in poll_fdsetup (fd=5, fds=0x7ffff3f47290, setup=true) at vfs/fs_poll.c:79
| #3  0x00005555555d3243 in poll_setup (fds=0x7ffff3f47290, nfds=2, sem=0x7ffff3f47206) at vfs/fs_poll.c:139
| #4  0x00005555555d39a6 in nx_poll (fds=0x7ffff3f47290, nfds=2, timeout=-1) at vfs/fs_poll.c:383
| #5  0x00005555555d3abd in poll (fds=0x7ffff3f47290, nfds=2, timeout=-1) at vfs/fs_poll.c:501
|                           --> daemon weak up
| #6  0x00005555555c62c7 in usrsocktest_daemon (param=0x5555555f5360 <g_ub_daemon>) at usrsocktest_daemon.c:1846
| #7  0x000055555559161e in pthread_startup (entry=0x5555555c60d3 <usrsocktest_daemon>, arg=0x5555555f5360 <g_ub_daemon>) at pthread/pthread_create.c:59
| #8  0x00005555555d45f0 in pthread_start () at pthread/pthread_create.c:175
| #9  0x0000000000000000 in ?? ()

Signed-off-by: chao an <anchao@xiaomi.com>
2022-09-01 12:59:46 +08:00
Nathan Hartman
849f760b77 Fix various typos 2022-07-08 02:15:54 +08:00
anjiahao
bc998f6072 fs:fs_getfilep changes fd judgment method
if a fd was closed,need return EBADF
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
2022-04-19 22:48:39 +08:00
田昕
463a437733 fs and unistd: increase OPEN_MAX by claiming a Kconfig.
Signed-off-by: 田昕 <tianxin7@xiaomi.com>
2022-04-01 12:17:47 +03:00
tianxin7
b53bbb15c5 fs:add _POSIX_OPEN_MAX limit to files_extend
Signed-off-by: 田昕 <tianxin7@xiaomi.com>
2022-03-29 17:51:20 +08:00
Jiuzhu Dong
66d854371b fs/files_allocate: assert when fd overflow
Signed-off-by: Jiuzhu Dong <dongjiuzhu1@xiaomi.com>
2022-03-29 00:19:13 +08:00
Xiang Xiao
ddde41018a Revert "sched: Don't duplicate caller file handler when creating kernel thread"
since the temporary kernel spawn proxy need duplicate file handler
from caller, please reference the follow source code for more info:
sched/task/task_posixspawn.c
sched/task/task_spawn.c

This reverts commit 5c5f1dec08.
2022-03-01 12:26:49 +01:00
Xiang Xiao
5c5f1dec08 sched: Don't duplicate caller file handler when creating kernel thread
kernel thread should have only the starndard file i/o just like idle thread

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2022-02-01 08:22:45 +09:00
ligd
6d3ccf37ab fs: move out nx_close from filelock
If the close is blocked(like net close timeout 5s),
then other file operation can't access in this 5s.

Change-Id: Ia6c4ec88d90db0330134d0aaa30e94cb71c8a066
Signed-off-by: ligd <liguiding1@xiaomi.com>
2021-07-03 07:25:51 -05:00
chao.an
58bd0468ab fs/inode: correct the return value
correct the return value of fs_getfilep() since the
semaphore take may probably fail if the thread canceled

Signed-off-by: chao.an <anchao@xiaomi.com>
2021-04-19 09:50:45 +01:00
Jiuzhu Dong
e96c8b9283 fs: allocate file/socket dynamically
Change-Id: I8aea63eaf0275f47f21fc8d5482b51ffecd5c906
Signed-off-by: Jiuzhu Dong <dongjiuzhu1@xiaomi.com>
2021-03-17 06:46:42 -07:00
Jiuzhu Dong
4d5a964f29 net: unify socket into file descriptor
Change-Id: I9bcd21564e6c97d3edbb38aed1748c114160ea36
Signed-off-by: Jiuzhu Dong <dongjiuzhu1@xiaomi.com>
2021-03-03 19:01:41 -08:00
Xiang Xiao
8d0fd4038b Remove the empty xxx_initialize functions
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: I63cb6b37f78e910663724659e11f53e3335d419f
2021-03-03 08:21:04 +00:00
Xiang Xiao
e49bae1300 fs: Remove fs_dupfd and fs_dupfd2 internal functions
let's call either nx_dup/nx_dup2 or file_dup/file_dup2
instead just like other fs api: xxx->nx_xxx->file_xxx

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: I1aacfb9e25dc7b3fcb0345ff7b269b1953a01e5b
2021-01-13 16:39:45 +08:00
Xiang Xiao
1e5bfa623a fs: file_dup2 shouldn't hold the file list lock
the argument passed to file_dup2 doesn't always come from task file list
so it doesn't make sense to hold the file list lock and then it is better
to do the protection in the new function files_dupfd2

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: Ibf02cea9b0b275e7472f9c04fd66b9242285b957
2021-01-13 16:39:45 +08:00
ligd
e772be8c8a fs/inode/fs_files.c: fix DEBUGASSERT in romfs
up_assert: Assertion failed at file:romfs/fs_romfs.c line: 643

newp->f_priv should be NULL.

Change-Id: I5f72b1329a8938ce39cf1ac5fbee561ca106f4f6
Signed-off-by: ligd <liguiding1@xiaomi.com>
2021-01-08 10:20:47 -06:00
Xiang Xiao
86fab49c46 fs: Fix the race condition in file_dup
NULL inode passed to files_allocate doesn't mark file struct in the
allocated state, so other threads which invovle in file allocation
(e.g. open or dup) may allocate the same file struct again.

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: I53ff876eae3c7a1e311e7f671686b73a4b4ef891
2021-01-08 11:01:46 +08:00
Xiang Xiao
b343caae4d fs: file_dup2 shouldn't destroy filep2 before the duplication of filep1 succeed
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: I10956f5e32cc0add414076465c06a24a858e52bc
2021-01-08 11:01:46 +08:00
Xiang Xiao
5a7d988a43 fs: Remove _files_close and reuse file_close
to save the code space

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: I08455010ba121a61b0e29334b580aa83e69a9418
2021-01-03 15:45:52 +01:00
Xiang Xiao
cfd7390676 fs: Let files_allocate return -EMFILE instead ERROR
since the internal function normally return the error code directly

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: Ifccbcfb07d2f8f7d5fef1b2a867538f51b73655c
2021-01-03 11:48:35 +01:00
Xiang Xiao
63bc3efd25 fs: Remove inode null check from file_dup and fs_dupfd2
since the same check already done in file_dup2

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Change-Id: I7ba1309c55be1ac564f798df02fc6725c4a0d469
2021-01-03 00:03:22 +01:00
dongjiuzhu
c22ab31728 fs: flashback to release all file descriptors in group to increase debugging time.
Change-Id: I47851f7e679addf36077c491ef7d1a5a29d1c2be
Signed-off-by: dongjiuzhu <dongjiuzhu1@xiaomi.com>
2020-11-17 18:54:31 -08:00
Gregory Nutt
a4218e2144 include/nuttx/sched.h: Make naming of all internal names consistent:
1. Add internal scheduler functions should begin with nxsched_, not sched_
2. Follow the consistent naming patter of https://cwiki.apache.org/confluence/display/NUTTX/Naming+of+OS+Internal+Functions
2020-05-09 14:19:08 -03:00
Gregory Nutt
a4d6af8343 Check return from nxsem_wait_initialize()
Resolution of Issue 619 will require multiple steps, this part of the first step in that resolution:  Every call to nxsem_wait_uninterruptible() must handle the return value from nxsem_wait_uninterruptible properly.  This commit is only for those files under fs/inode.  Utility functions under fs/incode were modified so the changes do extend to other fs/ sub-directories as well.
2020-03-30 01:33:31 +01:00
Xiang Xiao
cde88cabcc Run codespell -w with the latest dictonary again
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2020-02-23 22:27:46 +01:00
Xiang Xiao
5c80b94820 Replace #include <semaphore.h> to #include <nuttx/semaphore.h>
Since the kernel side should call nxsem_xxx instead and remove the unused inclusion
2020-02-01 08:27:30 -06:00
Xiang Xiao
6a3c2aded6 Fix wait loop and void cast (#24)
* Simplify EINTR/ECANCEL error handling

1. Add semaphore uninterruptible wait function
2 .Replace semaphore wait loop with a single uninterruptible wait
3. Replace all sem_xxx to nxsem_xxx

* Unify the void cast usage

1. Remove void cast for function because many place ignore the returned value witout cast
2. Replace void cast for variable with UNUSED macro
2020-01-02 10:54:43 -06:00
Gregory Nutt
bd3cc792ff fs/: Run all .c files under fs/ through tools/nxstyle. 2019-10-27 11:48:14 -06:00
Gregory Nutt
9568600ab1 Squashed commit of the following:
This commit backs out most of commit b4747286b1.  That change was added because sem_wait() would sometimes cause cancellation points inappropriated.  But with these recent changes, nxsem_wait() is used instead and it is not a cancellation point.

    In the OS, all calls to sem_wait() changed to nxsem_wait().  nxsem_wait() does not return errors via errno so each place where nxsem_wait() is now called must not examine the errno variable.

    In all OS functions (not libraries), change sem_wait() to nxsem_wait().  This will prevent the OS from creating bogus cancellation points and from modifying the per-task errno variable.

    sched/semaphore:  Add the function nxsem_wait().  This is a new internal OS interface.  It is functionally equivalent to sem_wait() except that (1) it is not a cancellation point, and (2) it does not set the per-thread errno value on return.
2017-10-04 15:22:27 -06:00
Gregory Nutt
42a0796615 Squashed commit of the following:
sched/semaphore:  Add nxsem_post() which is identical to sem_post() except that it never modifies the errno variable.  Changed all references to sem_post in the OS to nxsem_post().

    sched/semaphore:  Add nxsem_destroy() which is identical to sem_destroy() except that it never modifies the errno variable.  Changed all references to sem_destroy() in the OS to nxsem_destroy().

    libc/semaphore and sched/semaphore:  Add nxsem_getprotocol() and nxsem_setprotocola which are identical to sem_getprotocol() and set_setprotocol() except that they never modifies the errno variable.  Changed all references to sem_setprotocol in the OS to nxsem_setprotocol().  sem_getprotocol() was not used in the OS
2017-10-03 15:35:24 -06:00
Gregory Nutt
83cdb0c552 Squashed commit of the following:
libc/semaphore:  Add nxsem_getvalue() which is identical to sem_getvalue() except that it never modifies the errno variable.  Changed all references to sem_getvalue in the OS to nxsem_getvalue().

    sched/semaphore:  Rename all internal private functions from sem_xyz to nxsem_xyz.  The sem_ prefix is (will be) reserved only for the application semaphore interfaces.

    libc/semaphore:  Add nxsem_init() which is identical to sem_init() except that it never modifies the errno variable.  Changed all references to sem_init in the OS to nxsem_init().

    sched/semaphore:  Rename sem_tickwait() to nxsem_tickwait() so that it is clear this is an internal OS function.

    sched/semaphoate:  Rename sem_reset() to nxsem_reset() so that it is clear this is an internal OS function.
2017-10-03 12:52:31 -06:00
Gregory Nutt
e4dd33280d Squashed commit of the following:
psock_close() and net_close() are internal OS functions and should not set the errno variable.

    psock_ioctl() and netdev_ioctl() are internal OS functions and should not set the errno variable.

    net_dupsd() and net_dupsd2() are internal OS functions and should not set the errno variable.

    net/ and fs/: net_vfcntl(), file_fcntl(), file_dup(), and file_dup2() are all internal OS interfaces and should not modify the errno value.
2017-09-30 10:41:21 -06:00
Gregory Nutt
86b79b33cf Reserver the name 'err' for other purposes 2016-06-11 14:40:07 -06:00
Gregory Nutt
001715d57b FS: Add logic to detach a file structure from a file descriptor. This is for use only within the OS. It permits an open file or driver to be used across multiple threads. 2016-05-26 09:41:50 -06:00
Gregory Nutt
3f731241cb fs/inode/, fs/vfs/, and sched/task/: File and socket descriptors are no longer allocated for kernel threads. They must use SYSLOG for output and the low-level psock interfaces for network I/O. This saves a little memory which might be important for small footprint configurations. 2016-04-14 10:14:38 -06:00
Gregory Nutt
0fb035f76b Standardize some naming in code section comments 2016-02-21 18:09:04 -06:00
Gregory Nutt
c70987e551 nuttx/fs: Fix some spacing and alignment issues 2015-10-11 11:39:29 -06:00
Gregory Nutt
d914f3ceec Major structure of file system functions to better support asynchronous I/O. Respository should not be trusted until I have a chance to verify everything 2014-10-06 10:53:25 -06:00