nuttx/syscall/syscall.csv
Gregory Nutt 8868c58720 Fix Deadloop in VFS if CONFIG_CANCELLATION_POINTS is enabled
If cancellation points are enabled, then the following logic is activated in sem_wait().  This causes ECANCELED to be returned every time that sem_wait is called.

    int sem_wait(FAR sem_t *sem)
    {
      ...

      /* sem_wait() is a cancellation point */

      if (enter_cancellation_point())
        {
    #ifdef CONFIG_CANCELLATION_POINTS
          /* If there is a pending cancellation, then do not perform
           * the wait.  Exit now with ECANCELED.
           */

          errcode = ECANCELED;
          goto errout_with_cancelpt;
    #endif
        }
      ...

Normally this works fine.  sem_wait() is the OS API called by the application and will cancel the thread just before it returns to the application.  Since it is cancellation point, it should never be called from within the OS.

There there is is one perverse cases where sem_wait() may be nested within another cancellation point.  If open() is called, it will attempt to lock a VFS data structure and will eventually call nxmutex_lock().  nxmutex_lock() waits on a semaphore:

   int nxmutex_lock(FAR mutex_t *mutex)
   {
     ...

     for (; ; )
       {
         /* Take the semaphore (perhaps waiting) */

         ret = _SEM_WAIT(&mutex->sem);
         if (ret >= 0)
           {
             mutex->holder = _SCHED_GETTID();
             break;
           }

         ret = _SEM_ERRVAL(ret);
         if (ret != -EINTR && ret != -ECANCELED)
           {
             break;
           }
       }
   ...
}

In the FLAT build, _SEM_WAIT expands to sem_wait().  That causes the error in the logic:  It should always expand to nxsem_wait().  That is because sem_wait() is cancellation point and should never be called from with the OS or the C library internally.

The failure occurs because the cancellation point logic in sem_wait() returns -ECANCELED (via _SEM_ERRVAL) because sem_wait() is nested; it needs to return the -ECANCELED error to the outermost cancellation point which is open() in this case.  Returning -ECANCELED then causes an infinite loop to occur in nxmutex_lock().

The correct behavior in this case is to call nxsem_wait() instead of sem_wait().  nxsem_wait() is identical to sem_wait() except that it is not a cancelation point.  It will return -ECANCELED if the thread is canceled, but only once.  So no infinite loop results.

In addition, an nxsem_wait() system call was added to support the call from nxmutex_lock().

This resolves Issue #9695
2023-07-06 14:20:29 -03:00

18 KiB

1_assertassert.hvoidFAR const char *intFAR const char *FAR void *
2accept4sys/socket.hdefined(CONFIG_NET)intintFAR struct sockaddr *FAR socklen_t *int
3clock_nanosleeptime.hintclockid_tintFAR const struct timespec *FAR struct timespec *
4epoll_ctlsys/epoll.hintintintintFAR struct epoll_event *
5epoll_waitsys/epoll.hintintFAR struct epoll_event *intint
6fcntlfcntl.hintintint...int
7fs_fdopennuttx/fs/fs.hdefined(CONFIG_FILE_STREAM)intintintFAR struct tcb_s *FAR struct file_struct **
8ioctlsys/ioctl.hintintint...unsigned long
9mq_receivemqueue.h!defined(CONFIG_DISABLE_MQUEUE)ssize_tmqd_tFAR char *size_tFAR unsigned int *
10mq_sendmqueue.h!defined(CONFIG_DISABLE_MQUEUE)intmqd_tFAR const char *size_tunsigned int
11openfcntl.hintFAR const char *int...mode_t
12ppollpoll.hintFAR struct pollfd *nfds_tFAR const struct timespec *FAR const sigset_t *
13prctlsys/prctl.hintint...uintptr_tuintptr_t
14preadunistd.hssize_tintFAR void *size_toff_t
15pthread_cond_clockwaitpthread.h!defined(CONFIG_DISABLE_PTHREAD)intFAR pthread_cond_t *FAR pthread_mutex_t *clockid_tFAR const struct timespec *
16pwriteunistd.hssize_tintFAR const void *size_toff_t
17recvsys/socket.hdefined(CONFIG_NET)ssize_tintFAR void *size_tint
18sched_backtracesched.hdefined(CONFIG_SCHED_BACKTRACE)intpid_tFAR void **intint
19sendsys/socket.hdefined(CONFIG_NET)ssize_tintFAR const void *size_tint
20sendfilesys/sendfile.hssize_tintintFAR off_t *size_t
21socketpairsys/socket.hdefined(CONFIG_NET)intintintintint [2]|FAR int *
22timer_settimetime.h!defined(CONFIG_DISABLE_POSIX_TIMERS)inttimer_tintFAR const struct itimerspec *FAR struct itimerspec *
23timerfd_settimesys/timerfd.hdefined(CONFIG_TIMER_FD)intintintFAR const struct itimerspec *FAR struct itimerspec *
24waitidsys/wait.hdefined(CONFIG_SCHED_WAITPID) && defined(CONFIG_SCHED_HAVE_PARENT)intidtype_tid_t FAR siginfo_t *int