fs/procfs/fs_procfsproc: Extended the process ID ProcFS output to show per-thread maximum time for pre-emption disabled and maximum time within a critical section.
sched/sched/sched_critmonitor.c: Adds data collection logic in support of monitoring critical sections and pre-emption state.
I see the following behaviour on NuttX 7.26, where I have SD card mounted on /flash and a directory called "frm" on it:
opendir("/flash") returns (DIR *) 0x1000c580
opendir("/flash/") returns (DIR *) 0x1000c5d0
opendir("/flash/frm") returns (DIR *) 0x1000c620
opendir("/flash/frm/") returns (DIR *) 0x0
From POSIX specs for opendir(): "A pathname ... that ends with one or more trailing slashes shall be resolved as if a single dot character ( '.' ) were appended to the pathname."
So for mount points, opendir() works correctly, but for FAT32 filesystem it fails to open directory if the path has a trailing slash. I'm not quite sure how to cleanly fix this. Stripping the trailing slash in opendir() would require allocating a separate buffer, while fixing it in the FAT32 code seems somewhat complex due to the short/long filename logic.
It is not a big issue for me, I'm just going to fix it on the application side. But still a small portability and standards compliance issue.
NOTE: You would not see this problem if you call opendir() indirectly in NSH (like 'ls -R /') because NSH contains logic to remove trailing '/' characters from paths.
Pullreq FAT improvements
* NuttX: Add CONFIG_FAT_LFN_ALIAS_HASH to speed up creating long filenames.
Long filenames on FAT filesystems have associated 8.3 character alias
short filenames. The traditional form of these is FILENA~1.EXT with
a running count of the number of similar names. However creating this
unique count can take several seconds if there are many similarly named
files in the directory. Enabling FAT_LFN_ALIAS_HASH uses an alternative
format of FI0123~1.TXT where the four digits are a hash of the original
filename. This method is similar to what is used by Windows 2000 and
later.
* NuttX: Add CONFIG_FAT_LFN_ALIAS_TRAILCHARS alternative format for 8.3 filenames.
Traditional format for long filename 8.3 aliases takes first 6
characters of long filename. If this option is set to N > 0,
NuttX will instead take first 6-N and last N characters to form
the short name. This is useful for filenames like "datafile12.txt"
where the first characters would always remain the same.
* NuttX: FAT32: Fix file date corruption in fat_truncate().
* NuttX: if SD card wait seems to be a long one, give time for other threads to run.
Approved-by: GregoryN <gnutt@nuttx.org>
fs/spiffs: Finished review, update, and repartitioning of spiffs_core.c.
fs/spiffs: Converted macro SPIFFS_VALIDATE_OBJIX to a function.
fs/spiffs: Move SPIFFS_VALIDATE_DATA and SPIFFS_CHECK_RES macros inline.
Squashed commit of the following:
fs/spiffs: Fix last compilation issue. Now compiles without error. It is still not quite ready for testing as there is additional code review that must be be performed. It is now marked as EXPERIMENTAL so that it can be brought onto the master branch with little risk.
fs/spiffs: Remove some dead code.
fs/spiffs: Weak start of analysis of spiffs_nucleus.c. Renamed to spiffs_core.c
fs/spiffs: Rename spiffs_nucleus.c to spiffs_core.c
fs/spiffs: Remove spiffs_config.h. All configuration settings are now available in the SPIFFS Kconfig options.
fs/spiffs: Finished review, update, and repartitioning of spiffs_check.c. Added spiffs_check.h.
fs/spiffs: Finished review, update, and repartitioning of spiffs_cache.c. Added spiffs_cache.h.
fs/spiffs: Clean up some defines used in debug output statements.
fs/spiffs: Finished review, update, and repartitioning of spiffs_gc.c. Added spiffs_gc.h.
fs/spiffs: Now that VFS interface is completed, I have begun the long march of repartitioning the remaining functionality, reviewing logic, identifying dead code, and cleaning up loose ends.
fs/spiffs: Initial integration of MTD interface, replacing the SPIFFS native flash interface. Lots of open issues such as the use of pages vs. blocks vs. erase blocks and units of addresses, offsets, and lengths that are passed in function calls. Remove SPIFFS_USE_MAGIC support. That option (which default to OFF anyway), wrote a magic value at the beginning of every sector and support verifiable identification of the file system. It was not being and used and removing it makes life simpler.
fs/spiffs: Remove semaphore lock on the file object structure. Ultimately, the file access must modify the volume and access the volume structue which also has a exclusivity lock. So use of the volume lock alone should be sufficient.
Integrated the SPIFFS rename logic into the NuttX VFS. Removed non-standard application calls or convert them to IOCTL commands. These were converted to IOCTL commands: (1) integrity check, (2) garbage collection, and (3) format flash. These were removed: (1) Integrity check callback. These provided a lot of good information about the state of the file system, but such callbacks are not compatible with a POSIX compliant file system. (2) Index maps. The index maps were a performance improvement feature. The user could provide the memory and request that a region of a a file use that memory for improved lookup performance when accessing parts of the file. The fallback is the less performance lookup by traversing the FLASH memory. (3) Removed the quick garbage collection interface (the code is still used internally). Only the full garbage collection is available to the user application via IOCTL.
configs/sim/spiffs: A simulator configuration to use for testing SPIFFS.
fs/spiffs: Integrate SPIFFS logic into NuttX VFS bind() and unbind() methods.
fs/mount/fs_mount.c: Add SPIFFS to the list of drivers that require MTD vs block drivers.
fs/spiffs: Trivial changes, mostly from analysis of how to integrate the rename() VFS method.
fs/spiffs: Connect NuttX VFS unlink method to the SPIFFS_remove() function. Lots of name-changing.
fs/spiffs: Remove non-standard errno support. Remove bogus SPIFFS_LOCK() and SPIFFS_UNLOCK() macros.
fs/spiffs: Add NuttX VFS implementation for statfs() method. Clean up some of the accumulating compilation problems.
fs/spiffs: Add stat(), truncate() methods. Dummy out unsupport mkdir() and rmdir() methods.
fs/spiffs: Replace some of the custom error numbers with standard error numbers.
fs/spiffs: Hooks read(), write(), fstat(), ioctl(), opendir(), closedir(), rewindif(), and readdir() into the NuttX VFS.
fs/spiffs: Beginning the organization to work with the NuttX VFS. Lots of things are get broken!
fs/spiffs: Add spiffs.c which will be the interface between SPIFFS and NuttX. No very close at present, however.
fs/spiffs: Clean up some compile problems introduced by coding standard changes.
fs/spiffs: A little closer to NuttX coding standard.
fs/spiffs: Ran tools/indent.sh against all files. Closer to NuttX coding standard, but needs a lot more effort to be fully compliant.
fs/spiffs: This commit brings in version 0.3.7 of Peter Anderson's SPIFFS. The initial commit includes the core FS files (with some definitions destributed to their correct header files) and hooks into the build system.
fs/driver and fs/mount: Add mount() support for file systems that require MTD drivers (vs. block drivers).
fs/drivers: Add support for named MTD drivers in the psuedo file system. This will, eventually, allow us to mount file systems that need MTD drivers without having to fake an intervening block driver.
many locations: Change occurences of open() followed by file_detach() to file_open(). Change most non-controversion calls to open() to nx_open().
fs/inode/fs_fileopen.c: Flesh out file_open() with some interim, placeholder logic.
fs/inode/fs_fileopen.c: Add a framework for a file_open() implementation (no real logic in place yet).
fs/vfs/fs_open.c: Add nx_open() which is the same as open() except that it does not create a cancellation point nor does it modify the errno variable.
Squashed commit of the following:
mm/iob: The IOB available notifier is now just a wrapper around the common signal notifier.
sched/signal: Add a generic signal notification facility.
sched/signal/sig_evthread.c: More trivial naming changes.
sched/signal: Rename nxsig_notification() to nxsig_evthread() to make forthcoming naming additions more consistent.
Squashed commit of the following:
Add procfs support to show stopped tasks. Add nxsig_action() to solve a chicken and egg problem: We needed to use sigaction to set default actions, but sigaction() would refuse to set actions if the default actions could not be caught or ignored.
sched/signal: Add configuration option to selectively enabled/disable default signal actions for SIGSTOP/SIGSTP/SIGCONT and SIGKILL/SIGINT. Fix some compilation issues.
sched/sched: Okay.. I figured out a way to handle state changes that may occur while they were stopped. If a task/thread was already blocked when SIGSTOP/SIGSTP was received, it will restart in the running state. I will appear that to the task/thread that the blocked condition was interrupt by a signal and returns the EINTR error.
sched/group and sched/sched: Finish framework for continue/resume logic.
sched/signal: Roughing out basic structure to support task suspend/resume
Squashed commit of the following:
fs/fat/fs_fat32util.c: Most costmetic naming MBR to FBR in numerous locations. Change some ferr macros to fwarn. Use FBR macros insteac of MBR macros.
Add definitions for the FAT boot record (FBR).
fs/fat: Clean up some name BS_ and MBR_ refer to the same record and should use the same naming (MBR_).
Squashed commit of the following:
configs/viewtool-stm32f107: Clean up some compilation errors. Now all builds correctly and is ready for test.
arch/arm/src/stm32: Do not condition building STM32 USB host logic on CONFIG_USBHOST. That then precludes building STM32 platforms with external USB host chips like the MAX3421E. Add an new, STM32-internal variable CONFIG_STM32_USBHOST that does the same job without such side-effects.
configs/viewtool-stm32f107: Add support for MAX3421E USB host.
Squashed commit of the following:
sched: Rename all use of system_t to clock_t.
syscall: Rename all use of system_t to clock_t.
net: Rename all use of system_t to clock_t.
libs: Rename all use of system_t to clock_t.
fs: Rename all use of system_t to clock_t.
drivers: Rename all use of system_t to clock_t.
arch: Rename all use of system_t to clock_t.
include: Remove definition of systime_t; rename all use of system_t to clock_t.
Squashed commit of the following:
tools/gencromfs.c: Seems basically functional but has not has not yet been integrated with CROMFS.
tools/gencromfs.c: Revist previous commit. Let's make no assumptions about the compatibility of the host system and NuttX.
tools/gencromfs.c: I think access mode bits are sufficiently standard that we can just copy them from the host. This is don't mostly to pick up the executable bits and other conbinations of bits without doing a bit-for-bit decode.
tools/gencromfs.c: Code complete but only partially functional.
tools/gencromfs.c: type definitions must precede data definitions
tools/gencromfs.c: Adds a little more logic to gencromfs tools. Still not complete but can traverse directories and generate directory and hard-link nodes.
tools/gencromfs.c: Beginning of a genromfs tool that will eventually be used for generation of CROMFS file system images.
Squashed commit of the following:
fs/cromfs: Now depends on EXPERIMENTAL because it is not yet verified.
fs/cromfs: This commit brings the CROMFS file system to code complete. Still completely untested. Next steps: Need a tool to generate CROMFS file system images and a test case under apps/
fs/cromfs: Add logic to traverse the in-memory file system nodes.
fs/cromfs: Add initial support for an in-memory, compressed, read-only file system.
sched/sched: Correct some build issues introduced by last set of changes.
sched/sched: Add new internal OS function nxsched_setaffinity() that is identical to sched_isetaffinity() except that it does not modify the errno value. All usage of sched_setaffinity() within the OS is replaced with nxsched_setaffinity().
sched/sched: Internal functions sched_reprioritize() and sched_setpriority() no longer movidify the errno value. Also renamed to nxsched_reprioritize() and sched_setpriority().
sched/sched: Add new internal OS function nxsched_getscheduler() that is identical to sched_getscheduler() except that it does not modify the errno value. All usage of sched_getscheduler() within the OS is replaced with nxsched_getscheduler().
sched/sched: Add new internal OS function nxsched_setparam() that is identical to sched_setparam() except that it does not modify the errno value. All usage of sched_setparam() within the OS is replaced with nxsched_setparam().
sched/sched: Add new internal OS function nxsched_getparam() that is identical to sched_getparam() except that it does not modify the errno value (actually, the previous value erroneously neglected to set the errno value to begin with, but this fixes both issues). All usage of sched_getparam() within the OS is replaced with nxsched_getparam().
configs/flipnclick-sam3x: Add support for the 4 user-controllable LEDs on the board.
configs/flipnclick-sam3x: Add board support for the Mikroe Flip&Click SAM3X.
configs/arduino-due: Update this old configuration so that it is structured a little more like recent board support.
fs/nxffs: Giving up on truncate() support in NXFFS for now. There is too much that has to be done to make that work.
configs/sim: Add support for testing NXFFS.
fs/smartfs: Add logic needed to ftruncate to shrink a file to any arbitrary size.
fs/smartfs: Repartition some functionality to better fit in new ftruncate features.
FAT. Effectively handles the situation when a new file position is within the current sector.
* Find begin of the next token
* EOL
* revert
* Accelerates the work of the FS with a multitude of operations to write small pieces of data within the current sector.
Approved-by: Gregory Nutt <gnutt@nuttx.org>
Squashed commit of the following:
fs/fat: Resolves issues with truncating the cluster chain when shrinking files via ftruncate().
fs/fat: First cut at implementation file shrinkage logic needed to support ftruncate(). Certainly shrinks the file size but it does not appear to correctly disconnect the cluster chains.
fs/fat: Restructure some functions in files to better support forthcoming file shrinkage logic. Put framework for file shrinkage in place. That logic is incomplete on initial commit.
fs/nxffs: Add partial implementation of the truncate method: It extend files, but cannot yet shrink them.
fs/smartfs: Add partial implementation of the truncate method: It extend files, but cannot yet shrink them.
fs/fat: Add partial implementation of the truncate method: It extend files, but no yet shrink them.
fs/nfs: Add support for the truncate method to the NFS file system.
fs: Add truncate() support for userfs
fs/unionfs: Add truncate() support to the unionfs
fs/tmpfs: Add ftruncate() support to tmpfs
syscall/: Add system call support for ftruncate()
net/route: Adding ftruncate() support eliminates an issue in file-based routing table management.
fs: Add basic framework to support truncate() and ftruncate(). The infrastructure is complete. Now, however, the actual implementation of ftruncate() will have to be done for each file system.
This conversion is unfortunate in the sense that Unix local domain sockets are relatively lightweight. LocalHost UDP sockets are much heavier weight since they rely on the full UDP stack. If anyone is up for a complete redesign, then using some shared memory and a POSIX message queue would be lightweight again.
This commit also fixes several bugs that were not testable before the inode tree deadlock. I cannot say that the logic is 100% stable but it does not have basic functionality.
Squashed commit of the following:
fs/userfs: Order locking so that access to the shared I/O buffer is also locked.
fs/userfs: Converts to use LocalHost UDP loopback for IPC.
fs/userfs: This completes coding of the UserFS client and of the UserFS feature in general. This feature is being merged to main now because I believe it is innocuous. It is, however, untesed. The next step will be to develop a test case to verify the feature.
fs/userfs: Completes the request logic for the UserFS client. Still need the logic that receives the responses.
fs/userfs: Completes coding for most of the server side of the user filesystem logic.
fs/userfs: Big design changes, simplications. Use Unix domain local sockets instead of message queues. Easier to transfer big data in local sockets than message queues. Remove character drvier 'factory' it is not necessary.
fs/userfs: Minor reparitioning; volume private info does not need to be held on the OS client side.
libc/userfs: Add some of the server side logic.
fs/userfs: Add some UserFS initialization logic.
fs/userfs: Add frame work for the UserFS proxy. Remove all references to a block driver. There is no block dricer... what was I thinking?
fs/userfs: Add some initialization of the character driver, 'factory' device.
fs/userfs: Rename from fusefs to userfs to that we don't stomp on someone else's cool name.
Add a header file describing the fusefs interface.
This happened after adding these three new procfs entries:
fs/block
fs/mount
fs/usage
Of course, there is no directory fs in this case, only three files that have fs/ in their relative pathnames. The logic was detecting that fs was the name of the enty to report, but it was then declaring that fs was a file (because fs/block is of type file).
This was fixed by adding a check for matching lenghts. i.e., if strlen(fs) != strlen(fs/block), then report fs as a directory instead of a file.
fs/mount: Implements procfs /proc/fs/blocks and /proc/fs/usage files, replacing the NSH df command.
fs/mount: Implements procfs /proc/fs/mount file, replacing the NSH mount command when there are not arguments.
fs/: Move prototype of foreach_mountpoint out of include/nuttx/fs/fs.h to fs/mount/mount.h. Add framework for the mount procfs (initial commit is just a close of the net/route table procfs.
Change all calls to mq_send() and mq_timedsend() in the OS to calls to nxmq_send() and nxmq_timedsend(), making appropriate changes for differences in return values.
sched/mqueue: Add internal function nxmq_send() and nxmq_timedsend() that are equivalent to mq_send() and mq_timedsend() except that they do not create cancellation points and do to not modify the errno variable.
Replace all calls to sigqueue() in the OS proper with calls to nxsig_queue() to avoid accessing the errno variable.
sched/signal: Add nxsig_queue() which is functionally equivalent to sigqueue() except that it does not modify the errno variable.
This is analogous to similar renaming that was done previously for semaphores.
Squashed commit of the following:
sched/signal: Fix a few compile warnings introduced by naming changes.
sched/signal: Rename all private, internal signl functions to use the nxsig_ prefix.
sched/signal: Rename sig_removependingsignal, sig_unmaskpendingsignal, and sig_mqnotempty to nxsig_remove_pendingsignal, nxsig_unmask_pendingsignal, and nxsig_mqnotempty to make it clear that these are OS internal interfaces.
sched/signal: Rename sig_findaction and sig_lowest to nxsig_find_action and nxsig_lowest to make it clear that these are OS internal interfaces.
sched/signal: Rename sig_allocatepingsigaction and sig_deliver to nxsig_alloc_pendingsigaction and nxsig_deliver to make it clear that these are OS internal interfaces.
sched/signal: Rename sig_cleanup, sig_release, sig_releasependingaction, and sig_releasependingsignal to nxsig_cleanup, nxsig_release, nxsig_release_pendingaction, and nxsig_release_pendingsignal to make it clear that these are OS internal interfaces.
sched/signal: Rename sig_tcbdispatch and sig_dispatch to nxsig_tcbdispatch and nxsig_dispatch to make it clear that these are OS internal interfaces.
sched/signal: Rename sig_releaseaction and sig_pendingset to nxsig_release_action and nxsig_pendingset to make it clear that these are OS internal interfaces.
sched/signal: Rename sig_initialize and sig_allocateactionblock to nxsig_initialize and nxsig_alloc_actionblock to make it clear that these are OS internal interfaces.
This commit backs out most of commit b4747286b1. That change was added because sem_wait() would sometimes cause cancellation points inappropriated. But with these recent changes, nxsem_wait() is used instead and it is not a cancellation point.
In the OS, all calls to sem_wait() changed to nxsem_wait(). nxsem_wait() does not return errors via errno so each place where nxsem_wait() is now called must not examine the errno variable.
In all OS functions (not libraries), change sem_wait() to nxsem_wait(). This will prevent the OS from creating bogus cancellation points and from modifying the per-task errno variable.
sched/semaphore: Add the function nxsem_wait(). This is a new internal OS interface. It is functionally equivalent to sem_wait() except that (1) it is not a cancellation point, and (2) it does not set the per-thread errno value on return.
sched/semaphore: Add nxsem_post() which is identical to sem_post() except that it never modifies the errno variable. Changed all references to sem_post in the OS to nxsem_post().
sched/semaphore: Add nxsem_destroy() which is identical to sem_destroy() except that it never modifies the errno variable. Changed all references to sem_destroy() in the OS to nxsem_destroy().
libc/semaphore and sched/semaphore: Add nxsem_getprotocol() and nxsem_setprotocola which are identical to sem_getprotocol() and set_setprotocol() except that they never modifies the errno variable. Changed all references to sem_setprotocol in the OS to nxsem_setprotocol(). sem_getprotocol() was not used in the OS
libc/semaphore: Add nxsem_getvalue() which is identical to sem_getvalue() except that it never modifies the errno variable. Changed all references to sem_getvalue in the OS to nxsem_getvalue().
sched/semaphore: Rename all internal private functions from sem_xyz to nxsem_xyz. The sem_ prefix is (will be) reserved only for the application semaphore interfaces.
libc/semaphore: Add nxsem_init() which is identical to sem_init() except that it never modifies the errno variable. Changed all references to sem_init in the OS to nxsem_init().
sched/semaphore: Rename sem_tickwait() to nxsem_tickwait() so that it is clear this is an internal OS function.
sched/semaphoate: Rename sem_reset() to nxsem_reset() so that it is clear this is an internal OS function.
psock_close() and net_close() are internal OS functions and should not set the errno variable.
psock_ioctl() and netdev_ioctl() are internal OS functions and should not set the errno variable.
net_dupsd() and net_dupsd2() are internal OS functions and should not set the errno variable.
net/ and fs/: net_vfcntl(), file_fcntl(), file_dup(), and file_dup2() are all internal OS interfaces and should not modify the errno value.
net/: psock_recvfrom() is an internal interface and should not set the errno nor should it be a cancellation point.
net/: psock_accept() is not a cancellation point.
net/: psock_getsockopt() and psock_socket*9 are an internal interfaces and should not set the errno.
net/: psock_getsockopt() is an internal interface and should not set the errno.
net/: psock_listen() is an internal interface and should not set the errno.
net/: psock_connect(( is an internal interface and should not set the errno nor should it be a cancellation point.
net/: psock_bind() is an internal interface and should not set the errno.
net/: psock_accept() is an internal interface and should not set the errno.
fs/vfs:null check for path on open and buf on write
Null path check is depend on CONFIG_DEBUG_FEATURES and CONFIG_DEBUG_ASSERTIONS, added null checking so it's always performed
Added null checking on buf for write()
Approved-by: Gregory Nutt <gnutt@nuttx.org>
There are two issues with it however:
1. Reading from file system on a per packet basis could be slow. I think it probably should have a small, in-memory cache of most frequently used routes for good problem.
2. Currently the delroute logic is disabled due to a problem with the design. NuttX does not currently support truncate(). Therefore, it is not possible to delete entries from the routing table file.
In this current implementation, that leaves the last entry intact at the end of the file. An alternative design might include a tag on each record to indicate if the record is valid or not. That would work but would add complexity to the other routing table functions.
The existing implementation is available for testing purposes only if CONFIG_EXPERIMENTAL=y.
Squashed commit of the following:
net/route: The current delroute design depends on file truncation. However, NuttX does not currently support truncate. Alternative, more complex designs are possible but not implemented. At present the file routing table are otherwise functional but the delroute logic is disabled via CONFIG_EXPERIMENTAL. Enable it only if you plan to fix it.
net/route: Fix some issues with locking routing table files.
net/route: Add partial implementation of delroute for the case where the routing table is in a file.
net/route: Add support for seeking to positions in the routing table.
net/route: Add net_addroute_ipv4/6() using a file-based routing table.
net/route: Add net_foreach_ipv4/4() using a file-based routing table.
net/route: Initial build, configuration, and fs utilies to support routing tables in a file.
