fs: Add VFS docs

This commit adds VFS docmentation, and details about the VFS interface.

Signed-off-by: Saurav Pal <resyfer.dev@gmail.com>
This commit is contained in:
Saurav Pal 2024-02-20 18:35:43 +00:00 committed by Xiang Xiao
parent 81d3e123cb
commit 93d03d0418
6 changed files with 529 additions and 13 deletions

View File

@ -23,10 +23,12 @@
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?= -j 1 -W -A nuttx_versions="latest,${NUTTX_VERSIONS}"
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build
SPHINXOPTS ?= -j 1 -W -A nuttx_versions="latest,${NUTTX_VERSIONS}"
SPHINXAUTOOPTS ?= -j 8 -W
SPHINXBUILD ?= sphinx-build
SPHINXAUTOBUILD ?= sphinx-autobuild
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@ -37,7 +39,7 @@ help:
html: clean
autobuild: clean
sphinx-autobuild . _build -j=8
@$(SPHINXAUTOBUILD) "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXAUTOOPTS) $(O)
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).

View File

@ -147,7 +147,7 @@ the volume is given to a file.
File attributes
--------------------
File attributes are denoted by a bit flag of the size of a single bit.
File attributes are denoted by a bit flag of the size of a single byte.
The file flags in FAT, with their bit representation, are as follows:
.. list-table:: File Attributes

View File

@ -2,9 +2,10 @@
NuttX File System
=================
**Overview**. NuttX includes an optional, scalable file system.
This file-system may be omitted altogether; NuttX does not depend
on the presence of any file system.
NuttX includes an optional, scalable file system. This file-system may be
omitted altogether; NuttX does not depend on the presence of any file system.
.. _root_fs:
**Pseudo Root File System**. A simple *in-memory*, *pseudo* file
system can be enabled by default. This is an *in-memory* file
@ -18,8 +19,7 @@ system).
Any user supplied data or logic can be accessed via the
pseudo-file system. Built in support is provided for character and
block `drivers <#DeviceDrivers>`__ in the ``/dev`` pseudo file
system directory.
block drivers in the ``/dev`` pseudo file system directory.
**Mounted File Systems** The simple in-memory file system can be
extended my mounting block devices that provide access to true
@ -41,6 +41,492 @@ pseudo file systems may be mounted in the true, root file system.
The approach selected by NuttX is intended to support greater
scalability from the very tiny platform to the moderate platform.
Virtual File System (VFS)
=========================
Virtual File System provides a unified interface for various file systems to
be able to co-exist together by exposing a blueprint that each file system
needs to implement. This also allows the file system to be free from worry
about the device driver implementations for storage devices, as they also
expose a unified way of accessing the underlying devices.
How VFS works
-------------
Threads are controllable sequences of instruction execution with their own
stacks. Each task in NuttX is represented by a Task Control Block (TCB) (TCB
is defined in ``include/nuttx/sched.h``) and tasks are organized in task
lists.
All threads that are created by ``pthread_create()`` are part of the same task
group. A task group (defined in ``include/nuttx/sched.h``) is a shared
structure pointed to by the TCBs of all the threads that belong to the same
task group, and this task group contains all the resources shared across the
task group which includes *file descriptors* in the form of a **file list**.
A file list (defined in ``include/nuttx/fs/fs.h``) contains file structures
that denote open files (along with a spinlock to manage access to the file
list). With the devices listed in the :ref:`root file system <root_fs>` (on
points like ``/dev/led``, ``/dev/mmcsd0``, etc. which are henceforth called
blockdriver mount points) in an unmounted state, storage devices can be
mounted using the ``mount()`` command (to any point like ``/dir/abcd``) with
any specific supported file system, which internally calls its implemented
``mountpt_operations->bind()`` method and passes the blockdriver's mount
point inode to it, thus creating a **mount point**. The blockdriver mount
point inode will have a ``mountpt->i_private`` which contains any (file system
dependent) information about the mount and is to be filled by the file system
during the execution of ``mountpt_operations->bind()`` (and usually this data
includes a pointer to the blockdriver mount point as well). After that,
according to system calls, the other exposed functions of the filesystem
are called as per need.
VFS Interface
-------------
VFS allows file systems to expose their own implementations of methods
belonging to a unified interface:
* **File operations**
.. c:function:: int open(FAR struct file *filep, FAR const char *relpath, int oflags, mode_t mode)
Opens a file. Files are required to be opened before any other file
operations are performed on it.
:param FAR struct file * filep: Open file's file structure pointer. The
``filep->f_priv`` member needs to be set here with the file system
specific data that represents an open file.
:param FAR const char * relpath: Relative path of the file from the root of
the mounted file system.
:param int oflags: Flags in a bit field that specify the mode for openning
the file (eg. ``O_RDONLY``, ``O_RDWR``, etc. defined in
``include/fcntl.h``).
:param mode_t mode: Specifies the mode (permissions). If ``oflags`` include
``O_CREAT``, then this contains the mode for the file to be created.
:returns: Status of openning a file.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int close(FAR struct file *filep)
This closes the opened file, and ideally syncs all the changes to the file
to be written to the disk, as well as free the memory allocated to store the
open file's data.
:param FAR struct file * filep: Open file's file structure pointer.
:returns: Status of closing a file.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: ssize_t read(FAR struct file *filep, FAR char *buffer, size_t buflen)
Reads maximum ``buflen`` bytes from an opened file (from the current offset
the opened file descriptor is pointing at if the file system supports
seeking).
:param FAR struct file * filep: Open file's file structure pointer.
:param FAR char * buffer: Buffer to store the read data.
:param size_t buflen: Length of the maximum number of bytes to be read.
:returns: Number of bytes read.
:retval > 0: Size of bytes read.
:retval < 0: Error.
.. c:function:: ssize_t write(FAR struct file *filep, FAR const char *buffer, size_t buflen)
Writes maximum ``buflen`` bytes to an opened file (from the current offset
the opened file is at if the file system supports seeking).
:param FAR struct file * filep: Open file's file structure pointer.
:param FAR char * buffer: Buffer which contains the data to be written.
:param size_t buflen: Length of the maximum number of bytes to be written.
:returns: Number of bytes written.
:retval > 0: Size of bytes written.
:retval < ``buflen``: Insufficient storage or file size limit reached
:retval < 0: Error.
.. NOTE::
POSIX requires that a ``read()`` after a ``write()`` should get the newly
written data, but not all file systems conform to POSIX, especially as
POSIX requires atomic writes, which is not usually implemented
as it can impact performance.
To be POSIX compliant in concurrent situations, either the writes have to
be atomic, or read is blocked with a lock until an on-going write is
finished, which, as stated, would impact performance.
.. c:function:: off_t seek(FAR struct file *filep, off_t offset, int whence)
Underlying implementation of ``lseek()``, it allows the open file's file
structure to point to any particular location in the file.
:param FAR struct file * filep: Open file's file structure pointer.
:param off_t offset: The offset required.
:param int whence: This controls how the offset it applied. It can have
values (defined in ``/include/sys/types.h``):
* SEEK_SET: Offset from start of file.
* SEEK_CUR: Offset from current location in file.
* SEEK_END: Offset *after* end of file.
.. NOTE::
According to POSIX, ``lseek()`` to any point after the end of the file
*does not* by itself increase the size of the file. Later writes to this
part will, however, increase it to at least the end of the written data, and
the "gap" before this written data should be filled with ``\0`` in case of
any reads after such a write operation.
.. c:function:: int ioctl(FAR struct file *filep, int cmd, unsigned long arg)
It is the underlying implementation of ``ioctl()`` (I/O Control).
``ioctl()`` manipulates the underlying device parameters of files.
:param FAR struct file * filep: Open file's file structure pointer.
:param int cmd: It can take a variety of values (which are defined in
``include/nuttx/fs/ioctl.h``). It represents the command that will be
carried out on the file. Both the filesystem, as well as the device driver
needs to support the command in order for the function to run.
:param unsigned long arg: Additional argument that may be required for
ioctl. Details for what is required is written in the comments beside
the desired ioctl command in ``include/nuttx/fs/ioctl.h``.
:returns: Status of ioctl operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int mmap(FAR struct file *filep, FAR struct mm_map_entry_s *map)
Underlying implementation of ``mmap()``. ``mmap()`` creates a new mapping
in the virtual address space of the calling process.
:param FAR struct file * filep: Open file's file structure pointer.
:param FAR struct mm_map_entry_s * map: mmap entry strucutre pointer, which
includes the virtual address.
:returns: Status of mmap operation.
:retval OK (0): Success.
:retval < 0: Error.
.. NOTE::
NuttX operates in a flat open address space. Therefore, it generally does
not require ``mmap()`` functionality. There are two notable exceptions where
``mmap()`` functionality is required:
1. ``mmap()`` is the API that is used to support direct access to random
access media under the following very restrictive conditions:
a. The filesystem implements the mmap file operation. Any file
system that maps files contiguously on the media should support
this ioctl. (vs. file system that scatter files over the media
in non-contiguous sectors). As of this writing, ROMFS is the
only file system that meets this requirement.
b. The underlying block driver supports the BIOC_XIPBASE ioctl
command that maps the underlying media to a randomly accessible
address. At present, only the RAM/ROM disk driver does this.
2. If CONFIG_FS_RAMMAP is defined in the configuration, then mmap() will
support simulation of memory mapped files by copying files whole
into RAM.
.. c:function:: int truncate(FAR struct file *filep, off_t length)
Shrinks or expands the file to be of the desired size.
:param FAR struct file * filep: Open file's file structure pointer.
:param off_t length: Final size of the file.
:returns: Status of truncate operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int poll(FAR struct file *filep, FAR struct pollfd *fds, bool setup)
Underlying implementation of ``poll()``. The ``poll()`` function provides
applications with a mechanism for multiplexing input/output over a set of
file descriptors.
:param FAR struct file * filep: Open file's pointer.
:param FAR struct pollfd * fds: The structure describing the events to be
monitored, OR NULL if this is a request to stop monitoring events.
:param bool setup: true: Setup up the poll; false: Teardown the poll
:returns: Status of poll operation.
:retval OK (0): Success.
:retval < 0: Error.
* **Additional open file specific operations**
.. c:function:: int sync(FAR struct file *filep)
This synchronizes the on-disk file system state of the file with the
in-memory file system state, ie. commits the file system caches to the disk.
:param FAR struct file * filep: Open file's ``struct file`` (defined in
``include/nuttx/fs/fs.h``) pointer.
:returns: Status of syncing a file.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int dup(FAR const struct file *oldp, FAR struct file *newp)
Duplicate an open file structure.
:param FAR const struct file * oldp: Pointer to structure that is to be
duplicated.
:param FAR struct file * newp: Pointer to structure in which the duplicate
data will be stored.
:returns: Status of duplicating open file's structure.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int fstat(FAR const struct file *filep, FAR struct stat *buf)
Obtain information about an open file.
:param FAR const struct file * filep: Open file's pointer.
:param FAR struct stat * buf: Pointer to the ``struct stat`` (defined in
``include/sys/stat.h``).
:returns: Status of obtaining open file's information.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int fchstat(FAR const struct file *filep, FAR const struct stat *buf, int flags)
Change file stats. It can change the mode, timestamps and ownership.
:param FAR struct file * filep: Open file's pointer.
:param FAR const struct stat * buf: Pointer to stat structure describing
the values that need to be updated.
:param int flags: Bit field that can include (defined in
``include/nuttx/fs/fs.h``):
* ``CH_STAT_MODE``
* ``CH_STAT_UID``
* ``CH_STAT_GID``
* ``CH_STAT_ATIME``
* ``CH_STAT_MTIME``
This describes what needs to be updated.
:returns: Status of changin open file's stats.
:retval OK (0): Success.
:retval < 0: Error.
* **Directory operations**
.. c:function:: int opendir(FAR struct inode *mountpt, FAR const char *relpath, FAR struct fs_dirent_s **dir)
Opens a directory stream for the provided directory. Other directory
operations can be used after this to do various directory related operations
. We say the directory stream points to the first entry, but you need
``readdir()`` to read the first entry.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * relpath: Relative path from the root of the point
point of the directory.
:param FAR struct fs_dirent_s ** dir: A directory stream structure pointer
which needs to be populated with the required fields (defined in
``include/nuttx/fs/fs.h``).
:returns: Status of openning the directory.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int closedir(FAR struct inode *mountpt, FAR struct fs_dirent_s *dir)
Closes a directory stream, as well as deallocates any memory used while
while openning a directory stream.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR struct fs_dirent_s ** dir: A directory stream structure pointer
which was previously allocated (and needs to be freed).
:returns: Status of closing the directory.
:retval OK (0): Success.
.. c:function:: int readdir(FAR struct inode *mountpt, FAR struct fs_dirent_s *dir, FAR struct dirent *entry)
This reads the next directory entry in a directory stream. If the stream
points to the base of the directory, then the first directory entry in the
directory is given.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR struct fs_dirent_s ** dir: A directory stream structure pointer.
:param FAR struct dirent * entry: Pointer to the directory entry. This will
be modified to point to the directory entry after it in the directory.
:returns: Status of reading the directory.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int rewinddir(FAR struct inode *mountpt, FAR struct fs_dirent_s *dir)
Resets the directory stream back to the first entry, like it was after
openning.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR struct fs_dirent_s ** dir: A directory stream structure pointer.
:returns: Status of rewinding the directory.
:retval OK (0): Success.
:retval < 0: Error.
* **Volume-relations operations**
.. c:function:: int bind(FAR struct inode *blkdriver, FAR const void *data, FAR void **handle)
This is where the file system related data is initialized, and
is part of the mount process.
:param FAR struct inode * blkdriver: Pointer to the block driver's device
inode. This needs to be opened in this function.
:param FAR const void * data: The options provided during mount.
:param FAR void ** handle: Whatever data ``handle`` points to is attached
to the ``mountpt`` inode after this function is called during the mount
process. This way, this file system's other methods can receive
this information if they have access to ``mountpt`` inode, by accessing
``mountpt->i_private``.
:returns: Status of binding operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int unbind(FAR void *handle, FAR struct inode **blkdriver, unsigned int flags)
This is part of the unmounting process. The file system first
needs to assess the flags passed to it and appropriately do the tasks
required by these flags, and then it needs to free the private data
(``handle`` and any allocated members), as well as close the
previously-opened (during mount) block driver's inode.
:param FAR void * handle: Private data of the file-system.
:param FAR struct inode ** blkdriver: The device inode of the block driver's
device inode.
:param unsigned int flags: Flags dictate the actions needed to be carried
out before the file system data is removed and the block driver inode is
closed. The values can be (as defined in ``include/sys/mount.h``):
* ``MNT_FORCE``
* ``MNT_DETACH``
* ``MNT_EXPIRE``
* ``UMOUNT_NOFOLLOW``
:returns: Status of unbinding operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int statfs(FAR struct inode *mountpt, FAR struct statfs *buf)
Provides stats for that instance of the file system. The exact
stats that are provided can be viewed in the members of
``struct statfs`` (in file ``include/sys/statfs.h``).
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR struct statfs * buf: Buffer that needs to be filled with the
relevant file system information.
:returns: Status of finding the filesystem stats operation.
:retval OK (0): Success.
:retval < 0: Error.
* **Path operations**
.. c:function:: int unlink(FAR struct inode *mountpt, FAR const char *relpath)
Removes a file, specifically, removes a name from the file system.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * relpath: The relative path of the file from the root
of the file system.
:returns: Status of unlinking operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int mkdir(FAR struct inode *mountpt, FAR const char *relpath, mode_t mode)
Creates a directory.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * relpath: Relative path of the new directory from the
root of the file system.
:param mode_t mode: The mode (permissions) for the directory.
:returns: Status of creating a directory operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int rmdir(FAR struct inode *mountpt, FAR const char *relpath)
Removes a directory.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * relpath: Relative path of the directory from the
root of the file system.
:returns: Status of removing a directory operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int rename(FAR struct inode *mountpt, FAR const char *oldrelpath, FAR const char *newrelpath)
Renames a file or a directory
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * oldrelpath: Existing path of the file or directory.
:param FAR const char * newrelpath: New path of the file or directory.
:returns: Status of renaming a file or a directory operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int stat(FAR struct inode *mountpt, FAR const char *relpath, FAR struct stat *buf)
Information about a file or a directory. The exact information that is
provided can be viewed in the members of ``struct stat``
(in file ``include/sys/stat.h``).
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * relpath: Relative path of the file or directory from
the root of the file system.
:param FAR struct stat * buf: Buffer that needs to be filled with the
relevant file or directory information.
:returns: Status of finding information about a file or directory.1 operation.
:retval OK (0): Success.
:retval < 0: Error.
.. c:function:: int chstat(FAR struct inode *mountpt, FAR const char *relpath, FAR const struct stat *buf, int flags)
Change the stats of a file or directory.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:param FAR const char * relpath: Relative path of the file or directory from
the root of the file system.
:param FAR const struct stat * buf: Contains the new stat information. Access
only the ones that are required according to the flags.
:param int flags: A bit field that can have values including ``CH_STAT_MODE``
, ``CH_STAT_UID``, ``CH_STAT_GID``, ``CH_STAT_ATIME`` or ``CH_STAT_MTIME``
which are or-ed together.
.. c:function:: int syncfs(FAR struct inode *mountpt)
This works like ``sync()`` but instead of the file, it syncs the entire
filesystem's metadata.
:param FAR struct inode * mountpt: Mount point inode of the file system.
:returns: Status of syncing file system metadata operation.
:retval OK (0): Success.
:retval < 0: Error.
The file systems can have their own implementations for these functions
under-the-hood, but the user does not have to worry about the underlying file
system during file I/O, as the file system has to expose its implementations
in a unified interface.
.. NOTE::
Each file system has to globally expose their implementations of the unified
interface as defined by ``struct mountpt_operations`` (in
``include/fs/fs.h``) to one of the lists defined in ``fs/mount/fs_mount.c``
depending on the type of the file system.
They also need their own `magic number <https://en.wikipedia.org/wiki/Magic_number_(programming)>`_
to be listed in ``include/sys`` and in ``fs_gettype`` function (in
``fs/mount/fs_gettype.c``) for identification of the filesystem.
File systems
============
NuttX provides support for a variety of file systems out of the box.
.. toctree::
:maxdepth: 1
@ -65,3 +551,18 @@ scalability from the very tiny platform to the moderate platform.
unionfs.rst
userfs.rst
zipfs.rst
FS Categories
-------------
File systems can be divided into these categories on the basis of the drivers
they require:
1. They require a block device driver. They include vfat, romfs, smartfs, and
littlefs.
2. They require MTD drivers. They include romfs, spiffs, littlefs.
3. They require neither block nor MTD drivers. They include nxffs, tmpfs, nfs
binfs, procfs, userfs, hostfs, cromfs, unionfs, rpmsgfs, and zipfs.
The requirements are specified by declaring the filesystem in the proper
array in ``fs/mount/fs_mount.c``.

View File

@ -3,4 +3,15 @@
# Function and (enum or struct) name
.*Duplicate C declaration.*\n.*'\.\. c:.*:: net_driver_s'.*
.*Duplicate C declaration.*\n.*'\.\. c:.*:: sigaction'.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*sigaction.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*open.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*close.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*read.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*write.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*ioctl.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*mmap.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*poll.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*dup.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*rewinddir.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*bind.*
.*Duplicate C declaration.*\n.*'\.\. c:.*::.*unlink.*

View File

@ -1,3 +1,5 @@
.. _notifier_chain:
==============
Notifier Chain
==============

View File

@ -36,7 +36,7 @@
* Name: file_syncfs
*
* Description:
* Equivalent to the standard syncsf() function except that is accepts a
* Equivalent to the standard syncfs() function except that is accepts a
* struct file instance instead of a fd descriptor and it does not set
* the errno variable
*