NXFFS README
^^^^^^^^^^^^
This README file contains information about the implementation of the NuttX
wear-leveling FLASH file system, NXFFS.
Contents:
General NXFFS organization
General operation
Headers
NXFFS Limitations
Multiple Writers
ioctls
Things to Do
General NXFFS organization
==========================
The following example assumes 4 logical blocks per FLASH erase block. The
actual relationship is determined by the FLASH geometry reported by the MTD
driver.
ERASE LOGICAL Inodes begin with a inode header. inode may
BLOCK BLOCK CONTENTS be marked as "deleted," pending re-packing.
n 4*n --+--------------+
|BBBBBBBBBBBBBB| Logic block header
|IIIIIIIIIIIIII| Inodes begin with a inode header
|DDDDDDDDDDDDDD| Data block containing inode data block
| (Inode Data) |
4*n+1 --+--------------+
|BBBBBBBBBBBBBB| Logic block header
|DDDDDDDDDDDDDD| Inodes may consist of multiple data blocks
| (Inode Data) |
|IIIIIIIIIIIIII| Next inode header
| | Possibly a few unused bytes at the end of a block
4*n+2 --+--------------+
|BBBBBBBBBBBBBB| Logic block header
|DDDDDDDDDDDDDD|
| (Inode Data) |
4*n+3 --+--------------+
|BBBBBBBBBBBBBB| Logic block header
|IIIIIIIIIIIIII| Next inode header
|DDDDDDDDDDDDDD|
| (Inode Data) |
n+1 4*(n+1) --+--------------+
|BBBBBBBBBBBBBB| Logic block header
| | All FLASH is unused after the end of the final
| | inode.
--+--------------+
General operation
=================
Inodes are written starting at the beginning of FLASH. As inodes are
deleted, they are marked as deleted but not removed. As new inodes are
written, allocations proceed to toward the end of the FLASH -- thus,
supporting wear leveling by using all FLASH blocks equally.
When the FLASH becomes full (no more space at the end of the FLASH), a
re-packing operation must be performed: All inodes marked deleted are
finally removed and the remaining inodes are packed at the beginning of
the FLASH. Allocations then continue at the freed FLASH memory at the
end of the FLASH.
Headers
=======
BLOCK HEADER:
The block header is used to determine if the block has every been
formatted and also indicates bad blocks which should never be used.
INODE HEADER:
Each inode begins with an inode header that contains, among other things,
the name of the inode, the offset to the first data block, and the
length of the inode data.
At present, the only kind of inode support is a file. So for now, the
term file and inode are interchangeable.
INODE DATA HEADER:
Inode data is enclosed in a data header. For a given inode, there
is at most one inode data block per logical block. If the inode data
spans more than one logical block, then the inode data may be enclosed
in multiple data blocks, one per logical block.
NXFFS Limitations
=================
This implementation is very simple as, as a result, has several limitations
that you should be aware before opting to use NXFFS:
1. Since the files are contiguous in FLASH and since allocations always
proceed toward the end of the FLASH, there can only be one file opened
for writing at a time. Multiple files may be opened for reading.
2. Files may not be increased in size after they have been closed. The
O_APPEND open flag is not supported.
3. Files are always written sequential. Seeking within a file opened for
writing will not work.
4. There are no directories, however, '/' may be used within a file name
string providing some illusion of directories.
5. Files may be opened for reading or for writing, but not both: The O_RDWR
open flag is not supported.
6. The re-packing process occurs only during a write when the free FLASH
memory at the end of the FLASH is exhausted. Thus, occasionally, file
writing may take a long time.
7. Another limitation is that there can be only a single NXFFS volume
mounted at any time. This has to do with the fact that we bind to
an MTD driver (instead of a block driver) and bypass all of the normal
mount operations.
Multiple Writers
================
As mentioned in the limitations above, there can be only one file opened
for writing at a time. If one thread has a file opened for writing and
another thread attempts to open a file for writing, then that second
thread will be blocked and will have to wait for the first thread to
close the file.
Such behavior may or may not be a problem for your application, depending
(1) how long the first thread keeps the file open for writing and (2) how
critical the behavior of the second thread is. Note that writing to FLASH
can always trigger a major FLASH reorganization and, hence, there is no
way to guarantee the first condition: The first thread may have the file
open for a long time even if it only intends to write a small amount.
Also note that a deadlock condition would occur if the SAME thread
attempted to open two files for writing. The thread would would be
blocked waiting for itself to close the first file.
ioctls
======
The file system supports to ioctls:
FIOC_REFORMAT: Will force the flash to be erased and a fresh, empty
NXFFS file system to be written on it.
FIOC_OPTIMIZE: Will force immediate repacking of the file system. This
will avoid the delays to repack the file system in the emergency case
when all of the FLASH memory has been used. Instead, you can defer
the garbage collection to time when the system is not busy. Calling
this function on a thrashing file system will increase the amount of
wear on the FLASH if you use this frequently!
Things to Do
============
- The statfs() implementation is minimal. It should have some calculation
of the f_bfree, f_bavail, f_files, f_ffree return values.
- There are too many allocs and frees. More structures may need to be
pre-allocated.
- The file name is always extracted and held in allocated, variable-length
memory. The file name is not used during reading and eliminating the
file name in the entry structure would improve performance.
- There is a big inefficiency in reading. On each read, the logic searches
for the read position from the beginning of the file each time. This
may be necessary whenever an lseek() is done, but not in general. Read
performance could be improved by keeping FLASH offset and read positional
information in the read open file structure.
- Fault tolerance must be improved. We need to be absolutely certain that
any FLASH errors do not cause the file system to behavior incorrectly.
- Wear leveling might be improved (?). Files are re-packed at the front
of FLASH as part of the clean-up operation. However, that means the files
that are not modified often become fixed in place at the beginning of
FLASH. This reduces the size of the pool moving files at the end of the
FLASH. As the file system becomes more filled with fixed files at the
front of the device, the level of wear on the blocks at the end of the
FLASH increases.
- When the time comes to reorganization the FLASH, the system may be
unavailable for a long time. That is a bad behavior. What is needed,
I think, is a garbage collection task that runs periodically so that
when the big reorganization event occurs, most of the work is already
done. That garbage collection should search for valid blocks that no
longer contain valid data. It should pre-erase them, put them in
a good but empty state... all ready for file system re-organization.
NOTE: There is the FIOC_OPTIMIZE IOCTL command that can be used by an
application for force garbage collection when the system is not busy.
If used judiciously by the application, this can eliminate the problem.
- And worse, when NXFSS reorganization the FLASH a power cycle can
damage the file system content if it happens at the wrong time.
- The current design does not permit re-opening of files for write access
unless the file is truncated to zero length. This effectively prohibits
implementation of a proper truncate() method which should alter the
size of a previously written file. There is some fragmentary logic in
place but even this is conditioned out with __NO_TRUNCATE_SUPPORT__.
