ec498d2660
Squashed commit of the following:
fs/spiffs: Fix last compilation issue. Now compiles without error. It is still not quite ready for testing as there is additional code review that must be be performed. It is now marked as EXPERIMENTAL so that it can be brought onto the master branch with little risk.
fs/spiffs: Remove some dead code.
fs/spiffs: Weak start of analysis of spiffs_nucleus.c. Renamed to spiffs_core.c
fs/spiffs: Rename spiffs_nucleus.c to spiffs_core.c
fs/spiffs: Remove spiffs_config.h. All configuration settings are now available in the SPIFFS Kconfig options.
fs/spiffs: Finished review, update, and repartitioning of spiffs_check.c. Added spiffs_check.h.
fs/spiffs: Finished review, update, and repartitioning of spiffs_cache.c. Added spiffs_cache.h.
fs/spiffs: Clean up some defines used in debug output statements.
fs/spiffs: Finished review, update, and repartitioning of spiffs_gc.c. Added spiffs_gc.h.
fs/spiffs: Now that VFS interface is completed, I have begun the long march of repartitioning the remaining functionality, reviewing logic, identifying dead code, and cleaning up loose ends.
fs/spiffs: Initial integration of MTD interface, replacing the SPIFFS native flash interface. Lots of open issues such as the use of pages vs. blocks vs. erase blocks and units of addresses, offsets, and lengths that are passed in function calls. Remove SPIFFS_USE_MAGIC support. That option (which default to OFF anyway), wrote a magic value at the beginning of every sector and support verifiable identification of the file system. It was not being and used and removing it makes life simpler.
fs/spiffs: Remove semaphore lock on the file object structure. Ultimately, the file access must modify the volume and access the volume structue which also has a exclusivity lock. So use of the volume lock alone should be sufficient.
Integrated the SPIFFS rename logic into the NuttX VFS. Removed non-standard application calls or convert them to IOCTL commands. These were converted to IOCTL commands: (1) integrity check, (2) garbage collection, and (3) format flash. These were removed: (1) Integrity check callback. These provided a lot of good information about the state of the file system, but such callbacks are not compatible with a POSIX compliant file system. (2) Index maps. The index maps were a performance improvement feature. The user could provide the memory and request that a region of a a file use that memory for improved lookup performance when accessing parts of the file. The fallback is the less performance lookup by traversing the FLASH memory. (3) Removed the quick garbage collection interface (the code is still used internally). Only the full garbage collection is available to the user application via IOCTL.
configs/sim/spiffs: A simulator configuration to use for testing SPIFFS.
fs/spiffs: Integrate SPIFFS logic into NuttX VFS bind() and unbind() methods.
fs/mount/fs_mount.c: Add SPIFFS to the list of drivers that require MTD vs block drivers.
fs/spiffs: Trivial changes, mostly from analysis of how to integrate the rename() VFS method.
fs/spiffs: Connect NuttX VFS unlink method to the SPIFFS_remove() function. Lots of name-changing.
fs/spiffs: Remove non-standard errno support. Remove bogus SPIFFS_LOCK() and SPIFFS_UNLOCK() macros.
fs/spiffs: Add NuttX VFS implementation for statfs() method. Clean up some of the accumulating compilation problems.
fs/spiffs: Add stat(), truncate() methods. Dummy out unsupport mkdir() and rmdir() methods.
fs/spiffs: Replace some of the custom error numbers with standard error numbers.
fs/spiffs: Hooks read(), write(), fstat(), ioctl(), opendir(), closedir(), rewindif(), and readdir() into the NuttX VFS.
fs/spiffs: Beginning the organization to work with the NuttX VFS. Lots of things are get broken!
fs/spiffs: Add spiffs.c which will be the interface between SPIFFS and NuttX. No very close at present, however.
fs/spiffs: Clean up some compile problems introduced by coding standard changes.
fs/spiffs: A little closer to NuttX coding standard.
fs/spiffs: Ran tools/indent.sh against all files. Closer to NuttX coding standard, but needs a lot more effort to be fully compliant.
fs/spiffs: This commit brings in version 0.3.7 of Peter Anderson's SPIFFS. The initial commit includes the core FS files (with some definitions destributed to their correct header files) and hooks into the build system.
216 lines
10 KiB
Markdown
216 lines
10 KiB
Markdown
# SPIFFS (SPI Flash File System)
|
|
**V0.3.7**
|
|
|
|
[](https://travis-ci.org/pellepl/spiffs)
|
|
|
|
Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976 at gmail.com)
|
|
|
|
For legal stuff, see [LICENSE](https://github.com/pellepl/spiffs/blob/master/LICENSE). Basically, you may do whatever you want with the source. Use, modify, sell, print it out, roll it and smoke it - as long as I won't be held responsible.
|
|
|
|
Love to hear feedback though!
|
|
|
|
|
|
## INTRODUCTION
|
|
|
|
Spiffs is a file system intended for SPI NOR flash devices on embedded targets.
|
|
|
|
Spiffs is designed with following characteristics in mind:
|
|
- Small (embedded) targets, sparse RAM without heap
|
|
- Only big areas of data (blocks) can be erased
|
|
- An erase will reset all bits in block to ones
|
|
- Writing pulls one to zeroes
|
|
- Zeroes can only be pulled to ones by erase
|
|
- Wear leveling
|
|
|
|
|
|
## BUILDING
|
|
|
|
`mkdir build; make`
|
|
|
|
Otherwise, configure the `builddir` variable towards the top of `makefile` as something opposed to the default `build`. Sanity check on the host via `make test` and refer to `.travis.yml` for the official in-depth testing procedure. See the wiki for [integrating](https://github.com/pellepl/spiffs/wiki/Integrate-spiffs) spiffs into projects and [spiffsimg](https://github.com/nodemcu/nodemcu-firmware/tree/master/tools/spiffsimg) from [nodemcu](https://github.com/nodemcu) is a good example on the subject.
|
|
|
|
|
|
## FEATURES
|
|
|
|
What spiffs does:
|
|
- Specifically designed for low ram usage
|
|
- Uses statically sized ram buffers, independent of number of files
|
|
- Posix-like api: open, close, read, write, seek, stat, etc
|
|
- It can run on any NOR flash, not only SPI flash - theoretically also on embedded flash of a microprocessor
|
|
- Multiple spiffs configurations can run on same target - and even on same SPI flash device
|
|
- Implements static wear leveling
|
|
- Built in file system consistency checks
|
|
- Highly configurable
|
|
|
|
What spiffs does not:
|
|
- Presently, spiffs does not support directories. It produces a flat structure. Creating a file with path *tmp/myfile.txt* will create a file called *tmp/myfile.txt* instead of a *myfile.txt* under directory *tmp*.
|
|
- It is not a realtime stack. One write operation might last much longer than another.
|
|
- Poor scalability. Spiffs is intended for small memory devices - the normal sizes for SPI flashes. Going beyond ~128Mbyte is probably a bad idea. This is a side effect of the design goal to use as little ram as possible.
|
|
- Presently, it does not detect or handle bad blocks.
|
|
- One configuration, one binary. There's no generic spiffs binary that handles all types of configurations.
|
|
|
|
## NOTICE
|
|
|
|
0.4.0 is under construction. This is a full rewrite and will change the underlying structure. Hence, it will not be compatible with earlier versions of the filesystem. The API is the same, with minor modifications. Some config flags will be removed (as they are mandatory in 0.4.0) and some features might fall away until 0.4.1. If you have any worries or questions, it can be discussed in issue [#179](https://github.com/pellepl/spiffs/issues/179)
|
|
|
|
## MORE INFO
|
|
|
|
See the [wiki](https://github.com/pellepl/spiffs/wiki) for [configuring](https://github.com/pellepl/spiffs/wiki/Configure-spiffs), [integrating](https://github.com/pellepl/spiffs/wiki/Integrate-spiffs), [using](https://github.com/pellepl/spiffs/wiki/Using-spiffs), and [optimizing](https://github.com/pellepl/spiffs/wiki/Performance-and-Optimizing) spiffs.
|
|
|
|
For design, see [docs/TECH_SPEC](https://github.com/pellepl/spiffs/blob/master/docs/TECH_SPEC).
|
|
|
|
For a generic spi flash driver, see [this](https://github.com/pellepl/spiflash_driver).
|
|
|
|
## HISTORY
|
|
|
|
### 0.3.7
|
|
- fixed prevent seeking to negative offsets #158
|
|
- fixed file descriptor offsets not updated for multiple fds on same file #157
|
|
- fixed cache page not closed for removed files #156
|
|
- fixed a lseek bug when seeking exactly to end of a fully indexed first level LUT #148
|
|
- fixed wear leveling issue #145
|
|
- fixed attempt to write out of bounds in flash #130,
|
|
- set file offset when seeking over end #121 (thanks @sensslen)
|
|
- fixed seeking in virgin files #120 (thanks @sensslen)
|
|
- Optional file metadata #128 (thanks @cesanta)
|
|
- AFL testing framework #100 #143 (thanks @pjsg)
|
|
- Testframe updates
|
|
|
|
New API functions:
|
|
- `SPIFFS_update_meta, SPIFFS_fupdate_meta` - updates metadata for a file
|
|
|
|
New config defines:
|
|
- `SPIFFS_OBJ_META_LEN` - enable possibility to add extra metadata to files
|
|
|
|
### 0.3.6
|
|
- Fix range bug in index memory mapping #98
|
|
- Add index memory mapping #97
|
|
- Optimize SPIFFS_read for large files #96
|
|
- Add temporal cache for opening files #95
|
|
- More robust gc #93 (thanks @dismirlian)
|
|
- Fixed a double write of same data in certain cache situations
|
|
- Fixed an open bug in READ_ONLY builds
|
|
- File not visible in SPIFFS_readdir #90 (thanks @benpicco-tmp)
|
|
- Cache load code cleanup #92 (thanks @niclash)
|
|
- Fixed lock/unlock asymmetry #88 #87 (thanks @JackJefferson, @dpruessner)
|
|
- Testframe updates
|
|
|
|
New API functions:
|
|
- `SPIFFS_ix_map` - map index meta data to memory for a file
|
|
- `SPIFFS_ix_unmap` - unmaps index meta data for a file
|
|
- `SPIFFS_ix_remap` - changes file offset for index metadata map
|
|
- `SPIFFS_bytes_to_ix_map_entries` - utility, get length of needed vector for given amount of bytes
|
|
- `SPIFFS_ix_map_entries_to_bytes` - utility, get number of bytes a vector can represent given length
|
|
|
|
New config defines:
|
|
- `SPIFFS_IX_MAP` - enable possibility to map index meta data to memory for reading faster
|
|
- `SPIFFS_TEMPORAL_FD_CACHE` - enable temporal cache for opening files faster
|
|
- `SPIFFS_TEMPORAL_CACHE_HIT_SCORE` - for tuning the temporal cache
|
|
|
|
### 0.3.5
|
|
- Fixed a bug in fs check
|
|
- API returns actual error codes #84) (thanks @Nails)
|
|
- Fix compiler warnings for non-gcc #83 #81 (thanks @Nails)
|
|
- Unable to recover from full fs #82 (thanks @rojer)
|
|
- Define SPIFFS_O_* flags #80
|
|
- Problem with long filenames #79 (thanks @psjg)
|
|
- Duplicate file name bug fix #74 (thanks @igrr)
|
|
- SPIFFS_eof and SPIFFS_tell return wrong value #72 (thanks @ArtemPisarenko)
|
|
- Bunch of testframe updates #77 #78 #86 (thanks @dpreussner, @psjg a.o)
|
|
|
|
### 0.3.4
|
|
- Added user callback file func.
|
|
- Fixed a stat bug with obj id.
|
|
- SPIFFS_probe_fs added
|
|
- Add possibility to compile a read-only version of spiffs
|
|
- Make magic dependent on fs length, if needed (see #59 & #66) (thanks @hreintke)
|
|
- Exposed SPIFFS_open_by_page_function
|
|
- Zero-size file cannot be seek #57 (thanks @lishen2)
|
|
- Add tell and eof functions #54 (thanks @raburton)
|
|
- Make api string params const #53 (thanks @raburton)
|
|
- Preserve user_data during mount() #51 (thanks @rojer)
|
|
|
|
New API functions:
|
|
- `SPIFFS_set_file_callback_func` - register a callback informing about file events
|
|
- `SPIFFS_probe_fs` - probe a spi flash trying to figure out size of fs
|
|
- `SPIFFS_open_by_page` - open a file by page index
|
|
- `SPIFFS_eof` - checks if end of file is reached
|
|
- `SPIFFS_tell` - returns current file offset
|
|
|
|
New config defines:
|
|
- `SPIFFS_READ_ONLY`
|
|
- `SPIFFS_USE_MAGIC_LENGTH`
|
|
|
|
### 0.3.3
|
|
**Might not be compatible with 0.3.2 structures. See issue #40**
|
|
- Possibility to add integer offset to file handles
|
|
- Truncate function presumes too few free pages #49
|
|
- Bug in truncate function #48 (thanks @PawelDefee)
|
|
- Update spiffs_gc.c - remove unnecessary parameter (thanks @PawelDefee)
|
|
- Update INTEGRATION docs (thanks @PawelDefee)
|
|
- Fix pointer truncation in 64-bit platforms (thanks @igrr)
|
|
- Zero-sized files cannot be read #44 (thanks @rojer)
|
|
- (More) correct calculation of max_id in obj_lu_find #42 #41 (thanks @lishen2)
|
|
- Check correct error code in obj_lu_find_free #41 (thanks @lishen2)
|
|
- Moar comments for SPIFFS_lseek (thanks @igrr)
|
|
- Fixed padding in spiffs_page_object_ix #40 (thanks @jmattsson @lishen2)
|
|
- Fixed gc_quick test (thanks @jmattsson)
|
|
- Add SPIFFS_EXCL flag #36
|
|
- SPIFFS_close may fail silently if cache is enabled #37
|
|
- User data in callbacks #34
|
|
- Ignoring SINGLETON build in cache setup (thanks Luca)
|
|
- Compilation error fixed #32 (thanks @chotasanjiv)
|
|
- Align cand_scores (thanks @hefloryd)
|
|
- Fix build warnings when SPIFFS_CACHE is 0 (thanks @ajaybhargav)
|
|
|
|
New config defines:
|
|
- `SPIFFS_FILEHDL_OFFSET`
|
|
|
|
### 0.3.2
|
|
- Limit cache size if too much cache is given (thanks pgeiem)
|
|
- New feature - Controlled erase. #23
|
|
- SPIFFS_rename leaks file descriptors #28 (thanks benpicco)
|
|
- moved dbg print defines in test framework to params_test.h
|
|
- lseek should return the resulting offset (thanks hefloryd)
|
|
- fixed type on dbg ifdefs
|
|
- silence warning about signed/unsigned comparison when spiffs_obj_id is 32 bit (thanks benpicco)
|
|
- Possible error in test_spiffs.c #21 (thanks yihcdaso-yeskela)
|
|
- Cache might writethrough too often #16
|
|
- even moar testrunner updates
|
|
- Test framework update and some added tests
|
|
- Some thoughts for next gen
|
|
- Test sigsevs when having too many sectors #13 (thanks alonewolfx2)
|
|
- GC might be suboptimal #11
|
|
- Fix eternal readdir when objheader at last block, last entry
|
|
|
|
New API functions:
|
|
- `SPIFFS_gc_quick` - call a nonintrusive gc
|
|
- `SPIFFS_gc` - call a full-scale intrusive gc
|
|
|
|
### 0.3.1
|
|
- Removed two return warnings, was too triggerhappy on release
|
|
|
|
### 0.3.0
|
|
- Added existing namecheck when creating files
|
|
- Lots of static analysis bugs #6
|
|
- Added rename func
|
|
- Fix SPIFFS_read length when reading beyond file size
|
|
- Added reading beyond file length testcase
|
|
- Made build a bit more configurable
|
|
- Changed name in spiffs from "errno" to "err_code" due to conflicts compiling in mingw
|
|
- Improved GC checks, fixed an append bug, more robust truncate for very special case
|
|
- GC checks preempts GC, truncate even less picky
|
|
- Struct alignment needed for some targets, define in spiffs config #10
|
|
- Spiffs filesystem magic, definable in config
|
|
|
|
New config defines:
|
|
- `SPIFFS_USE_MAGIC` - enable or disable magic check upon mount
|
|
- `SPIFFS_ALIGNED_OBJECT_INDEX_TABLES` - alignment for certain targets
|
|
|
|
New API functions:
|
|
- `SPIFFS_rename` - rename files
|
|
- `SPIFFS_clearerr` - clears last errno
|
|
- `SPIFFS_info` - returns info on used and total bytes in fs
|
|
- `SPIFFS_format` - formats the filesystem
|
|
- `SPIFFS_mounted` - checks if filesystem is mounted
|