======= MNEMOFS ======= Mnemofs is a NAND Flash File System built for NuttX. Usage ===== If there's a NAND flash available at a location, for example, ``/dev/nand``, you can mount it with ``mnemofs`` to a location like ``/mydir`` using:: mount -t mnemofs /dev/nand /mydir The above command will only work if the device was already formatted using mnemofs. For a brand new device, or if you want to switch from an existing file system, this won't work, and would need a format. Instead try this:: mount -t mnemofs -o forceformat /dev/nand /mydir Unsure of whether you need to do a format? This will help:: mount -t mnemofs -o autoformat /dev/nand /mydir This will format the device only if it can not detect mnemofs being already formatted onto it. Do note this includes cases where mnemofs is formatted to the device, but it's been mutilated to the point of being unrecognizable. After this, use it like a regular file system. That's the job of a file system after all...to hide the storage device's pecularities behind an abstraction. A file system is considered good if you don't have to think about its existence during regular usage. Design ====== mnemofs is designed to be a middle ground between flash storage consumption, memory consumption, wear and speed. It sacrifices a little bit of everything, and ends up being acceptably good in all of them, instead of sacrificing multiple aspects, and being good in one. mnemofs consists of several components, however, a walkthrough of the process where a change requested by a user ends up being written to the NAND flash would serve well for an introduction. The details will be explained further below. The user requests some changes, say, add ``x`` bytes to ``y`` offset in a file. This change is copied into the LRU cache of mnemofs. This LRU cache exists in-memory, and serves as a tool for wear reduction. This LRU cache is a kernel list of nodes. Each node represents a file or a directory. When the LRU is full, the last node is popped from this list and the changes it contains, which is an accumulation of changes requested by the user for that particular file or directory since the node was added to the LRU cache, is written to the flash. Each file or directory is represented by a `CTZ skip list `_, and the only attributes required to access the list is the index of the last CTZ skip list block, the page number of that CTZ skip list block, and the size of the file. In mnemofs, CTZ skip list blocks take up exactly one page on the flash. Mnemofs works in a Copy-On-Write manner, similar to littlefs. When a CTZ skip list is updated, the new location is added to the Journal of mnemofs as a log. This log contains some information about the location of the new CTZ list, the path it belongs to, etc. and then the updated location is added as an update to its parent's CTZ skip list, and it undergoes the same process. This log is appended with a checksum of the entire log, which gives an assurance that the saved log was indeed saved completely before a power loos. The journal is a modified singly linked list of blocks on the flash that contains logs of changes in the file system. The last two blocks of the journal is reserved for master blocks, hence the number of blocks in the journal will be referred to as ``n + 2`` blocks. The area on storage other than the journal contains a certain "base" state of the file system. All changes to the base state since is written to the journal. The first block of the journal starts with an 8 byte magic sequence to identify the start of the journal (on mount), followed by the number of blocks in the journal and then finally an array of all the ``n + 2`` block numbers that are part of the journal. After this part, the entire area in the ``n`` blocks contain logs and their checksums. The last two blocks of a journal are called the master blocks, and they store multiple instances of the master node. They are duplicates of each other, and each instance of the master node takes one page each, and are written to these master blocks in a sequential manner. The master node points to the root. When the first ``n`` blocks of the journal are full, then they are flushed and since the root updates here as well, a new master node is written. Once the new master node is written, the file system's base state is updated and thus the old obsolete pages can be erased (if possible). The first ``n`` blocks of the journal move more than the master nodes. The block allocator of mnemofs is havily inspired from littlefs. It starts from a random block, and starts allocating pages or blocks sequentially in a circular manner. It skips pages upon block requirement, but since block requirements are only required by internal structures, they are always requested in bulk, and minimize wastage. However, unlike in littlefs, mnemofs keeps a bitmap in memory about the pages that are currently being used, as well as the count of pages inside each block that want to be erased. In mnemofs, the bind might take a lot of time in the worst possible theoretical case, as it's an ``O(n)`` mounting process, however, it's not the case in real life. Mnemofs only needs to scan the first page of every block in the device to look for the start of the journal. Leaving the actual location of the page aside, this will be pretty fast in real life as the larger the storage capacity is, the larger are the pages and the larger are the number of pages per block, and thus the number of blocks in the device do not increase at a rate similar to the increase in storage capacity of the device. Further, the journal has the journal array, which contains block numbers of each block in it, very close to the start of the array, and mnemofs can quickly jump from there to the latest master node, and scan the file system for used pages.