Terminolgy

Initialization

Page Faults

Fill Initiation

Fill Complete

Task Resumption

g_waitingforfill

An OS list that is used to hold the TCBs of tasks that are waiting for a page fill.

g_pendingfill

A variable that holds a reference to the TCB of the thread that is currently be re-filled.

g_pgworker

The process ID of of the thread that will perform the page fills.

pg_callback()

The callback function that is invoked from a driver when the fill is complete.

pg_miss()

The function that is called from chip-specific code to handle a page fault.

TCB

Task Control Block

The following declarations will be added.

• g_waitingforfill. A doubly linked list that will be used to implement a prioritized list of the TCBs of tasks that are waiting for a page fill.
• g_pgworker. The process ID of of the thread that will perform the page fills

During OS initialization in sched/os_start.c, the following steps will be performed:

• The g_waitingforfill queue will be initialized.
• The special, page fill worker thread, will be started. The pid of the page will worker thread will be saved in g_pgworker. Note that we need a special worker thread to perform fills; we cannot use the "generic" worker thread facility because we cannot be assured that all actions called by that worker thread will always be resident in memory.

Declarations for g_waitingforfill, g_pgworker, and other internal, private definitions will be provided in sched/pg_internal.h. All public definitions that should be used by the chip-specific code will be available in include/nuttx/page.h and include/nuttx/arch.h.

Page fault exception handling. Page fault handling is performed by the function pg_miss(). This function is called from architecture-specific memory segmentation fault handling logic. This function will perform the following operations:

1. Sanity checking. This function will ASSERT if the currently executing task is the page fill worker thread. The page fill worker thread is how the the page fault is resolved and all logic associated with the page fill worker must be "locked" and always present in memory.
2. Block the currently executing task. This function will call up_block_task() to block the task at the head of the ready-to-run list. This should cause an interrupt level context switch to the next highest priority task. The blocked task will be marked with state TSTATE_WAIT_PAGEFILL and will be retained in the g_waitingforfill prioritized task list.
3. Boost the page fill worker thread priority. Check the priority of the task at the head of the g_waitingforfill list. If the priority of that task is higher than the current priority of the page fill worker thread, then boost the priority of the page fill worker thread to that priority.
4. Signal the page fill worker thread. Is there a page already being filled? If not then signal the page fill worker thread to start working on the queued page fill requests.

When signaled from pg_miss(), the page fill worker thread will be awakenend and will initiate the fill operation.

Input Parameters. None -- The head of the ready-to-run list is assumed to be that task that caused the exception. The current task context should already be saved in the TCB of that task. No additional inputs are required.

Assumptions.

• It is assumed that this function is called from the level of an exception handler and that all interrupts are disabled.
• The pg_miss() must be "locked" in memory. Calling pg_miss() cannot cause a nested page fault.
• It is assumed that currently executing task (the one at the head of the ready-to-run list) is the one that cause the fault. This will always be true unless the page fault occurred in an interrupt handler. Interrupt handling logic must always be available and "locked" into memory so that page faults never come from interrupt handling.
• The chip-specific page fault exception handling has already verified that the exception did not occur from interrupt/exception handling logic.
• As mentioned above, the task causing the page fault must not be the page fill worker thread because that is the only way to complete the page fill.
• Chip specific logic will map the virtual address space into three regions:
  1. A .text region containing "locked-in-memory" code that is always avaialable and will never cause a page fault.
  2. A .text region containing pages that can be assigned allocated, mapped to various virtual addresses, and filled from some mass storage medium.
  3. And a fixed RAM space for .bss, .text, and .heap.

Locking code in Memory. One way to accomplish this would be a two phase link:

• In the first phase, create a partially linked objected containing all interrupt/exception handling logic, the page fill worker thread plus all parts of the IDLE thread (which must always be available for execution).
• All of the .text and .rodata sections of this partial link should be collected into a single section.
• The second link would link the partially linked object along with the remaining object to produce the final binary. The linker script should position the "special" section so that it lies in a reserved, "non-swappable" region.

The page fill worker thread will be awakened on one of two conditions:

• When signaled by pg_miss(), the page fill worker thread will be awakenend (see above), or
• From pg_fillcomplete() after completing last fill (see below).

The page fill worker thread will maintain a static variable called _TCB *g_pendingfill. If not fill is in progress, g_pendingfill will be NULL. Otherwise, will point to the TCB of the task which is receiving the fill that is in progess.

When awakened from pg_miss(), no fill will be in progress and g_pendingfill will be NULL. In this case, the page fill worker thread will call pg_startfill(). That function will perform the following operations:

• Call up_allocpage(vaddr, &page). This chip-specific function will set aside page in memory and map to virtual address (vaddr). If all pages available pages are in-use (the typical case), this function will select a page in-use, un-map it, and make it available.
• Call the chip-specific function up_fillpage(page, pg_callback). This will start asynchronous page fill. The page fill worker thread will provide a callback function, pg_callback, that will be called when the page fill is finished (or an error occurs). This callback will probably from interrupt level.
• Restore default priority of the page fill worker thread's default priority and wait to be signaled for the next event -- the fill completion event.

While the fill is in progress, other tasks may execute. If another page fault occurs during this time, the faulting task will be blocked and its TCB will be added (in priority order) to g_waitingforfill. But no action will be taken until the current page fill completes. NOTE: The IDLE task must also be fully locked in memory. The IDLE task cannot be blocked. It the case where all tasks are blocked waiting for a page fill, the IDLE task must still be available to run.

The chip-specific functions, up_allocpage(vaddr, &page) and up_fillpage(page, pg_callback) will be prototyped in include/nuttx/arch.h

When the chip-specific driver completes the page fill, it will call the pg_callback() that was provided to up_fillpage. pg_callback() will probably be called from driver interrupt-level logic. The driver ill provide the result of the fill as an argument. NOTE: pg_callback() must also be locked in memory.

When pg_callback() is called, it will perform the following operations:

• Verify that g_pendingfill is non-NULL.
• If the priority of thread in g_pendingfill is higher than page fill worker thread, boost work thread to that level.
• Signal the page fill worker thread.

When the page fill worker thread is awakened and g_pendingfill is non-NULL (and other state variables are in concurrence), the page fill thread will know that is was awakened because of a page fill completion event. In this case, the page fill worker thread will:

• Verify consistency of state information and g_pendingfill.
• Verify that the page fill completed successfully, and if so,
• Call up_unblocktask(g_pendingfill) to make the task that just received the fill ready-to-run.
• Check if the g_waitingforfill list is empty. If not:
  • Remove the highest priority task waiting for a page fill from g_waitingforfill,
  • Save the task's TCB in g_pendingfill,
  • If the priority of the thread in g_pendingfill, is higher in priority than the default priority of the page fill worker thread, then set the priority of the page fill worker thread to that priority.
  • Call pg_startfill() which will start the next fill (as described above).
• Otherwise,
  • Set g_pendingfill to NULL.
  • Restore the default priority of the page fill worker thread.
  • Wait for the next fill related event (a new page fault).