nuttx/mm/iob/Kconfig
chao an 34d2cde8a8 net/l2/l3/l4: add support of iob offload
1. Add new config CONFIG_NET_LL_GUARDSIZE to isolation of l2 stack,
   which will benefit l3(IP) layer for multi-MAC(l2) implementation,
   especially in some NICs such as celluler net driver.

new configuration options: CONFIG_NET_LL_GUARDSIZE

CONFIG_NET_LL_GUARDSIZE will reserved l2 buffer header size of
network buffer to isolate the L2/L3 (MAC/IP) data on network layer,
which will be beneficial to L3 network layer protocol transparent
transmission and forwarding

------------------------------------------------------------
Layout of frist iob entry:

        iob_data (aligned by CONFIG_IOB_ALIGNMENT)
            |
            |                  io_offset(CONFIG_NET_LL_GUARDSIZE)
            |                                |
            -------------------------------------------------
      iob   |            Reserved            |    io_len    |
            -------------------------------------------------

-------------------------------------------------------------
Layout of different NICs implementation:

        iob_data (aligned by CONFIG_IOB_ALIGNMENT)
            |
            |                 io_offset(CONFIG_NET_LL_GUARDSIZE)
            |                                |
            -------------------------------------------------
 Ethernet   |       Reserved    | ETH_HDRLEN |    io_len    |
            ---------------------------------|---------------
 8021Q      |   Reserved  | ETH_8021Q_HDRLEN |    io_len    |
            ---------------------------------|---------------
 ipforward  |            Reserved            |    io_len    |
            -------------------------------------------------

--------------------------------------------------------------------

2. Support iob offload to l2 driver to avoid unnecessary memory copy

Support send/receive iob vectors directly between the NICs and l3/l4
stack to avoid unnecessary memory copies, especially on hardware that
supports Scatter/gather, which can greatly improve performance.

new interface to support iob offload:

  ------------------------------------------
  |    IOB version     |     original      |
  |----------------------------------------|
  |  devif_iob_poll()  |   devif_poll()    |
  |       ...          |       ...         |
  ------------------------------------------

--------------------------------------------------------------------

1> NIC hardware support Scatter/gather transfer

TX:

                tcp_poll()/udp_poll()/pkt_poll()/...(l3|l4)
                           /              \
                          /                \
devif_poll_[l3|l4]_connections()     devif_iob_send() (nocopy:udp/icmp/...)
           /                                   \      (copy:tcp)
          /                                     \
  devif_iob_poll("NIC"_txpoll)                callback() // "NIC"_txpoll
                                                  |
                            dev->d_iob:           |
                                                ---------------         ---------------
                             io_data       iob1 |  |          |    iob3 |  |          |
                                    \           ---------------         ---------------
                                  ---------------  |       --------------- |
                             iob0 |  |          |  |  iob2 |  |          | |
                                  ---------------  |       --------------- |
                                     \             |          /           /
                                        \          |       /           /
                                   ----------------------------------------------
                    NICs io vector |    |    |    |    |    |    |    |    |    |
                                   ----------------------------------------------

RX:

  [tcp|udp|icmp|...]ipv[4|6]_data_handler()(iob_concat/append to readahead)
                    |
                    |
      [tcp|udp|icmp|...]_ipv[4|6]_in()/...
                    |
                    |
          pkt/ipv[4/6]_input()/...
                    |
                    |
     NICs io vector receive(iov_base to each iobs)

--------------------------------------------------------------------

2> CONFIG_IOB_BUFSIZE is greater than MTU:

TX:

"(CONFIG_IOB_BUFSIZE) > (MAX_NETDEV_PKTSIZE + CONFIG_NET_GUARDSIZE + CONFIG_NET_LL_GUARDSIZE)"

                tcp_poll()/udp_poll()/pkt_poll()/...(l3|l4)
                           /              \
                          /                \
devif_poll_[l3|l4]_connections()     devif_iob_send() (nocopy:udp/icmp/...)
           /                                   \      (copy:tcp)
          /                                     \
  devif_iob_poll("NIC"_txpoll)                callback() // "NIC"_txpoll
                                                  |
                                             "NIC"_send()
                          (dev->d_iob->io_data[CONFIG_NET_LL_GUARDSIZE - NET_LL_HDRLEN(dev)])

RX:

  [tcp|udp|icmp|...]ipv[4|6]_data_handler()(iob_concat/append to readahead)
                    |
                    |
      [tcp|udp|icmp|...]_ipv[4|6]_in()/...
                    |
                    |
          pkt/ipv[4/6]_input()/...
                    |
                    |
     NICs io vector receive(iov_base to io_data)

--------------------------------------------------------------------

3> Compatible with all old flat buffer NICs

TX:
                tcp_poll()/udp_poll()/pkt_poll()/...(l3|l4)
                           /              \
                          /                \
devif_poll_[l3|l4]_connections()     devif_iob_send() (nocopy:udp/icmp/...)
           /                                   \      (copy:tcp)
          /                                     \
  devif_iob_poll(devif_poll_callback())  devif_poll_callback() /* new interface, gather iobs to flat buffer */
       /                                           \
      /                                             \
 devif_poll("NIC"_txpoll)                     "NIC"_send()(dev->d_buf)

RX:

  [tcp|udp|icmp|...]ipv[4|6]_data_handler()(iob_concat/append to readahead)
                    |
                    |
      [tcp|udp|icmp|...]_ipv[4|6]_in()/...
                    |
                    |
               netdev_input()  /* new interface, Scatter/gather flat/iob buffer */
                    |
                    |
          pkt/ipv[4|6]_input()/...
                    |
                    |
    NICs io vector receive(Orignal flat buffer)

3. Iperf passthrough on NuttX simulator:

  -------------------------------------------------
  |  Protocol      | Server | Client |            |
  |-----------------------------------------------|
  |  TCP           |  813   |   834  |  Mbits/sec |
  |  TCP(Offload)  | 1720   |  1100  |  Mbits/sec |
  |  UDP           |   22   |   757  |  Mbits/sec |
  |  UDP(Offload)  |   25   |  1250  |  Mbits/sec |
  -------------------------------------------------

Signed-off-by: chao an <anchao@xiaomi.com>
2022-12-03 11:47:04 +08:00

126 lines
4.1 KiB
Plaintext

#
# For a description of the syntax of this configuration file,
# see the file kconfig-language.txt in the NuttX tools repository.
#
menu "Common I/O Buffer Support"
config MM_IOB
bool "Enable generic I/O buffer support"
default n
---help---
This setting will build the common I/O buffer (IOB) support
library.
if MM_IOB
config IOB_NBUFFERS
int "Number of pre-allocated I/O buffers"
default 36 if NET_WRITE_BUFFERS
default 24 if !NET_WRITE_BUFFERS && NET_READAHEAD
default 8 if !NET_WRITE_BUFFERS && !NET_READAHEAD
---help---
Each packet is represented by a series of small I/O buffers in a
chain. This setting determines the number of preallocated I/O
buffers available for packet data.
config IOB_BUFSIZE
int "Payload size of one I/O buffer"
default 196
---help---
Each packet is represented by a series of small I/O buffers in a
chain. This setting determines the data payload each preallocated
I/O buffer.
config IOB_HEADSIZE
int "Head size of each I/O buffer"
default 0
---help---
This setting determines the reserved size in front of the payload
buffer in each I/O buffer.
config IOB_ALIGNMENT
int "Alignment size of each I/O buffer"
default 4
---help---
The member io_head of all I/O buffers is aligned to the value
specified by this configuration.
config IOB_SECTION
string "The section where iob buffer is located"
---help---
The section where iob buffer is located.
The section must be zero-initialized on system boot.
config IOB_NCHAINS
int "Number of pre-allocated I/O buffer chain heads"
default 0 if !NET_READAHEAD
default IOB_NBUFFERS if NET_READAHEAD
---help---
These tiny nodes are used as "containers" to support queueing of
I/O buffer chains. This will limit the number of I/O transactions
that can be "in-flight" at any give time. The default value of
zero disables this features.
These generic I/O buffer chain containers are not currently used
by any logic in NuttX. That is because their other specialized
I/O buffer chain containers that also carry a payload of usage
specific information.
Note: TCP doesn't use this.
Note: UDP and CAN use this.
config IOB_THROTTLE
int "I/O buffer throttle value"
default 0 if !NET_WRITE_BUFFERS || !NET_READAHEAD
default 8 if NET_WRITE_BUFFERS && NET_READAHEAD
---help---
TCP write buffering and read-ahead buffer use the same pool of free
I/O buffers. In order to prevent uncontrolled incoming TCP packets
from hogging all of the available, pre-allocated I/O buffers, a
throttling value is required. This throttle value assures that
I/O buffers will be denied to the read-ahead logic before TCP writes
are halted.
config IOB_NOTIFIER
bool "Support IOB notifications"
default n
depends on SCHED_WORKQUEUE
select WQUEUE_NOTIFIER
---help---
Enable building of IOB notifier logic that will execute a worker
function on the high priority work queue when an IOB is available.
This is is a general purpose notifier, but was developed specifically to
support poll() logic where the poll must wait for an IOB to become
available.
config IOB_NOTIFIER_DIV
int "Notification divider"
default 4
range 1 64
depends on IOB_NOTIFIER
---help---
IOBs may become available at very high rates and the resulting
notification processing can be substantial even if there is nothing
waiting for a free IOB. This divider will reduce that rate of
notification. This must be an even power of two. Supported values
include: 1, 2, 4, 8, 16, 32, 64. The default value of 4 means that
a notification will be sent only when there are a multiple of 4 IOBs
available.
config IOB_DEBUG
bool "Force I/O buffer debug"
default n
depends on DEBUG_FEATURES && !SYSLOG_BUFFER
---help---
This option will force debug output from I/O buffer logic. This
is not normally something that would want to do but is convenient
if you are debugging the I/O buffer logic and do not want to get
overloaded with other un-related debug output.
NOTE that this selection is not available if IOBs are being used
to syslog buffering logic (CONFIG_SYSLOG_BUFFER=y)!
endif # MM_IOB
endmenu # Common I/O buffer support