281 lines
11 KiB
Markdown
281 lines
11 KiB
Markdown
# System / `zmodem`
|
|
|
|
## Contents
|
|
|
|
- Buffering Notes
|
|
- Hardware Flow Control
|
|
- RX Buffer Size
|
|
- Buffer Recommendations
|
|
- Using NuttX ZModem with a Linux Host
|
|
- Sending Files from the Target to the Linux Host PC
|
|
- Receiving Files on the Target from the Linux Host PC
|
|
- Building the ZModem Tools to Run Under Linux
|
|
- Status
|
|
|
|
## Buffering Notes
|
|
|
|
### Hardware Flow Control
|
|
|
|
Hardware flow control must be enabled in serial drivers in order to prevent data
|
|
overrun. However, in the most NuttX serial drivers, hardware flow control only
|
|
protects the hardware RX FIFO: Data will not be lost in the hardware FIFO but
|
|
can still be lost when it is taken from the FIFO. We can still overflow the
|
|
serial driver's RX buffer even with hardware flow control enabled! That is
|
|
probably a bug. But the workaround solution that I have used is to use lower
|
|
data rates and a large serial driver RX buffer.
|
|
|
|
Those measures should be unnecessary if buffering and hardware flow control are
|
|
set up and working correctly.
|
|
|
|
### Software Flow Control
|
|
|
|
The ZModem protocol has `XON/XOFF` flow control built into it. The protocol
|
|
permits `XON` or `XOFF` characters placed at certain parts of messages. If
|
|
software flow control is enabled on the receiving end it will consume the `XON`s
|
|
and `XOFF`s. Otherwise they will be ignored in the data by the ZModem logic.
|
|
|
|
NuttX, however, does not implement `XON/XOFF` flow control so these do nothing.
|
|
On NuttX you will have to use hardware flow control in most cases.
|
|
|
|
The `XON`/`XOFF` controls built into ZModem could be used if you enabled
|
|
software flow control in the host. But that would only work in one direction: If
|
|
would prevent the host from overrunning the the target Rx buffering. So you
|
|
should be able to do host-to-target software flow control. But there would still
|
|
be no target-to-host flow control. That might not be an issue because the host
|
|
is usually so much faster than that target.
|
|
|
|
### RX Buffer Size
|
|
|
|
The ZModem protocol supports a message that informs the file sender of the
|
|
maximum size of data that you can buffer (`ZRINIT`). However, my experience is
|
|
that the Linux sz ignores this setting and always sends file data at the maximum
|
|
size (`1024`) no matter what size of buffer you report. That is unfortunate
|
|
because that, combined with the possibilities of data overrun mean that you must
|
|
use quite large buffering for ZModem file receipt to be reliable (none of these
|
|
issues effect sending of files).
|
|
|
|
### Buffer Recommendations
|
|
|
|
Based on the limitations of NuttX hardware flow control and of the Linux sz
|
|
behavior, I have been testing with the following configuration (assuming `UART1`
|
|
is the ZModem device):
|
|
|
|
1) This setting determines that maximum size of a data packet frame:
|
|
|
|
```conf
|
|
CONFIG_SYSTEM_ZMODEM_PKTBUFSIZE=1024
|
|
```
|
|
|
|
2) Input Buffering. If the input buffering is set to a full frame, then data
|
|
overflow is less likely.
|
|
|
|
```conf
|
|
CONFIG_UART1_RXBUFSIZE=1024`
|
|
```
|
|
|
|
3) With a larger driver input buffer, the ZModem receive I/O buffer can be
|
|
smaller:
|
|
|
|
```conf
|
|
CONFIG_SYSTEM_ZMODEM_RCVBUFSIZE=256
|
|
```
|
|
|
|
4) Output buffering. Overrun cannot occur on output (on the NuttX side) so there
|
|
is no need to be so careful:
|
|
|
|
```conf
|
|
CONFIG_SYSTEM_ZMODEM_SNDBUFSIZE=512
|
|
CONFIG_UART1_TXBUFSIZE=256
|
|
```
|
|
|
|
## Using NuttX ZModem with a Linux Host
|
|
|
|
### Sending Files from the Target to the Linux Host PC
|
|
|
|
The NuttX ZModem commands have been verified against the rzsz programs running
|
|
on a Linux PC. To send a file to the PC, first make sure that the serial port is
|
|
configured to work with the board (Assuming you are using 9600 baud for the data
|
|
transfers - high rates may result in data overruns):
|
|
|
|
```bash
|
|
$ sudo stty -F /dev/ttyS0 9600 # Select 9600 BAUD
|
|
$ sudo stty -F /dev/ttyS0 crtscts # Enables CTS/RTS handshaking *
|
|
$ sudo stty -F /dev/ttyS0 raw # Puts the TTY in raw mode
|
|
$ sudo stty -F /dev/ttyS0 # Show the TTY configuration
|
|
```
|
|
|
|
\* Only if hardware flow control is enabled.
|
|
|
|
Start `rz` on the Linux host (using `/dev/ttyS0` as an example):
|
|
|
|
```bash
|
|
$ sudo rz < /dev/ttyS0 > /dev/ttyS0
|
|
```
|
|
|
|
You can add the `rz -v` option multiple times, each increases the level of debug
|
|
output. If you want to capture the Linux `rz` output, then re-direct `stderr` to
|
|
a log file by adding `2>rz.log` to the end of the `rz` command.
|
|
|
|
**Note**: The NuttX ZModem does sends `rz\n` when it starts in compliance with
|
|
the ZModem specification. On Linux this, however, seems to start some other,
|
|
incompatible version of `rz`. You need to start `rz` manually to make sure that
|
|
the correct version is selected. You can tell when this evil `rz`/`sz` has
|
|
inserted itself because you will see the `^` (`0x5e`) character replacing the
|
|
standard ZModem `ZDLE` character (`0x19`) in the binary data stream.
|
|
|
|
If you don't have the `rz` command on your Linux box, the package to install
|
|
`rzsz` (or possibly `lrzsz`).
|
|
|
|
Then on the target (using `/dev/ttyS1` as an example).
|
|
|
|
```
|
|
nsh> sz -d /dev/ttyS1 <filename>
|
|
```
|
|
|
|
Where filename is the full path to the file to send (i.e., it begins with the
|
|
`/` character). `/dev/ttyS1` or whatever device you select **must** support
|
|
Hardware flow control in order to throttle therates of data transfer to fit
|
|
within the allocated buffers.
|
|
|
|
### Receiving Files on the Target from the Linux Host PC
|
|
|
|
**Note**: There are issues with using the Linux `sz` command with the NuttX `rz`
|
|
command. See _Status_ below. It is recommended that you use the NuttX `sz`
|
|
command on Linux as described in the next paragraph.
|
|
|
|
To send a file to the target, first make sure that the serial port on the host
|
|
is configured to work with the board (Assuming that you are using `9600` baud
|
|
for the data transfers - high rates may result in data overruns):
|
|
|
|
```bash
|
|
$ sudo stty -F /dev/ttyS0 9600 # Select 9600 (or other) BAUD
|
|
$ sudo stty -F /dev/ttyS0 crtscts # Enables CTS/RTS handshaking *
|
|
$ sudo stty -F /dev/ttyS0 raw # Puts the TTY in raw mode
|
|
$ sudo stty -F /dev/ttyS0 # Show the TTY configuration
|
|
```
|
|
|
|
\* Only is hardware flow control is enabled.
|
|
|
|
Start `rz` on the on the target. Here, in this example, we are using
|
|
`/dev/ttyS1` to perform the transfer
|
|
|
|
```shell
|
|
nsh> rz -d /dev/ttyS1
|
|
```
|
|
|
|
`/dev/ttyS1` or whatever device you select **must** support Hardware flow
|
|
control in order to throttle therates of data transfer to fit within the
|
|
allocated buffers.
|
|
|
|
Then use the `sz` command on Linux to send the file to the target:
|
|
|
|
```bash
|
|
$ sudo sz <filename> [-l nnnn] [-w nnnn] </dev/ttyS0 >/dev/ttyS0
|
|
```
|
|
|
|
Where `<filename>` is the file that you want to send. If `-l nnnn` and `-w nnnn`
|
|
is not specified, then there will likely be packet buffer overflow errors.
|
|
`nnnn` should be set to a value less than or equal to
|
|
`CONFIG_SYSTEM_ZMODEM_PKTBUFSIZE`.
|
|
|
|
The resulting file will be found where you have configured the ZModem
|
|
**sandbox** via `CONFIG_SYSTEM_ZMODEM_MOUNTPOINT`.
|
|
|
|
You can add the `sz -v` option multiple times, each increases the level of debug
|
|
output. If you want to capture the Linux `sz` output, then re-direct `stderr` to
|
|
a log file by adding `2>sz.log` to the end of the `sz` command.
|
|
|
|
If you don't have the sz command on your Linux box, the package to install
|
|
`rzsz` (or possibly `lrzsz`).
|
|
|
|
## Building the ZModem Tools to Run Under Linux
|
|
|
|
Build support has been added so that the NuttX ZModem implementation can be
|
|
executed on a Linux host PC. This can be done by
|
|
|
|
- Change to the `apps/systems/zmodem` directory
|
|
- Make using the special makefile, `Makefile.host`
|
|
|
|
**Notes**:
|
|
|
|
1. `TOPDIR` and `APPDIR` must be defined on the make command line: `TOPDIR` is
|
|
the full path to the `nuttx/` directory; `APPDIR` is the full path to the
|
|
`apps/` directory. For example, if you installed nuttx at
|
|
`/home/me/projects/nuttx` and apps at `/home/me/projects/apps`, then the
|
|
correct make command line would be:
|
|
|
|
```bash
|
|
make -f Makefile.host TOPDIR=/home/me/projects/nuttx APPDIR=/home/me/projects/apps
|
|
```
|
|
|
|
2. Add `CONFIG_DEBUG_FEATURES=1` to the make command line to enable debug output
|
|
3. Make sure to clean old target `.o` files before making new host `.o` files.
|
|
|
|
This build is has been verified as of `2013-7-16` using Linux to transfer files
|
|
with an Olimex LPC1766STK board. It works great and seems to solve all of the
|
|
problems found with the Linux `sz`/`rz` implementation.
|
|
|
|
## Status
|
|
|
|
- `2013-7-15`: Testing against the Linux `rz`/`sz` commands.
|
|
|
|
I have tested with the `boards/arm/lpc17xx_40xx/olimex-lpc1766stk`
|
|
configuration. I have been able to send large and small files with the target
|
|
`sz` command. I have been able to receive small files, but there are problems
|
|
receiving large files using the Linux `sz` command: The Linux `sz` does not
|
|
obey the buffering limits and continues to send data while `rz` is writing the
|
|
previously received data to the file and the serial driver's RX buffer is
|
|
overrun by a few bytes while the write is in progress. As a result, when it
|
|
reads the next buffer of data, a few bytes may be missing. The symptom of this
|
|
missing data is a CRC check failure.
|
|
|
|
Either (1) we need a more courteous host application, or (2) we need to
|
|
greatly improve the target side buffering capability!
|
|
|
|
My thought now is to implement the NuttX `sz` and `rz` commands as PC side
|
|
applications as well. Matching both sides and obeying the handshaking will
|
|
solve the issues. Another option might be to fix the serial driver hardware
|
|
flow control somehow.
|
|
|
|
- `2013-7-16`. More Testing against the Linux `rz`/`sz` commands.
|
|
|
|
I have verified that with debug off and at lower serial BAUD (`2400`), the
|
|
transfers of large files succeed without errors. I do not consider this a
|
|
_solution_ to the problem. I also found that the LPC17xx hardware flow control
|
|
caused strange hangs; ZModem works better with hardware flow control disabled
|
|
on the LPC17xx.
|
|
|
|
At this lower BAUD, RX buffer sizes could probably be reduced; Or perhaps the
|
|
BAUD could be increased. My thought, however, is that tuning in such an
|
|
unhealthy situation is not the approach: The best thing to do would be to use
|
|
the matching NuttX sz on the Linux host side.
|
|
|
|
- `2013-7-16`. More Testing against the NuttX `rz`/`sz` on Both Ends.
|
|
|
|
The NuttX `sz`/`rz` commands have been modified so that they can be built and
|
|
executed under Linux. In this case, there are no transfer problems at all in
|
|
either direction and with large or small files. This configuration could
|
|
probably run at much higher serial speeds and with much smaller buffers
|
|
(although that has not been verified as of this writing).
|
|
|
|
- `2018-5-27`
|
|
|
|
Updates to checksum calculations. Verified correct operation with hardware
|
|
flow control using the `olimex-stm32-p407/zmodem` configuration. Only the
|
|
host-to-target transfer was verified.
|
|
|
|
This was using the Linux `sz` utility. There appears to still be a problem
|
|
using the NuttX `sz` utility running on Linux.
|
|
|
|
- `2018-5-27`
|
|
|
|
Verified correct operation with hardware flow control using the
|
|
`olimex-stm32-p407/zmodem` configuration with target-to-host transfers was
|
|
verified. Again, there are issues remaining if I tried the NuttX `rz` utility
|
|
running on Linux.
|
|
|
|
- `2018-6-26`
|
|
|
|
With `-w nnnn` option, the host-to-target transfer can work reliably without
|
|
hardware flow control.
|