* webclient_perform
* Add a new flag to use non-blocking mode (WEBCLIENT_FLAG_NON_BLOCKING)
* Implement restarting
* Add a few associated API functions
* webclient_get_poll_info: get the descriptor info for poll/select
* webclient_abort: abort the operation (instead of restarting)
- check for minimum packet length (44 bytes, which includes the fields we
read next)
- pass the correct remaining buffer length to dhcpc_parseoptions()
- dhcpc_parseoptions(): ensure we never read past the end of the buffer
Gregory Nutt has submitted the SGA
Max Holtzberg has submitted the SGA
as a result we can migrate the licenses to Apache.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
It matches the original iperf behavior.
Note: -t is a client-only option for the original iperf 2.xx.
Note: You can still choose the previous behavior by
explictly specifying -t.
Gregory Nutt is has submitted the SGA
Sebastien Lorquet has submitted the ICLA
as a result we can migrate the licenses to Apache.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
Gregory Nutt is has submitted the SGA
Max Holtzberg has submitted the SGA
as a result we can migrate the licenses to Apache.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
Gregory Nutt is has submitted the SGA
Mateusz Szafoni has submitted the ICLA
as a result we can migrate the licenses to Apache.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
Gregory Nutt is has submitted the SGA
Sebastien Lorquet has submitted the ICLA
as a result we can migrate the licenses to Apache.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
Gregory Nutt is the copyright holder for those files and he has submitted the
SGA as a result we can migrate the licenses to Apache.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
Similarly to the fix in wget_parseheaders.
But simply always bail out as i guess it's very rare to see
that long status line.
Tested with an aritifically small CONFIG_WEBCLIENT_MAXHTTPLINE=20,
which is smaller than "HTTP/1.1 301 Moved Permanently".
* Detect a long header line which doesn't fit the buffer.
* If the header line in question doesn't seem important for us,
just ignore it.
* Otherwise, bail out with -E2BIG.
The overrun was found by LLVM UBSan on macOS.
The following is an example of a long header line,
which doesn't fit the default CONFIG_WEBCLIENT_MAXHTTPLINE=200.
```
pacetanuki% curl -v https://www.midokura.com
:
:
< HTTP/2 200
< server: nginx
< date: Fri, 14 May 2021 02:16:24 GMT
< content-type: text/html; charset=UTF-8
< content-length: 131313
< x-powered-by: PHP/7.4.18
< link: <https://www.midokura.com/wp-json/>; rel="https://api.w.org/", <https://www.midokura.com/wp-json/wp/v2/pages/7>; rel="alternate"; type="application/json", <https://www.midokura.com/>; rel=shortlink
< x-powered-by: PleskLin
<
```
For EINVAL, it doesn't make sense to keep parsing.
(For E2BIG, it might make some sense.)
Found by LLVM ASan.
```
=================================================================
==81622==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000000f2 at pc 0x00010d2746ca bp 0x7ffee29a9980 sp 0x7ffee29a9978
READ of size 1 at 0x6020000000f2 thread T0
#0 0x10d2746c9 in netlib_parseurl netlib_parseurl.c:121
#1 0x10d26b293 in parseurl webclient.c:479
#2 0x10d265e48 in webclient_perform webclient.c:690
#3 0x10d277c5b in main main.c:210
#4 0x7fff7a06f3d4 in start+0x0 (libdyld.dylib:x86_64+0x163d4)
0x6020000000f2 is located 0 bytes to the right of 2-byte region [0x6020000000f0,0x6020000000f2)
allocated by thread T0 here:
#0 0x10d3996d3 in wrap_strdup+0x203 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x3e6d3)
#1 0x10d276abe in main main.c:147
#2 0x7fff7a06f3d4 in start+0x0 (libdyld.dylib:x86_64+0x163d4)
SUMMARY: AddressSanitizer: heap-buffer-overflow netlib_parseurl.c:121 in netlib_parseurl
Shadow bytes around the buggy address:
0x1c03ffffffc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c03ffffffd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c03ffffffe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c03fffffff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c0400000000: fa fa fd fd fa fa fd fd fa fa 00 00 fa fa 00 00
=>0x1c0400000010: fa fa 00 fa fa fa 00 00 fa fa 00 06 fa fa[02]fa
0x1c0400000020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c0400000030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c0400000040: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c0400000050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c0400000060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==81622==ABORTING
```
The following errors are false positives.
https://github.com/apache/incubator-nuttx/issues/387
```
/Users/yamamoto/git/nuttx/apps/netutils/netlib/netlib_parseurl.c:57:13: error: C++ style comment
/Users/yamamoto/git/nuttx/apps/netutils/netlib/netlib_parseurl.c:59:35: error: C++ style comment
```
Summary:
- This commit fixes the renew command with the usrsock
Impact:
- None
Testing:
- Tested with spresense:wifi and esp32-devkitc:wapi
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
It's used in the code but missing in Kconfig.
I guess the PR [1] forgot to add this.
Make it "y" by default because it seems like the original behavior
before the PR [1]. I have no strong opinions on the default
either ways.
[1] https://github.com/apache/incubator-nuttx-apps/pull/570
from:
return - ECONNABORTED;
to:
return -ECONNABORTED;
The latter style is more commonly used within NuttX code base.
Note: nxstyle doesn't complain on either of them.
The current calculation easily overflows if the local time is
around the unix epoch. I guess it isn't too unusual for
devices without RTC. Or, the battery is dead. Or, whatever.
This commit avoids the overflow by simply dividing everything by 2.
While more sophisticated and precise solutions are possible,
I feel that they are overkill for this simple implementation.
For example,
The unix epoch (1970) is 0x83aa7e8000000000 in
64-bit NTP timestamp. (1900-origin)
The timestamp now, as of writing this, is 0xe3cda16b00000000.
With the code before this commit, the offset will be:
(lldb) p (long long)((0xe3cda16b00000000 - 0x83aa7e8000000000) + (0xe3cda16b00000000 - 0x83aa7e8000000000)) / 2
(long long) $16 = -2295952992316162048
(lldb)
with the new code, it would be:
(lldb) p (long long)((0xe3cda16b00000000 / 2 - 0x83aa7e8000000000 / 2) + (0xe3cda16b00000 / 2 - 0x83aa7e8000000000 / 2))
(long long) $17 = 6927419044538613760
(lldb)
It's the correct offset from the unix epoch:
(lldb) p 6927419044538613760 >> 32
(long) $0 = 1612915435
(lldb)
spacetanuki% date -r 1612915435
Wed Feb 10 09:03:55 JST 2021
spacetanuki%
Summary:
- I noticed that lc823450-xgevk:rndis does not work with iperf
- Finally, I found that it does not call pthread_join()
- This commit fixes this issue
Impact:
- No impact
Testing:
- Tested with lc823450-xgevk:rndis
- NOTE: need to add iperf in the defconfig
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
The buffer doesn't have the space for the terminating NUL.
Fix it by replacing it with fwrite, as there seems to be little reason
to use fprintf and NUL-terminated string in the first place.
Allow multiple NTP servers, also with runtime configurable list
Add validation of received NTP packets
NTPv4 support with 'Kiss o' Death' message handling
IPv6 support
Collect multiple NTP samples and filter outsiders
Check if system clock has been altered during NTP (if CONFIG_CLOCK_MONOTONIC)
Signed-off-by: Juha Niskanen <juha.niskanen@haltian.com>
This happened frequently for me with Docker Desktop's
/var/run/docker.sock on macOS.
But I believe it can happen on other environments, even with TCP.
Alternatively, this case can be handled by the callback
implementations. But it's simpler to handle the corner case here.
Summary:
- This commit fixes compile warnings with Arm GCC 9.3.1
Impact:
- None
Testing:
- Tested with lm3s6965-ek:discover
- NOTE: need to add the following configs
+CONFIG_EXAMPLES_IPERF=y
+CONFIG_EXAMPLES_IPERFTEST_DEVNAME="eth0"
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
* It doesn't make much sense to have the default values for
these highly environment-dependent settings.
* netutils/netinit/netinit_associate.c calls wpa_driver_wext_associate
if the SSID is not empty. Depending on the environment, it can take
long to fail. It slows down the boot of some configurations
considerably. eg. esp32-devkitc:wapi
wget is missing from some system (like macOS and Windows native),
it's better to use curl to simplify build environment.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
The following nxstyle error is intentionally left.
It's a nested function. I guess nxstyle doesn't support it.
netutils/chat/chat.c:110:8: error: Bad comment alignment
The following nxstyle errors are intentionally left.
The first one is in the copyright notice.
The others are identifiers like ClientData etc.
netutils/thttpd/thttpd.c:10:81: error: Long line found
netutils/thttpd/thttpd.c:99:2: error: Mixed case identifier found
netutils/thttpd/thttpd.c💯2: error: Mixed case identifier found
netutils/thttpd/thttpd.c:131:17: error: Mixed case identifier found
netutils/thttpd/thttpd.c:131:57: error: Mixed case identifier found
netutils/thttpd/thttpd.c:132:36: error: Mixed case identifier found
netutils/thttpd/thttpd.c:133:52: error: Mixed case identifier found
netutils/thttpd/thttpd.c:134:23: error: Mixed case identifier found
netutils/thttpd/thttpd.c:134:63: error: Mixed case identifier found
netutils/thttpd/thttpd.c:527:2: error: Mixed case identifier found
netutils/thttpd/thttpd.c:597:17: error: Mixed case identifier found
netutils/thttpd/thttpd.c:597:57: error: Mixed case identifier found
netutils/thttpd/thttpd.c:608:14: error: Mixed case identifier found
netutils/thttpd/thttpd.c:615:38: error: Mixed case identifier found
netutils/thttpd/thttpd.c:620:14: error: Mixed case identifier found
netutils/thttpd/thttpd.c:625:37: error: Mixed case identifier found
netutils/thttpd/thttpd.c:632:36: error: Mixed case identifier found
netutils/thttpd/thttpd.c:633:52: error: Mixed case identifier found
netutils/thttpd/thttpd.c:643:23: error: Mixed case identifier found
netutils/thttpd/thttpd.c:643:63: error: Mixed case identifier found
netutils/thttpd/thttpd.c:730:35: error: Mixed case identifier found
netutils/thttpd/thttpd.c:739:29: error: Mixed case identifier found
This fixes the case when webclient_set_static_body is used
for data larger than webclient_context::buflen.
Note: as of writing this, webclient_set_static_body is the
only user of body_callback in NuttX apps tree.
Uses logic from apps/fsutils/ipcfg to obtain the IP address configuration from a file, if available. This only effects the behavior of netinit if CONFIG_FSUTILS_IPCFG is selected and if an IP configuration file is available on a file system. Otherwise, it reverts to the preceding behavior.
Also adds definitions to apps/include/fsutils/ipcfg.h to formalize a bit encoding of the prototype which was already in use, but not documented.
Also fixes some bad assertions (duplication of PR #420) and makes use of variable names consistent in ipcfg.h comments and in ipcfg_text implementation.
Highlights:
* TLS support (a hook to allow users to provide TLS implementation)
* ability to add extra request headers
* ability to use PUT method
* ability to report http status
* error handling improvements
Proposed on the ML while ago:
https://www.mail-archive.com/dev@nuttx.apache.org/msg03803.html
The original API is kept for now.
I plan to remove them after adapting the existing users.
(examples in this repo)
Summary:
- dhcpc now uses gethostname() but the hostname might be empty
- This commit fixes this issue
Impact:
- All use cases which use dhcpc
Testing:
- Tested with spresense:wifi
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Adapt to the change in the main repo.
mallinfo is meant to be API compatible with Linux,
where it's provided by malloc.h.
(I think the API actually originated with System V. I don't
remember how it was there though. Anyway, I guess the
compatibility with Linux is more important than System V
these days.)
When recv() failed, the current code assumes the return value
of recv() is a negative errno. It's wrong. Actually the return value
in case of error is -1. The wrong assumption ends up with reporting
EPERM, as EPERM happens to be -(-1).
This commit changes the code to leave the errno set by the failed
recv() as it is. It should be fine as wget_base() has the same
error returning convention as recv(). That is, return -1 (ERROR)
and set errno in the case of failure.
NOTE: the close() after the errout label can also fail and overwrite
the errno. I don't feel it's a big problem as wget_base() doesn't have
any promise about which error should be reported.
http -> https redirection is rather common. The old code was
just broken in that case.
Also, this commit is a step towards https support.
* Switch to netlib_parseurl
* Fix error propagation in wget_parseheaders
* Bail out on a redirect to a URL with unsupported scheme
Not all files were correctly modified when usrsock_rpmsg was moved from apps/system to apps/netutils. Changes were overlooked in the Makefile and Make.defs file.
There were several places where default sector and buffer sizes of 2048 were changed to DEFAULT_TASK_STACKSIZE. This is not correct. This was noted by Xiao Xiang.
This is the companion to PR #766. It removes the CONFIG_CAN_PASS_STRUCT option as recommended by Issue #620
NuttX PR #766 depends on PR being in place but not vice versa. This PR should be merge-able without #766 and then PR #766 should also pass its checks.
Where:
* Under examples and testing
* And the default value is less than 2048
Using a bit more memory for examples and tests should not
be a critical problem.
This commit changes only ones with the default 2048 and
leaves the others.
E.g. this leaves SYSTEM_RAMTEST_STACKSIZE, whose default is 1024.
I guess those need to be inspected one-by-one.
Recent change from Xiao Xiang enabled the SO_REUSEADDR socket option unconditionally. This, of course, causes link time failures if socket options are not enabled:
apps/netutils/telnetd/telnetd_daemon.c:182: undefined reference to 'setsockopt'.
Observed during build testing with configuration rddrone-uavcan144:nsh
Author: Xiang Xiao <xiaoxiang@xiaomi.com>
Check POLLHUP and POLLERR in telnet poll loop
to handle the remote end close correctly
Send "NVT"(network virtual terminal) as the default if getenv("TERM") return NULL
telnet should trigger the error handling if inet_pton return zero
since zero mean the string has format error
Don't return 1 in _environ_telnet to avoid trigger the compression
and remove the redundant TELNET_TELOPT_COMPRESS2 check
Change telnet_error_u to telnet_error_e required by the coding standard
Ensure telnet object get freed before the abnormal exit
The sendmail example hasn't been used in years. I am not sure it was ever debugged. It is so old that it still expects hand editted .config files. As a result, all of the configuration settings were missing from the Kconfig file.
This commit adds the missing apps/examples/sendmail configuration settings so that the example at least builds. I cannot test it because I have no smtp server that I am willing to spam.
This configuration also adds missing configuration dependencies and runs the C files in apps/examples/sendmail and apps/netutils/smtp through nxstyle.
This commit resolves an issue reported by surya prakash rased in the Google group: https://groups.google.com/forum/#!topic/nuttx/idr-M164Y24
apps/netutils/dhcpd/dhcpd.c: Fix unused const variable
dhcpd.c:273:29: warning: 'g_anyipaddr' defined but not used [-Wunused-const-variable=]
static const uint8_t g_anyipaddr[4] = {0, 0, 0, 0};
apps/netutils/dhcpd: Make dhcpd interface configurable
apps/netutils/dhcpd: Lease address directly if hit in address range
apps/netutils/netinit: Initialize dhcpc_state
update the addresses only on request success
NuttX commit 346336bb9e5c9199b82e34d08bf23dd1bc389460 make CONFIG_NET_TCP_READADHEAD unselectable and removed all references to CONFIG_NET_TCP_READHEAD from the the NuttX repositories. However, there was no corresponding removal of CONFIG_NET_TCP_READHEAD from the apps/ repository. As a result, there was logic that depended on the selection of CONFIG_NET_TCP_READHEAD that could never be enabled.
This was found during build testing using the configuration olimex-lpc1766stk/thttpd-binfs. THTTPD was not being build fully because it had dependencies on CONFIG_NET_TCP_READHEAD. There resulted int numerous new warnings and, finally, the error:
(.../apps.examples.thttpd.o): in function `thttp_main':
thttpd_main.c:(.text+0x104): undefined reference to `thttpd_main'
* fix warning: format '%u' expects argument of type 'unsigned int *', but argument 3 has type 'uint16_t *
Change-Id: I3eccf8e2cd917f19b7a9edab233b327297d74bb7
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
* fix warning: format '%ld' expects argument of type 'long int', but argument 2 has type 'int'
Change-Id: Ic59a1e9e49256637fa73459c46b8cded036cf971
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
1. This fixes the issue reported in Bitbucket apps/ Issue 15. That is, if fopen() fails, the NULL stream handle would be used in subsequent logic. The handling of the failure was incomplete.
2. Some of the error handling was fixed. In the event of an error, it was return return an error value ONLY if we failure to report the error and even then the wrong error would be reported. This change now preserves and returns the correct error number that is the root cause of the failure.
3. Ran the file through tools/nxstyle and cleaned up many formatting problems.
1.Remove void cast for function because many place ignore the returned value witout cast
2.Replace void cast for variable with UNUSED macro
Change-Id: Ie644129a563244a6397036789c4c3ea83c4e9b09
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
tftpc_get.c: In function 'tftp_write': tftpc_get.c:307:12: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
int fd = (int)ctx;
^
tftpc_get.c: In function 'tftpget': tftpc_get.c:368:57: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
result = tftpget_cb(remote, addr, binary, tftp_write, (void*)fd);
^
ftpc_transfer.c:652:40: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
FAR struct ftpc_session_s *session = (FAR struct ftpc_session_s *)arg1;
^
ftpc_getreply.c: In function 'fptc_getreply': ftpc_getreply.c:218:58:
warning: passing argument 3 of 'wd_start' from incompatible pointer type [-Wincompatible-pointer-types]
ret = wd_start(session->wdog, session->replytimeo, ftpc_timeout, 1, session);
^~~~~~~~~~~~
In file included from
/workspace/mirtos/out/sim/miai/mirtos/nuttx/include/nuttx/sched.h:56:0,
from /workspace/mirtos/out/sim/miai/mirtos/nuttx/include/sched.h:49,
from /workspace/mirtos/out/sim/miai/mirtos/nuttx/include/stdio.h:48,
from ftpc_internal.h:48,
from ftpc_getreply.c:46: /workspace/mirtos/out/sim/miai/mirtos/nuttx/include/nuttx/wdog.h:247:5: note: expected 'wdentry_t {aka void
(*)(int, long long unsigned int)}' but argument is of type 'void (*)(int, uint32_t) {aka void (*)(int, unsigned int)}'
int wd_start(WDOG_ID wdog, int32_t delay, wdentry_t wdentry, int argc, ...);
^~~~~~~~
Some improvements to netutils/cjson and testing/unity
* netutils/cjson/Makefile: fix the error when creating Make.dep
* testing/unity/Makefile: fix the error when creating Make.dep
* include/.gitignore: ignore cJSON headers
* netutils/cjson/Kconfig: update default version to 1.7.12. Previous releases raise compilation warnings about redefined true/false
Approved-by: Gregory Nutt <gnutt@nuttx.org>
netutils/dhcpd: Fixes bug where hwaddr was being used instead of ipaddr in the ARP update.
This bug seems to have been introduced 3 years ago during a rewrite to remove a direct OS call to arp. I'm not sure how others have not run into issues with this. The behavior I was observing was that the DHCPD would go to offer an IP address, but then send a ARP request for the IP instead because it didn't know where to send the data.
Approved-by: Gregory Nutt <gnutt@nuttx.org>
netutils/webserver: Add web server info based on uname()
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Approved-by: Gregory Nutt <gnutt@nuttx.org>
apps/examples/webserver: Fix to compile with CONFIG_NETUTILS_HTTPD_SENDFILE=y
Also, in this PR, some configs such as CONFIG_NETUTILS_HTTPDFILESTATS
can be selected by menuconfig.
Signed-off-by: Masayuki Ishikawa <Masayuki.Ishikawa@jp.sony.com>
Approved-by: Gregory Nutt <gnutt@nuttx.org>
Some improvements could be made by anyone. For example, I know the main routine in perform() shall be split into several parts for readability. I apologize in advance for this kind of spaghetti code, but I was short on time to refactor it. Also chunked HTTP transfer encoding would be a nice contribution from anyone interested. It is detected but not yet supported.
include/ and netutils/: Remove references to CONFIG_DISABLE_SIGNALS. Signals can no longer be disabled.
nshlib/: Remove references to CONFIG_DISABLE_SIGNALS. Signals can no longer be disabled.
system/: Remove references to CONFIG_DISABLE_SIGNALS. Signals can no longer be disabled.
testing/: Remove references to CONFIG_DISABLE_SIGNALS. Signals can no longer be disabled.
examples/: Remove references to CONFIG_DISABLE_SIGNALS. Signals can no longer be disabled.
Squashed commit of the following:
apps/nshlib: Remove NSH initialization. Now uses the common apps/netutils/netinit logic (which was cloned from nshlib to begin with).
apps/netutils/netinit: Clone network initialization logic from NSH to this directory so that it can be available for general use.
The patch provided in issue 135, which adds support for HTTP Chunked Encoding, covers all paths in the webserver app except CGI callbacks. As a result, if a page being served happens to use CGI, it could generate stream content that does not comply with HTTP Chunked Encoding.
The patch attached amends the webserver app's CGI callbacks to use the HTTP Chunked Encoding sender function instead of using send() directly.
The webserver/httpd app is currently broken when script support is enabled (CONFIG_NETUTILS_HTTPD_SCRIPT_DISABLE). The root cause has been tracked down to the "Content-length" not being available ahead of time in this case (length of -1 passed to send_headers() ). On the other hand, the server closing the socket does not result in FIN being sent to the browser either (FIN not supported by NuttX yet).
Simple solution: Add support for HTTP Chunked Encoding to webserver/httpd (attached patch).
The attached patch is simple. It adds a configuration option to enable chunked encoding. When enabled, the implementation will auto-detect the cases where content length is not available ahead of time, and will automatically engage chunked encoding transfers.
Without this patch, the browser/client hangs forever, as it is expecting more data. With this patch, the browser displays the content.
