Commit Graph

93 Commits

Author SHA1 Message Date
YAMAMOTO Takashi
ce2b9519a7 webclient: Add a way to specify timeout 2021-06-15 05:18:14 -05:00
Alin Jerpelea
ec339bc49a Makefiles: Gregory Nutt: update licenses to Apache
Gregory Nutt is the copyright holder for those files and he has submitted the
SGA as a result we can migrate the licenses to Apache.

Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
2021-06-07 21:35:33 -05:00
YAMAMOTO Takashi
95c9007668 webclient: Fix buffer overrun in wget_parsestatus
Similarly to the fix in wget_parseheaders.
But simply always bail out as i guess it's very rare to see
that long status line.

Tested with an aritifically small CONFIG_WEBCLIENT_MAXHTTPLINE=20,
which is smaller than "HTTP/1.1 301 Moved Permanently".
2021-05-16 11:32:47 -03:00
YAMAMOTO Takashi
b53375074b webclient.c: Fix buffer overrun in HTTP header parsing
* Detect a long header line which doesn't fit the buffer.

* If the header line in question doesn't seem important for us,
  just ignore it.

* Otherwise, bail out with -E2BIG.

The overrun was found by LLVM UBSan on macOS.

The following is an example of a long header line,
which doesn't fit the default CONFIG_WEBCLIENT_MAXHTTPLINE=200.
```
pacetanuki% curl -v https://www.midokura.com
:
:
< HTTP/2 200
< server: nginx
< date: Fri, 14 May 2021 02:16:24 GMT
< content-type: text/html; charset=UTF-8
< content-length: 131313
< x-powered-by: PHP/7.4.18
< link: <https://www.midokura.com/wp-json/>; rel="https://api.w.org/", <https://www.midokura.com/wp-json/wp/v2/pages/7>; rel="alternate"; type="application/json", <https://www.midokura.com/>; rel=shortlink
< x-powered-by: PleskLin
<
```
2021-05-16 11:32:47 -03:00
YAMAMOTO Takashi
eb1a99fdd6 webclient.c: Don't parse the status line as the first header line 2021-05-16 11:32:47 -03:00
YAMAMOTO Takashi
f17af21f9c webclient: Add a few ninfo in header parsing 2021-05-16 11:32:47 -03:00
Atsunori Saito
a12f9d967c netutils/webclient: Fixed socket descriptor leak. 2021-05-03 16:44:28 +09:00
YAMAMOTO Takashi
2d0a174cec webclient: 304 and 305 are not really redirects
Eg. The docker API sometimes returns 304 even for requests without etags.
https://docs.docker.com/engine/api/v1.40/#operation/ContainerStop
2021-04-07 06:44:16 -05:00
YAMAMOTO Takashi
15c65b23c7 netutils/webclient/webclient.c: Appease the latest nxstyle 2021-04-07 06:44:16 -05:00
YAMAMOTO Takashi
f613da4f92 webclient.c: Remove a space after a negative sign
from:

    return - ECONNABORTED;

to:

    return -ECONNABORTED;

The latter style is more commonly used within NuttX code base.

Note: nxstyle doesn't complain on either of them.
2021-02-12 08:51:23 +00:00
YAMAMOTO Takashi
412cf2b955 webclient: Fix a use of uninitialized variable
Detected by clang static analyzer.
2021-01-19 19:06:24 -08:00
YAMAMOTO Takashi
ee1f4fdcdb webclient: Don't call the sink callback if no data is available
This happened frequently for me with Docker Desktop's
/var/run/docker.sock on macOS.
But I believe it can happen on other environments, even with TCP.

Alternatively, this case can be handled by the callback
implementations.  But it's simpler to handle the corner case here.
2021-01-17 18:44:17 -08:00
YAMAMOTO Takashi
1131cbe14d webclient: Implement AF_LOCAL 2021-01-17 18:44:17 -08:00
YAMAMOTO Takashi
ebe3b0d302 webclient: Report unexpected connection loss as an explicit error
Otherwise, it can end up with mysterious results in user apps.
2021-01-13 20:51:06 -06:00
YAMAMOTO Takashi
73fb7baa05 webclient: Distinguish request size and buffer size in body callback
This fixes the case when webclient_set_static_body is used
for data larger than webclient_context::buflen.

Note: as of writing this, webclient_set_static_body is the
only user of body_callback in NuttX apps tree.
2020-11-19 18:46:38 -08:00
YAMAMOTO Takashi
a1db71fa43 webclient improvements
Highlights:

* TLS support (a hook to allow users to provide TLS implementation)
* ability to add extra request headers
* ability to use PUT method
* ability to report http status
* error handling improvements

Proposed on the ML while ago:
https://www.mail-archive.com/dev@nuttx.apache.org/msg03803.html

The original API is kept for now.
I plan to remove them after adapting the existing users.
(examples in this repo)
2020-10-01 15:32:25 +08:00
YAMAMOTO Takashi
b78370dbf2 webclient: move WGET_USE_URLENCODE definiton to the header
Because it's used by the header.

This fixes warnings while building wgetjson.
2020-09-30 08:22:16 +02:00
SPRESENSE
78e8090842 examples: wget: Fix nxstyle issues 2020-08-18 10:07:47 -05:00
SPRESENSE
7a1d626504 netutils/webclient: Enable to send long HTTP request
Call send() repeatedly until whole buffer to be sent.
2020-08-18 10:07:47 -05:00
Xiang Xiao
deaa6c5b7b build: Replace $(TOPDIR)/Make.defs with $(APPDIR)/Make.defs
and move NUTTXLIB defintion to the common place

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2020-07-09 15:17:37 -03:00
YAMAMOTO Takashi
83324b68f9 webclient: Fix errno in case of recv failure
When recv() failed, the current code assumes the return value
of recv() is a negative errno. It's wrong. Actually the return value
in case of error is -1. The wrong assumption ends up with reporting
EPERM, as EPERM happens to be -(-1).

This commit changes the code to leave the errno set by the failed
recv() as it is. It should be fine as wget_base() has the same
error returning convention as recv(). That is, return -1 (ERROR)
and set errno in the case of failure.

NOTE: the close() after the errout label can also fail and overwrite
the errno. I don't feel it's a big problem as wget_base() doesn't have
any promise about which error should be reported.
2020-06-12 05:22:45 -07:00
YAMAMOTO Takashi
a455f13ec1 webclient: Check the return value of netlib_parseurl correctly 2020-05-28 19:58:53 +08:00
YAMAMOTO Takashi
415010b7fd webclient: include string.h for strlen
Instead of relying on indirect namespace pollutions.
2020-05-28 11:15:18 +02:00
YAMAMOTO Takashi
276bf42e0d webclient: Fix http -> https redirection
http -> https redirection is rather common. The old code was
just broken in that case.

Also, this commit is a step towards https support.

* Switch to netlib_parseurl
* Fix error propagation in wget_parseheaders
* Bail out on a redirect to a URL with unsupported scheme
2020-05-26 15:38:12 +08:00
YAMAMOTO Takashi
e939e55a50 webclient.c: Limit the number of redirections 2020-05-25 23:54:38 +08:00
Xiang Xiao
69f013e74e build: Remve the unnecessary .gitignore
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2020-05-23 15:56:35 +01:00
Xiang Xiao
ead498a788 build: Remove the workaround for the inexistence of .config/Make.defs
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2020-05-21 17:43:42 +01:00
Gregory Nutt
b2d8c501f6 Remove use of set_errno() from application code.
The normal assignment errno = errcode should work fine now.
2020-05-07 22:23:19 +01:00
Juha Niskanen
2aabd1c9a7 webclient: run through nxstyle
Signed-off-by: Juha Niskanen <juha.niskanen@haltian.com>
2020-04-23 09:29:36 -06:00
Juha Niskanen
f8f16975a9 webclient: use getaddrinfo to get IPv4 addresses on dual stack systems
Signed-off-by: Juha Niskanen <juha.niskanen@haltian.com>
2020-04-23 09:29:36 -06:00
Xiang Xiao
1511403fdf Revert "Don't generate .depend anymore"
This reverts commit cc5ad09caa.
2020-03-22 23:09:40 -05:00
Xiang Xiao
cc5ad09caa
Don't generate .depend anymore 2020-03-22 18:15:21 +00:00
Xiang Xiao
e0dcfa0c55 Remove extra whitespace from files (#43)
* Remove multiple newlines at the end of file
* Remove the white space from the end of line
2020-01-31 08:29:24 -06:00
Xiang Xiao
857158451b Unify the void cast usage
1.Remove void cast for function because many place ignore the returned value witout cast
2.Replace void cast for variable with UNUSED macro

Change-Id: Ie644129a563244a6397036789c4c3ea83c4e9b09
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2020-01-02 23:21:01 +08:00
Xiang Xiao
7808eb62d2 apps/: In all Make.def files, append to CONFIGURED_APPS patch with the absolute path. 2019-10-17 11:33:59 -06:00
Gregory Nutt
3a23523147 Makefiles: This reverts part of commit cf0365ea9. It restores 'conditional' inclusion of TOPDIR/Make.defs. Otherwise all make targets fail if the board has not been configured. That is okay most of the time, but not for things like clean and distclean which should not depend on being configured. 2019-10-15 09:25:48 +08:00
Xiang Xiao
cf0365ea92 Clean up Makefile under apps folder no functional changes. 2019-10-04 08:35:46 -06:00
Gregory Nutt
9e1cda95fb Cosmetic fixes to spacing. 2019-07-27 08:49:34 -06:00
Alan Carvalho de Assis
5492fcdafa apps//netutils/webclient/Kconfig: NET_SOCKOPTS is needed to get webclient compiled. 2019-02-24 13:55:19 -06:00
anchao
b5cfd93444 apps/: Modification to build system: Unified application compilation rules 2018-09-03 09:29:56 -06:00
Gregory Nutt
7a8561728c The last change added .PRECIOUS: $(BIN) to everyway Makefile in hopes the libapps.a will not be deleted on Control-C (unproven so far). However, $(BIN) may contain back slashes in some build platforms and that causes problems for make: It can't handle the backslashes in the .PRECIOUS dependency list. $(BIN) replaced with ../../libapp.a, where the number of ../ depends on the depth of the Makefile in the apps/ source tree. 2017-08-13 11:03:17 -06:00
Gregory Nutt
a66da34f18 All Makefiles: Add .PRECIOUS: apps/libapps.a to every Makefile. Hopefully this will end awkward problems when you Control-C out of a build and libapps.a is deleted. 2017-08-13 09:35:14 -06:00
Gregory Nutt
81645fe18f strcasecmp, strncasecmp, bzero, bcmp, and bcopy should be defined in strings.h, not string.h. bzero, bcmp, and bcopy are legacy functions; the contemporary counterparts should be used instead. 2017-02-16 15:58:15 -06:00
Sebastien Lorquet
2cbad44f1d This is the Pokemon patch to change all includes fronm <apps/bla/bla.h> to "bla/bla.h" 2016-07-11 10:11:18 -06:00
Gregory Nutt
a50383db3d apps/graphics and netutils: Change some *err() message to *info() messages if what was a *dbg() message does not indicate and error condition. 2016-06-14 17:53:47 -06:00
Gregory Nutt
586b0aa7e0 Change names of *dbg() * *err() 2016-06-11 15:51:27 -06:00
Gregory Nutt
344c92e2a4 Replace all occurrences of vdbg with vinfo 2016-06-11 11:55:38 -06:00
Sebastien Lorquet
bfff5e7179 Add empty preconfig rules to 'leaf' makefiles 2016-06-02 16:24:30 +02:00
Sebastien Lorquet
9a406515d7 make the full tree use wildcards make.defs 2016-06-02 16:21:21 +02:00
Gregory Nutt
e9447c6058 Standardize nameing of the pre-processor definitiongs group header 2015-10-02 14:06:11 -06:00