* http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2:
http: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR
test: add logging for http/read_on_write_error and rearrange code
http: do not call deferred readcb if readcb is not set
Refs: #749
(cherry picked from commit 7bfe93886d7fccad2d9a6c76cf47c47d3668f9d1)
* travis-ci-osx-fixes:
travis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8
travis-ci/osx: install gcc and fix CC
(cherry picked from commit 5613bfb8dcd70ea1c89d04b550d9f97958cc48d2)
In 755fbf16c ("Add void* arguments to request_new and reply_new
evrpc hooks") this new functions had been introduced, but newer used,
what for? So let's use them.
(cherry picked from commit 99b231b0d875bc0814b0e4a940b6c9890d2a7754)
We should not attemp to establishe the connection if there is retry
timer active, since otherwise there will be a bug.
Imagine next situation:
con = evhttp_connection_base_new()
evhttp_connection_set_retries(con, 2)
req = evhttp_request_new()
evhttp_make_request(con, req, ...)
# failed during connecting, and timer for 2 second scheduler (retry_ev)
Then another request scheduled for this evcon:
evhttp_make_request(con, req, ...)
# got request from server,
# and now it tries to read the response from the server
# (req.kind == EVHTTP_RESPONSE)
#
# but at this point retry_ev scheduled,
# and it schedules the connect again,
# and after the connect will succeeed, it will pick request with
# EVHTTP_RESPONSE for sending and this is completelly wrong and will
# fail in evhttp_make_header_response() since there is no
# "http_server" for this evcon
This was a long standing issue, that I came across few years ago
firstly, bad only now I had time to dig into it (but right now it was
pretty simple, by limiting amount of CPU for the process and using rr
for debug to go back and forth).
(cherry picked from commit f3f7aa5afff2c0be4a1a299a1a7d0a64558abc23)
Could fail from time to time in travis-ci:
https://travis-ci.org/libevent/libevent/jobs/458554097#L1702
Follow-up-for: fe5b0719 ("Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)")
(cherry picked from commit 1d2ef90032bc842bc2e295ee4adce3408b6d85da)
MSVC does not support SHARED and STATIC libraries with the same name,
so let's just build SHARED libraries by default instead (yes we can add
prefix but let's stick with this).
The reason for this is that in windows shared libraries requires .lib
file too, but this is not static library it is imported library for
shared (doh...), for more info [1] and [2].
[1]: https://docs.microsoft.com/en-us/windows/desktop/dlls/dynamic-link-library-creation
[2]: https://blogs.msdn.microsoft.com/oldnewthing/20091013-00/?p=16403
And when we build both static library can and will override shared
library imported part, let's take a look at event_extra.lib:
- before patch [3]:
$ less libevent-fail/lib/Debug/event_extra.lib | head
==> use library:contained_file to view a file in the archive
rw-rw-rw- 100666/100666 59568 Nov 21 23:55 2018 event_extra_static.dir/Debug/evrpc.obj
rw-rw-rw- 100666/100666 252219 Nov 21 23:55 2018 event_extra_static.dir/Debug/evdns.obj
rw-rw-rw- 100666/100666 203850 Nov 21 23:55 2018 event_extra_static.dir/Debug/http.obj
rw-rw-rw- 100666/100666 25907 Nov 21 23:55 2018 event_extra_static.dir/Debug/event_tagging.obj
[3]: https://ci.appveyor.com/project/azat/libevent/builds/20472024/job/t0o93v042jai0dj7
- "after patch" [4] (not after but the same effect):
$ less libevent-ok/lib/Debug/event_extra.lib | head
==> use library:contained_file to view a file in the archive
--------- 0/0 509 Nov 21 23:38 2018 event_extra.dll
...
[4]: https://ci.appveyor.com/project/azat/libevent/builds/20478998/job/ca9k3c76amc4qr76
Refs: #691
(cherry picked from commit 90d80ef4167d97b11e01e80fcc4eaa447712e92f)
Long time ago in [1] cmake build was forced to compile both libraries
(SHARED and STATIC), since this is how our autotools build works.
[1]: 7182c2f561570cd9ceb704623ebe9ae3608c7b43 ("cmake: build SHARED and STATIC libraries (like autoconf does)")
And there is no way to configure this (and indeed you need to do this
for MSVC for example), so let's introduce option for this --
EVENT__LIBRARY_TYPE.
Plus now we have INTERFACE libraries, that we can use internally in
libevent's cmake rules to avoid strict to _shared/_static variant of the
libraries to link with samples/tests (we prefer SHARED over STATIC for
linking).
Also bump minimal cmake required version to 3.1 by the following
reasons:
- 3.1 is required for RPATH configuration under APPLE
- 3.0 is required for add_library(INTERFACE) (did not found it in 2.8.x
documentation)
- remove extra conditions
(anyway 3.1 was release 4 years ago, so I guess that most of the systems
will have it)
(cherry picked from commit c9a073eae8f86a74fbbb125db34b881ef40108b7)
* TT_RETRIABLE:
Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)
regress: introduce TT_RETRIABLE
Fixes: #704
(cherry picked from commit 4d2f013b5d20e674b22e5a8244f7fa63172dbdbf)
We have calls to the next functions but do not check return values,
though they can be invalid and it is better to show this somehow.
Also do bufferevent_setfd() first and only after it
bufferevent_enable()/bufferevent_disable() since:
a) it is more natural
b) it will avoid extra operations
c) it will not fail first bufferevent_enable() (this is the case for
buffbufferevent_async at least)
In this case we could add more information for issues like #709
(cherry picked from commit b98d32d0c90864ff60bc689f3bcf738af58ca79c)
* iocp-fixes:
regress: test for HTTP/HTTPS with IOCP enabled
bev_async: trigger/run only deferred callbacks
bev_async: do not initialize timeouts multiple times
bev_async: set "ok" on setfd if fd>=0 (like we do during creation)
bev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp
Closes: #709
Refs: nmathewson/Libevent#160
(cherry picked from commit 3d815cf22074792f31274fd6d810a94984661dbf)
Although this is not a problem, since bufferevent uses finalizers and
will free itself only from the loop (well this is not a problem if you
do not play games with various event_base in different threads) it
generates questions, so rewrite it in more reliable way.
Fixes: #712
(cherry picked from commit 5dc88b387f7baa4bcd528832e94987a85be3b263)
Refs: 7c8d0152dda18ecc52d3099fea235b04ddb850d9 ("Free event queues even
for recursive finalizers")
(cherry picked from commit f2da6198401e2ef6763e5acc5d4e51a9928b9dde)
Some improvements for http-server sample:
- getopt
- persistent port via -p option
- IOCP for win32 via -I
- disable buffering
- enable debug logging via -v/EVENT_DEBUG_LOGGING_ALL
- cleanup (by signal and separate error path on errors)
* sample-http-server:
s/http-server: graceful cleanup
s/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset
s/http-server: turn off buffering (otherwise do output on win32)
s/http-server: add an option to use IOCP
s/http-server: add options (for persistent port)
Refs: #709
(cherry picked from commit 9a4b8ec1b64fab27544f154076261afdf1efac07)
Even after referenced patch there is still possible recursive callbacks
from evbuffer_drain(bev_input), i.e.:
wm_transfer() -> evbuffer_drain() -> wm_transfer()
inc(ctx->get)
But if we will increment ctx->get before drain that we will not add more
data to buffer.
Refs: 54c6fe3c ("regress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance")
CI: https://ci.appveyor.com/project/nmathewson/libevent/build/job/f0rv299i71wnuxdq#L2546
(cherry picked from commit 1fc1c7ef1dabbbb358b99b893712bccc5761f82d)
various build checks (i.e. detecting headers/macroses/functions) takes
7 minutes (from 13 minutes in total) for cmake, which is too high.
By using cache we can reduce this to ~0.
And set APPVEYOR_SAVE_CACHE_ON_ERROR so that cmake checks will be
cached (anyway all sources will be built from scratch due to timestamp
updates while extracting from sources).
(cherry picked from commit 54fdd6bb6ebe6b0a672c79d30a712d97536500c8)
Otherwise cmake complains:
Policy CMP0075 is not set: Include file check macros honor
CMAKE_REQUIRED_LIBRARIES. Run "cmake --help-policy CMP0075" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.
CMAKE_REQUIRED_LIBRARIES is set to:
ws2_32.lib
For compatibility with CMake 3.11 and below this check is ignoring it.
(cherry picked from commit 6d3a53966bc822fd580a6a02301838aca601391b)
We have $env:OPENSSL_ROOT (env) equals to -DOPENSSL_ROOT (cmake
variable) anyway.
cmake complains:
Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
Run "cmake --help-policy CMP0074" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
Environment variable OpenSSL_ROOT is set to:
C:/OpenSSL-Win64/bin
For compatibility, CMake is ignoring the variable.
This warning is for project developers. Use -Wno-dev to suppress it.
(cherry picked from commit 65904773f2bf965e2050d4d7c91e30d4f123a787)
Due to inplace callbacks (i.e. no BEV_OPT_DEFER_CALLBACKS) we cannot be
sure that wm_transfer() will not be called recursively and indeed it
still happens sometimes, and the referenced patch increase amount of
this times, especially for linux/poll.
Fixes: 66304a23cf748714159c988e78f35401c5352827 ("Fix
ssl/bufferevent_wm_filter when bev does not reach watermark on break")
(cherry picked from commit 54c6fe3c06cb5342130467ecf0f78e3f6d854a36)
EVHTTP_CON_READ_ON_WRITE_ERROR works only if an error already read from
the socket, but if we already got EPIPE on write we cannot read from the
socket anymore, and win32 does not guarantee that read will happens
before (although it happens from time to time).
In the referenced patch I just replaced callback with not expecting 417,
but like I already wrote, this is not always true (i.e. it is flacky).
Fixes: 3b581693ac1967f7f8d98491cb772a1b415eb4cd ("test/http:
read_on_write_error: fix it for win32")
(cherry picked from commit 9040707fb176040acf250b09a223d6b4400f4086)
* ssl_bufferevent_wm_filter-fix:
Fix ssl/bufferevent_wm_filter when bev does not reach watermark on break
regress_ssl: cover watermarks with deferred callbacks
regress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging
(cherry picked from commit 9d93fbe77968da5ca5b2dbe8973e6f3c3045c5ba)
