Write a first draft of whatsnew-2.1.txt

ChangeLog-2.1

@@ -70,7 +70,8 @@ Contains stuff through 3e9612cd81c8a9881ef298fa59c40af343e4f126
    o Allow evbuffer_ptr to point to position 0 in an empty evbuffer (7aeb2fd Nir Soffer)
    o Set the special "not found" evbuffer_ptr consistantly. (e3e97ae Nir Soffer)
    o support adding buffers to other buffers non-destructively (9d7368a Joachim Bauch)
-   o prevent nested multicast references, reworked locking (26041a8 Joachim Bauch)   o New EVBUFFER_EOL_NUL to read NUL-terminated strings from an evbuffer (d7a8b36 Andrea Montefusco, 54142c9)
+   o prevent nested multicast references, reworked locking (26041a8 Joachim Bauch)
+   o New EVBUFFER_EOL_NUL to read NUL-terminated strings from an evbuffer (d7a8b36 Andrea Montefusco, 54142c9)
    o Make evbuffer_file_segment_types adaptable (c6bbbf1)
    o Added evbuffer_add_iovec and unit tests. (aaec5ac Mark Ellzey, 27b5398)
    o Add evbuffer_copyout_from to copy data from the middle of a buffer (27e2225)

whatsnew-2.1.txt

@@ -1,17 +1,389 @@
                          What's new in Libevent 2.1
+                             Nick Mathewson
 
 0. Before we start
 
 0.1. About this document
 
-  This document describes the key differences between Libevent 1.4 and
-  Libevent 2.0, from a user's point of view.  It's a work in progress.
+  This document describes the key differences between Libevent 2.0 and
+  Libevent 2.1, from a user's point of view.  It's a work in progress.
 
   For better documentation about libevent, see the links at
   http://libevent.org/
 
-  To
+  Libevent 2.1 would not be possible without the generous help of
+  numerous volunteers.  For a list of who did what in Libevent 2.1,
+  please see the ChangeLog!
+
+  NOTE: I am very sure that I missed some thing on this list.  Caveat
+  haxxor.
 
 0.2. Where to get help
 
+  Try looking at the other documentation too.  All of the header files
+  have documentation in the doxygen format; this gets turned into nice
+  HTML and linked to from the libevent.org website.
+
+  There is a work-in-progress book with reference manual at
+  http://www.wangafu.net/~nickm/libevent-book/ .
+
+  You can ask questions on the #libevent IRC channel at irc.oftc.net or
+  on the mailing list at libevent-users@freehaven.net.  The mailing list
+  is subscribers-only, so you will need to subscribe before you post.
+
+0.3. Compatibility
+
+  Our source-compatibility policy is that correct code (that is to say,
+  code that uses public interfaces of Libevent and relies only on their
+  documented behavior) should have forward source compatibility: any
+  such code that worked with a previous version of Libevent should work
+  with this version too.
+
+  We don't try to do binary compatibility except within stable release
+  series, so binaries linked against any version of Libevent 2.0 will
+  probably need to be recompiled against Libevent 2.1.1-alpha if you
+  want to use it.  It is probable that we'll break binary compatibility
+  again before Libevent 2.1 is stable.
+
+1. New APIs and features
+
+1.1. New ways to build libevent
+
+  We now provide an --enable-gcc-hardening configure option to turn on
+  GCC features designed for increased code security.
+
+  There is also an --enable-silent-rules configure option to make
+  compilation run more quietly with automake 1.11 or later.
+
+  You no longer need to use the --enable-gcc-warnings option to turn on
+  all of the GCC warnings that Libevent uses.  The only change from
+  using that option now is to turn warnings into errors.
+
+  For IDE users, files that are not supposed to be built are now
+  surrounded with appropriate #ifdef lines to keep your IDE from getting
+  upset.
+
+
+1.2. New functions for events and the event loop
+
+  If you're running Libevent with multiple event priorities, you might
+  want to make sure that Libevent checks for new events frequently, so
+  that time-consuming or numerous low-priority events don't keep it from
+  checking for new high-priority events.  You can now use the
+  event_config_set_max_dispatch_interval() interface to ensure that the
+  loop checks for new events either every N microseconds, every M
+  callbacks, or both.
+
+  There is an EVLOOP_NO_EXIT_ON_EMPTY flag that tells event_base_loop()
+  to keep looping even when there are no pending events.  (Ordinarily,
+  event_base_loop() will exit as soon as no events are pending.)
+
+  Past versions of Libevent have been annoying to use with some
+  memory-leak-checking tools, because Libevent allocated some global
+  singletons but provided no means to free them.  There is now a
+  function, libevent_global_shutdown(), that you can use to free all
+  globally held resources before exiting, so that your leak-check tools
+  don't complain.  (Note: this function doesn't free non-global things
+  like events, bufferevents, and so on; and it doesn't free anything
+  that wouldn't otherwise get cleaned up by the operating system when
+  your process exit()s.  If you aren't using a leak-checking tool, there
+  is not much reason to call libevent_global_shutdown().)
+
+  There is a new event_base_get_npriorities() function to return the
+  number of priorities set in the event base.
+
+  Libevent 2.0 added an event_new() function to construct a new struct
+  event on the heap.  Unfortunately, with event_new(), there was no
+  equivalent for:
+
+         struct event ev;
+         event_assign(&ev, base, fd, EV_READ, callback, &ev);
+
+  In other words, there was no easy way for event_new() to set up an
+  event so that the event itself would be its callback argument.
+  Libevent 2.1 lets you do this by passing "event_self_cbarg()" as the
+  callback argument:
+
+         struct event *evp;
+         evp = event_new(base, fd, EV_READ, callback,
+         event_self_cbarg());
+
+1.3. New debugging features
+
+  You can now turn on debug logs at runtime using a new function,
+  event_enable_debug_logging().
+
+  There's also been some work done to try to make the debugging logs
+  more generally useful.
+
+1.4. New evbuffer functions
+
+  In Libevent 2.0, we introduced evbuffer_add_file() to add an entire
+  file's contents to an evbuffer, and then send them using sendfile() or
+  mmap() as appropriate.  This API had some drawbacks, however.
+  Notably, it created one mapping or fd for every instance of the same
+  file added to any evbuffer.  Also, adding a file to an evbuffer could
+  make that buffer unusable with SSL bufferevents, filtering
+  bufferevents, and any code that tried to read the contents of the
+  evbuffer.
+
+  Libevent 2.1 adds a new evbuffer_file_segment API to solve these
+  problems.  Now, you can use evbuffer_file_segment_new() to construct a
+  file-segment object, and evbuffer_add_file_segment() to insert it (or
+  part of it) into an evbuffer.  These segments avoid creating redundant
+  maps or fds.  Better still, the code is smart enough (when the OS
+  supports sendfile) to map the file when that's necessary, and use
+  sendfile() otherwise.
+
+  The evbuffer_ptr interface has been extended so that an evbuffer_ptr
+  can now yield a point just after the end of the buffer.  This makes
+  many algorithms simpler to implement.
+
+  There's a new evbuffer_add_buffer() interface that you can use to add
+  one buffer to another nondestructively.  When you say
+  evbuffer_add_buffer_reference(outbuf, inbuf), outbuf now contains a
+  reference to the contents of inbuf.
+
+  To aid in adding data in bulk while minimizing evbuffer calls, there
+  is an evbuffer_add_iovec() function.
+
+  There's a new evbuffer_copyout_from() variant function to enable
+  copying data nondestructively from the middle of a buffer.
+
+  evbuffer_readln() now supports an EVBUFFER_EOL_NUL argument to fetch
+  NUL-terminated strings from buffers.
+
+1.5. New functions and features: bufferevents
+
+  You can now use the bufferevent_getcb() function to find out a
+  bufferevent's callbacks.  Previously, there was no supported way to do
+  that.
+
+  The largest chunk readable or writeable in a single bufferevent
+  callback is no longer hardcoded; it's now configurable with
+  the new functions bufferevent_set_max_single_read() and
+  bufferevent_set_max_single_write().
+
+  For consistency, OpenSSL bufferevents now make sure to always set one
+  of BEV_EVENT_READING or BEV_EVENT_WRITING when invoking an event callback.
+
+1.6. New functions and features: evdns
+
+  The previous evdns interface used an "open a test UDP socket" trick in
+  order to detect IPv6 support.  This was a hack, since it would
+  sometimes badly confuse people's firewall software, even though no
+  packets were sent.  The current evdns interface-detection code uses
+  the appropriate OS functions to see which interfaces are configured.
+
+1.7. New functions and features: evconnlistener
+
+  Libevent 2.1 adds the following evconnlistener flags:
+
+    LEV_OPT_DEFERRED_ACCEPT -- Tells the OS that it doesn't need to
+    report sockets as having arrived until the initiator has sent some
+    data too.  This can greatly improve performance with protocols like
+    HTTP where the client always speaks first.  On operating systems
+    that don't support this functionality, this option has no effect.
+
+    LEV_OPT_DISABLED -- Creates an evconnlistener in the disabled (not
+    listening) state.
+
+  Libevent 2.1 changes the behavior of the LEV_OPT_CLOSE_ON_EXEC
+  flag.  Previously, it would apply to the listener sockets, but not to
+  the accepted sockets themselves.  That's almost never what you want.
+  Now, it applies both to the listener and the accepted sockets.
+
+1.8. New functions and ffeatures: evhttp
+
+  **********************************************************************
+  NOTE: The evhttp module will eventually be deprecated in favor of Mark
+  Ellzey's libevhtp library.  Don't worry -- this won't happen until
+  libevhtp provides every feature that evhttp does, and provides a
+  compatible interface that applications can use to migrate.
+  **********************************************************************
+
+  Previously, you could only set evhttp timeouts in increments of one
+  second.  Now, you can use evhttp_set_timeout_tv() and
+  evhttp_connection_set_timeout_tv() to configure
+  microsecond-granularity timeouts.
+
+  There are a new pair of functions: evhttp_set_bevcb() and
+  evhttp_connection_base_bufferevent_new(), that you can use to
+  configure which bufferevents will be used for incoming and outgoing
+  http connections respectively.  These functions, combined with SSL
+  bufferevents, should enable HTTPS support.
+
+  There's a new evhttp_foreach_bound_socket() function to iterate over
+  every listener on an evhttp object.
+
+2. Cross-platform performance improvements
+
+2.1. Better data structures
+
+  We replaced several users of the sys/queue.h "TAILQ" data structure
+  with the "LIST" data structure.  Because this data type doesn't
+  require FIFO access, it requires fewer pointer checks and
+  manipulations to keep it in line.
+
+  All previous versions of Libevent have kept every pending (added)
+  event in an "eventqueue" data structure.  Starting in Libevent 2.0,
+  however, this structure became redundant: every pending timeout event
+  is stored in the timeout heap or in one of the common_timeout queues,
+  and every pending fd or signal event is stored in an evmap.  Libevent
+  2.1 removes this data structure, and thereby saves all of the code
+  that we'd been using to keep it updated.
+
+2.2. Faster activations and timeouts
+
+  It's a common pattern in older code to use event_base_once() with a
+  0-second timeout to ensure that a callback will get run 'as soon as
+  possible' in the current iteration of the Libevent loop.  We optimize
+  this case by calling event_active() directly, and bypassing the
+  timeout pool.  (People who are using this pattern should also consider
+  using event_active() themselves.)
+
+  Libevent 2.0 would wake up a polling event loop whenever the first
+  timeout in the event loop was adjusted--whether it had become earlier
+  or later.  We now only notify the event loop when a change causes the
+  expiration time to become _sooner_ than it would have been otherwise.
+
+  The timeout heap code is now optimized to perform fewer comparisons
+  and shifts when changing or removing a timeout.
+
+  Instead of checking for a wall-clock time jump every time we call
+  clock_gettime(), we now check only every 5 seconds.  This should save
+  a huge number of gettimeofday() calls.
+
+2.3. Microoptimizations
+
+  Internal event list maintainance no longer use the antipattern where
+  we have one function with multiple totally independent behaviors
+  depending on an argument:
+      #define OP1 1
+      #define OP2 2
+      #define OP3 3
+      void func(int operation, struct event *ev) {
+        switch (op) {
+          ...
+        }
+      }
+  Instead, these functions are now split into separate functions for
+  each operation:
+      void func_op1(struct event *ev) { ... }
+      void func_op2(struct event *ev) { ... }
+      void func_op3(struct event *ev) { ... }
+
+  This produces better code generation and inlining decisions on some
+  compilers, and makes the code easier to read and check.
+
+2.4. Evbuffer performance improvements
+
+  The EVBUFFER_EOL_CRLF line-ending type is now much faster, thanks to
+  smart optimizations.
+
+2.5. HTTP performance improvements
+
+   o Performance tweak to evhttp_parse_request_line. (aee1a97 Mark Ellzey)
+   o Add missing break to evhttp_parse_request_line (0fcc536)
+
+
+3. Backend/OS-specific improvements
+
+3.1. Linux-specific improvements
+
+  The logic for deciding which arguements to use with epoll_ctl() is now
+  a table-driven lookup, rather than the previous pile of cascading
+  branches.  This should minimize epoll_ctl() calls and make the epoll
+  code run a little faster on change-heavy loads.
+
+  Libevent now takes advantage of Linux's support for enhanced APIs
+  (e.g., SOCK_CLOEXEC, SOCK_NONBLOCK, accept4, pipe2) that allow us to
+  simultaneously create a socket, make it nonblocking, and make it
+  close-on-exec.  This should save syscalls throughout our codebase, and
+  avoid race-conditions if an exec() occurs after a socket is socket is
+  created but before we can make it close-on-execute on it.
+
+3.2. Windows-specific improvements
+
+  We now use GetSystemTimeAsFileTime to implement gettimeofday.  It's
+  significantly faster and more accurate than our old ftime()-based approach.
+
+3.3. Improvements in the solaris evport backend.
+
+  The evport backend has been updated to use many of the infrastructure
+  improvements from Libevent 2.0.  Notably, it keeps track of per-fd
+  information using the evmap infrastructure, and removes a number of
+  linear scans over recently-added events.  This last change makes it
+  efficient to receive many more events per evport_getn() call, thereby
+  reducing evport overhead in general.
+
+3.4. OSX backend improvements
+
+  The OSX select backend doesn't like to have more than a certain number
+  of fds set unless an "unlimited select" option has been set.
+  Therefore, we now set it.
+
+4. Infrastructure improvements
+
+4.1. Faster tests
+
+  I've spent some time to try to make the unit tests run faster in
+  Libevent 2.1.  Nearly all of this was a matter of searching slow tests
+  for unreasonably long timeouts, and cutting them down to reasonably
+  long delays, though on one or two cases I actually had to parallelize
+  an operation or improve an algorithm.
+
+  On my desktop, a full "make verify" run of Libevent 2.0.18-stable
+  requires about 218 seconds.  Libevent 2.1.1-alpha cuts this down to
+  about 78 seconds.
+
+  Faster unit tests are great, since they let programmers test their
+  changes without losing their train of thought.
+
+4.2. Portability
+
+  Libevent now uses large-file support internally on platforms where it
+  matters.  You shouldn't need to set _LARGEFILE or OFFSET_BITS or
+  anything magic before including the Libevent headers, either, since
+  Libevent now sets the size of ev_off_t to the size of off_t that it
+  received at compile time, not to some (possibly different) size based
+  on current macro definitions when your program is building.
+
+  We now also use the Autoconf AC_USE_SYSTEM_EXTENSIONS mechanism to
+  enable per-system macros needed to enable not-on-by-default features.
+  Unlike the rest of the autoconf macros, we output these to an
+  internal-use-only evconfig-private.h header, since their names need to
+  survive unmangled.  This lets us build correctly on more platforms,
+  and avoid inconsistencies when some files define _GNU_SOURCE and
+  others don't.
+
+  Libevent now tries to detect OpenSSL via pkg-config.
+
+4.3. Standards conformance
+
+  Previous Libevent versions had no consistent convention for internal
+  vs external identifiers, and used identifiers starting with the "_"
+  character throughout the codebase.  That's no good, since the C
+  standard says that identifiers beginning with _ are reserved.  I'm not
+  aware of having any collisions with system identifiers, but it's best
+  to fix these things before they cause trouble.
+
+  We now avoid all use of the _identifiers in the Libevent source code.
+  These changes were made *mainly* through the use of automated scripts,
+  so there shouldn't be any mistakes, but you never know.
+
+  As an exception, the names _EVENT_LOG_DEBUG, _EVENT_LOG_MSG_,
+  _EVENT_LOG_WARN, and _EVENT_LOG_ERR are still exposed in event.h: they
+  are now deprecated, but to support older code, they will need to stay
+  around for a while.  New code should use EVENT_LOG_DEBUG,
+  EVENT_LOG_MSG, EVENT_LOG_WARN, and EVENT_LOG_ERR instead.
+
+5. Testing
+
+  On the bright side, there are an extra ~1000 lines of tests since
+  Libevent 2.0. Sadly, though, there are ~1850 more lines of C code in
+  Libevent itself.  This drops the line-coverage rate of our tests from
+  our previous 80.2% to a miserable 78.8%.  That's better than a lot of
+  programs, but it's still not acceptable! Let's write more unit tests.
+