Write a huge pile of whatsnew-2.0.txt

svn:r1188

whatsnew-2.0.txt

@@ -1,23 +1,32 @@
 What's New In Libevent 2.0 so far:
 
-0. About this document
+1. About this document
 
   This document describes the key differences between Libevent 1.4 and
   Libevent 2.0, from a user's point of view.  It was most recently
-  updated based on features in subversion trunk as of 27 Dec 2007.
+  updated based on features in subversion trunk as of 16 April 2009.
 
   NOTE 1: If any features or fixes get backported from trunk to 1.4,
   they should get moved from here into whatsnew-14.txt, since they
   will no longer be differences between 1.4 and this version.
 
-2. New and Improved APIs
+  NOTE 2: We may have missed some things on this list.  Caveat haxxor.
+
+2. New and Improved Event APIs
 
   Many APIs are improved, refactored, or deprecated in Libevent 2.0.
 
-  All existing code that worked with should Libevent 1.4 should still work
-  correctly with Libevent 2.0.  However, if you are writing new code, or if
-  you want to port old code, we strongly recommend using the new APIs and
-  avoiding deprecated APIs as much as possible.
+  COMPATIBILITY:
+
+  Nearly all existing code that worked with should Libevent 1.4 should still
+  work correctly with Libevent 2.0.  However, if you are writing new code,
+  or if you want to port old code, we strongly recommend using the new APIs
+  and avoiding deprecated APIs as much as possible.
+
+  Binaries linked against Libevent 1.4 will need to be recompiled to link
+  against Libevent 2.0.  This is nothing new; we have never been good at
+  preserving binary compatibility between releases.  We'll try harder in the
+  future, though: see 2.1 below.
 
 2.1. New header layout for improved compatibility
 
@@ -30,15 +39,16 @@ What's New In Libevent 2.0 so far:
 
   There are *backward compatibility headers*, like event2/event_compat.h.
   These headers contain declarations for deprecated functions from older
-  versions of Libevent.  Documentation in these headers should suggest what
-  functions you want to start using instead of the old ones. New programs
-  should generally not include these headers.
+  versions of Libevent.  Documentation in these headers should suggest what's
+  wrong with the old functions, and what functions you want to start using
+  instead of the old ones.  Some of these functions might be removed in a
+  future release.  New programs should generally not include these headers.
 
   Finally, there are *structure headers*, like event2/event_struct.h.
   These headers contain definitions of some structures that Libevent has
   historically exposed.  Exposing them caused problems in the past, since
-  programs that were compiled to work with one version of libevent would
-  often stop working with another version that changed the size of layout
+  programs that were compiled to work with one version of Libevent would
+  often stop working with another version that changed the size or layout
   of some object.  We've moving them into separate headers so that
   programmers can know that their code is not depending on any unstable
   aspect of the Libvent ABI.  New programs should generally not include
@@ -52,6 +62,8 @@ What's New In Libevent 2.0 so far:
   "bufferevent" functions for higher-level buffered IO are in
   event2/bufferevent.h.
 
+  COMPATIBILITY:
+
   All of the old headers (event.h, evdns.h, evhttp.h, evrpc.h, and
   evutil.h) will continue to work by including the corresponding new
   headers.  Old code should not be broken by this change.
@@ -65,15 +77,15 @@ What's New In Libevent 2.0 so far:
 
      1) Dependence on the "current" event_base.  In an application with
         multiple event_bases, Libevent previously had a notion of the
-        "current" event_base.  New events were linked to use this base, and
+        "current" event_base.  New events were linked to this base, and
         the caller needed to explicitly reattach them to another base.
         This was horribly error-prone.
 
-        Functions like "event_set" that worked with the "current" are now
-        deprecated but still available (see 2.1).  There are new functions
-        like "event_assign" that take an explicit event_base argument when
-        setting up a structure.  Using these functions will help prevent
-        errors in your applications, and to be more threadsafe.
+        Functions like "event_set" that worked with the "current" event_base
+        are now deprecated but still available (see 2.1).  There are new
+        functions like "event_assign" that take an explicit event_base
+        argument when setting up a structure.  Using these functions will help
+        prevent errors in your applications, and to be more threadsafe.
 
      2) Structure dependence.  Applications needed to allocate 'struct
         event' themselves, since there was no function in Libevent to do it
@@ -85,6 +97,9 @@ What's New In Libevent 2.0 so far:
         functions.  For example, instead of malloc and event_set, you
         can use event_new().
 
+        (For people who do really want to allocate a struct event on the
+        stack, or put one inside another structure, you can still use
+        event2/event_compat.h.)
 
    So in the case where old code would look like this:
 
@@ -113,7 +128,14 @@ What's New In Libevent 2.0 so far:
   event_set_mem_functions.  It takes replacements for malloc(),
   free(), and realloc().
 
-2.X. Configurable event_base creation
+  If you're going to use this facility, you need to call it _before_
+  Libevent does any memory allocation; otherwise, Libevent may allocate some
+  memory with malloc(), and free it with the free() function you provide.
+
+  You can disable this feature when you are building Libevent by passing
+  the --disable-malloc-replacement argument to configure.
+
+2.4. Configurable event_base creation
 
   Older versions of Libevent would always got the fastest backend
   available, unless you reconfigured their behavior with the environment
@@ -148,14 +170,7 @@ What's New In Libevent 2.0 so far:
       }
       event_config_free(config);
 
-2.4. More flexible readline support
-
-  The old evbuffer_readline() function (which accepted any sequence of
-  CR and LF characters as a newline, and which couldn't handle lines
-  containing NUL characters), is now deprecated.  The preferred
-  function is evbuffer_readln(), which supports a variety of
-  line-ending styles, and which can return the number of characters in
-  the line returned.
+  Supported features are documented in event2/event.h
 
 2.5. Socket is now an abstract type
 
@@ -181,15 +196,13 @@ What's New In Libevent 2.0 so far:
   The callback 'cb' will be invoked whenever fd is ready to read, OR whenever
   a second has passed since the last invocation of cb.
 
-2.X. kqueue event ordering consistency
-
-2.X. Multiple events allowed per fd
+2.7. Multiple events allowed per fd
 
   Older versions of Libevent allowed at most one EV_READ event and at most
   one EV_WRITE event per socket, per event base.  This restriction is no
   longer present.
 
-2.X. evthread_* functions for thread-safe structures.
+2.8. evthread_* functions for thread-safe structures.
 
   Libevent structures can now be built with locking support.  You can
   enable this on a per-event-base level by writing functions to implement
@@ -199,38 +212,214 @@ What's New In Libevent 2.0 so far:
 
   If you want threading support and you're using pthreads, you can just
   call evthread_use_pthreads().  (You'll need to link against the
-  libevent_pthreads library in addition to libevent.)
+  libevent_pthreads library in addition to libevent_core.  These functions are
+  not in libevent_core)
 
   If you want threading support and you're using Windows, you can just
   call evthread_use_windows_threads().
 
-2.X. bufferevent_setfd/cb
+  Once locking functions are enabled, every new event_base is created with a
+  lock.  You can prevent a single event_base from being built with a lock
+  disabled by using the EVENT_BASE_FLAG_NOLOCK flag in its
+  event_config.  If an event_base is created with a lock, it is safe to call
+  event_del, event_add, and event_active on its events from any thread.  The
+  event callbacks themselves are still all executed from the thread running
+  the event loop.
 
-2.X. Bufferevent IO filters (????)
+  To make an evbuffer or a bufferevent object threadsafe, call its
+  _enable_locking() function.
 
-2.X. Edge-triggered events on some backends.
+  The HTTP and DNS apis are not currently threadsafe.
 
-2.X. Multiple callbacks per evbuffer
+  To build Libevent with threading support disabled, pass
+  --disable-thread-support to the configure script.
 
-3. Big bugfixes
+2.9. Edge-triggered events on some backends.
 
-3.X. Win32 bufferevents work
+  With some backends, it's now possible to add the EV_ET flag to an event
+  in order to request that the event's semantics be edge-triggered.  Right
+  now, epoll and kqueue support this.
 
-4. Big performance improvements
+  The corresponding event_config feature is EV_FEATURE_ET; see 2.4 for more
+  information.
 
-4.X. Faster windows backend(s)
+3. Backend-specific improvements.
 
-4.X. Faster evbuffer implementation
+3.1. kqueue event ordering consistency
 
-4.X. Generic notify support
+  TODO(niels)
 
-5. Extras improvements
+3.2. Improved notification on Linux
 
-5.X. DNS: IPv6 nameservers
+  When we need to wake the event loop up from another thread, we use
+  an epollfd to do so, instead of a socketpair.  This is supposed to be
+  faster.
 
-5.X. DNS: 0x20 hack support
+4. Improvements to evbuffers
+
+  Libevent has long had an "evbuffer" implementation to wrap access to an
+  input or output memory buffer.  In previous versions, the implementation
+  was very inefficient and lacked some desirable features.  We've made many
+  improvements in Libevent 2.0.
+
+4.1. Chunked-memory internal representation
+
+  Previously, each evbuffer was a huge chunk of memory.  When we ran out of
+  space in an evbuffer, we used realloc() to grow the chunk of memory.  When
+  data was misaligned, we used memmove to move the data back to the front
+  of the buffer.
+
+  Needless to say, this is a terrible interface for networked IO.
+
+  Now, evbuffers are implemented as a linked list of memory chunks, like
+  most Unix kernels use for network IO.  Data is added at the end of the
+  linked list and removed from the front, so that we don't ever need
+  realloc huge chunks or memmove the whole buffer contents.
+
+  To avoid multiple calls to read and write, we use the readv/writev
+  interfaces (or WSASend/WSARecv on Windows) to do IO on multiple chunks at
+  once with a single system call.
+
+  COMPATIBILITY NOTE:
+  The evbuffer struct is no longer exposed in a header.  The code here is
+  too volatile to expose an official evbuffer structure, and there was never
+  any means provided to create an evbuffer except via evbuffer_new which
+  heap-allocated the buffer.
+
+  If you need access to the whole bufer as a linear chunk of memory, the
+  EVBUFFER_DATA() function still works.  Watch out, though: it needs to copy
+  the buffer's contents in a linear chunk before you can use it.
+
+4.2. More flexible readline support
+
+  The old evbuffer_readline() function (which accepted any sequence of
+  CR and LF characters as a newline, and which couldn't handle lines
+  containing NUL characters), is now deprecated.  The preferred
+  function is evbuffer_readln(), which supports a variety of
+  line-ending styles, and which can return the number of characters in
+  the line returned.
+
+4.3. Support for file-based IO in evbuffers.
+
+  You can now add chunks of a file into a evbuffer, and Libevent will have
+  your OS use mapped-memory functionality, sendfile, or splice to transfer
+  the data without ever copying it to userspace.  On OSs where this is not
+  supported, Libevent just loads the data.
+
+  There are probably some bugs remaining in this code.  On some platforms
+  (like Windows), it just reads the relevant parts of the file into RAM.
+
+4.4. Support for zero-copy writes in evbuffers.
+
+  You can add a piece of memory to an evbuffer without copying it.  Instead,
+  Libevent adds a new element to the evbuffer's linked list of chunks with a
+  pointer to the memory you supplied.
+
+4.5. Multiple callbacks per evbuffer
+
+  Previously, you could only have one callback active on an evbuffer at a
+  time.  In practice, this meant that if one part of Libevent was using an
+  evbuffer callback to notice when an internal evbuffer was reading or
+  writing data, you couldn't have your own callback on that evbuffer.
+
+  Now, you can now use the evbuffer_add_cb() function to add a callback that
+  does not interfere with any other callbacks.
+
+  The evbuffer_setcb() function is now deprecated.
+
+4.6. New callback interface
+
+  Previously, evbuffer callbacks were invoked with the old size of the
+  buffer and the new size of the buffer.  This interface could not capture
+  operations that simultaneously filled _and_ drained a buffer, or handle
+  cases where we needed to postpone callbacks until multiple operations were
+  complete.
+
+  Callbacks that are set with evbuffer_setcb still use the old API.
+  Callbacks added with evbuffer_add_cb() use a new interface that takes a
+  pointer to a struct holding the total number of bytes drained read and the
+  total number of bytes written.  See event2/buffer.h for full details.
+
+4.7. Misc new evbuffer features
+
+   You can use evbuffer_remove() to move a given number of bytes from one
+   buffer to another.
+
+   The evbuffer_search() function lets you search for repeated instances of
+   a pattern inside an evbuffer.
+
+   You can use evbuffer_freeze() to temporarily suspend drains from or adds
+   to a given evbuffer.  This is useful for code that exposes an evbuffer as
+   part of its public API, but wants users to treat it as a pure source or
+   sink.
+
+   You can have an evbuffer defer all of its callbacks, so that rather than
+   being invoked immediately when the evbuffer's length changes, they are
+   invoked from within the event_loop.  This is useful when you have a
+   complex set of callbacks that can change the length of other evbuffers,
+   and you want to avoid having them recurse and overflow your stack.
+
+5. Bufferevents improvements
+
+   Libevent has long included a "bufferevents" structure and related
+   functions that were useful for generic buffered IO on a TCP connection.
+   This is what Libevent uses for its HTTP implementation.  In addition to
+   the improvements that they get for free from the underlying evbuffer
+   implementation above, there are many new features in Libevent 2.0's
+   evbuffers.
+
+5.1. New OO implementations
+
+   The "bufferevent" structure is now an abstract base type with multiple
+   implementations.  This should not break existing code, which always
+   allocated bufferevents with bufferevent_new().
+
+   Current implementations of the bufferevent interface are described below.
+   See also section TODO(nickm).
+
+5.2. bufferevent_socket_new() replaces bufferevent_new()
+
+   Since bufferevents that use a socket are not the only kind,
+   bufferevent_new() is now deprecated.  Use bufferevent_socket_new()
+   instead.
+
+5.3. Filtered bufferevent IO
+
+   You can use bufferevent_filter_new() to create a bufferevent that wraps
+   around another bufferevent and transforms data it is sending and
+   receiving.  See test/regress_zlib.c for a toy example that uses zlib to
+   compress data before sending it over a bufferevent.
+
+5.3. Linked pairs of bufferevents
+
+   You can use bufferevent_pair_new() to produce two linked bufferevents.
+   This is like using socketpair, but doesn't require system-calls.
+
+   IMPLEMENTATION NOTE: These don't support timeouts properly yet.
+
+5.4. Upcoming bufferevent features (NOT DONE YET)
+
+   Nick is writing a bufferevents backend that supports IOCP on Windows.
+   Supposedly, this will make Windows IO much faster for programs using
+   bufferevents.  We'll have to see; the first implementation will probably
+   be quite poor.
+
+   Nick is writing a bufferevents filter to implement OpenSSL over a
+   bufferevents.
+
+6. Extras improvements
+
+6.1. DNS
+
+6.1.1. IPv6 nameservers
+
+   The evdns code now lets you have nameservers whose addresses are IPv6.
+
+6.1.2: Support for the 0x20 hack
+
+6.1.3: Better security.
+
+   TODO(nickm) writeme
 
-5.X. DNS: Better security.
 
 
-6. Removed/Deprecated code and features