170 Commits

Author SHA1 Message Date
David van Moolenbroek
241ebcae9b RMIB: expose full node path; improve restartability
A single function may be used to handle the implementation of more
than one node.  In some cases, the behavior of that function may
depend on the path name used to reach the node.  Therefore, provide
the full path name as part of the call information.

As a result, RMIB has to save the paths for each of its remote MIB
mount points.  That in turn also allows it to autonomously remount its
mount points after a MIB service restart, thus bringing us a step
closer to proper recovery after a MIB crash without requiring the
service using RMIB to perform explicit steps.  As before, the missing
ingredient is actual notification of MIB service restarts, and proper
support for *that* will likely require changes to the DS service.

Change-Id: Ic0c79931d6f3a76c2c998047f8b47350fd0fa5b0
2017-03-09 23:39:55 +00:00
David van Moolenbroek
bfa518c7ec PM/libsys: extend getepinfo, add getsockcred(3)
The service-only getepinfo(2) PM call returns information about a
given endpoint.  This patch extends that call so that it returns
enough information to allow correctly filling a sockcred structure.
A new getsockcred(3) function is added to libsys to fill an actual
sockcred structure with the obtained information.  However, for the
caller's convenience, the groups list is kept separate.

Change-Id: I9f1a6d1a221c77eabaa3498ff4ec9a5fb922e4fd
2017-03-09 23:39:55 +00:00
David van Moolenbroek
462713495a libsys: retire getnucred in favor of getepinfo
The getnucred() function was used by UDS to obtain credentials of user
processes in a form used in the UDS API, namely the ucred structure.
Since the NetBSD merge, this structure has changed drastically (aside
from being renamed to "uucred"), and it is no longer in UDS's best
interest to use this structure internally.  Therefore, getnucred() is
no longer a useful API either, and instead we directly use the
previously private getepinfo() function to obtain credentials.

Change-Id: I80bc809de716ec0a9b7497cb109d2f2708a629d5
2017-03-09 23:39:54 +00:00
David van Moolenbroek
dd96967135 UDS: prepare for socket file creation in bind(2)
This patch prepares for moving of the creation of socket files on the
file system from the libc bind(2) stub into the UDS service.  This
change is necessary for the socket type agnostic libc implementation.
The change is not yet activated - the code that is not yet used is
enclosed in "#if NOT_YET" blocks.  The activation needs to be atomic
with UDS's switch to libsockdriver; otherwise, user applications may
break.

As part of the change, various UDS bind(2) semantics are changed to
match the POSIX standard and other operating systems.  In
implementation terms, the service-only VFS API checkperms(2) is
renamed to socketpath(2), and extended with a new subcall which
creates a new socket file.  An extension to test56 checks the new
bind(2) semantics of UDS, although most new tests are still disabled
until activation as well.

Finally, as further preparation for a more structural redesign of the
UDS service, also return the <device,inode> number pair for the
created or checked file name, and make returning the canonized path
name optional.

Change-Id: I892d04b3301d4b911bdc571632ddde65fb747a8a
2017-03-09 23:39:53 +00:00
David van Moolenbroek
4c27a83389 Add libsockevent: a socket event dispatching library
This library provides an event-based abstraction model and dispatching
facility for socket drivers.  Its main goal is to eliminate any and
all need for socket drivers to keep track of pending socket calls.
Additionally, this library takes over responsibility of a number of
other tasks that would otherwise be duplicated between socket drivers,
but in such a way that individual socket drivers retain a large degree
of freedom in terms of API behavior.  The library's main features are:

- suspension, resumption, and cancellation of socket calls;
- an abstraction layer for select(2);
- state tracking of shutdown(2);
- pending (asynchronous) errors and the SO_ERROR socket option;
- listening-socket tracking and the SO_ACCEPTCONN socket option;
- generation of SIGPIPE signals; SO_NOSIGPIPE, MSG_NOSIGNAL;
- send and receive low-watermark tracking, SO_SNDLOWAT, SO_RCVLOWAT;
- send and receive timeout support and SO_SNDTIMEO, SO_RCVTIMEO;
- an abstraction layer for the SO_LINGER socket option;
- tracking of various on/off socket options as well as SO_TYPE;
- a range of pre-checks on socket calls that are required POSIX.

In order to track per-socket state, the library manages an opaque
"sock" object for each socket.  The allocation of such objects is left
entirely to the socket driver.  Each sock object has an associated
callback table for calls from libsockevent to the socket driver.  The
socket driver can raise events on the sock object in order to flag
that any previously suspended operations of a particular type should
be resumed.  The library may defer processing such raised events if
immediate processing could interfere with internal consistency.

The sockevent library is layered on top of libsockdriver, and should
be used by all socket driver implementations if at all possible.

Change-Id: I3eb2c80602a63ef13035f646473360293607ab76
2017-03-09 23:39:53 +00:00
David van Moolenbroek
85723df033 Add libsockdriver: a library for socket drivers
This library provides abstractions for socket drivers, and should be
used as the basis for all socket driver implementations.  It provides
the following functionality:

  - a function call table abstraction, hiding the details of the
    socket driver protocol with simple parameters and presenting the
    socket driver with callback functions very similar to the BSD
    socket API calls made from userland;
  - abstracting data structures and helper functions for suspending
    and resuming blocking calls;
  - abstracting data structures and helper functions for copying data
    from and to the caller.

Overall, the library is similar to lib{block,char,fs,input,net}driver
in concept.  Some of the abstractions provided here should in fact be
applied to libchardriver as well.  As always, for the case that the
provided message loop is too restrictive, a set of more low-level
message processing functions is provided.

Change-Id: I79ec215f5e195c3b0197e223636f987d3755fb13
2017-03-09 23:39:52 +00:00
David van Moolenbroek
491d647a3b VFS: support for suspending close(2) for sockets
This change effectively adds the VFS side of support for the SO_LINGER
socket option, by allowing file descriptor close operations to be
suspended (and later resumed) by socket drivers.  Currently, support
is limited to the close(2) system call--in all other cases where file
descriptors are closed (dup2, close-on-exec, process exit..), the
close operation still completes instantly.  As a general policy, the
close(2) return value will always indicate that the file descriptor
has been closed: either 0, or -1 with errno set to EINPROGRESS.  The
latter error may be thrown only when a suspended close is interrupted
by a signal.

As necessary for UDS, this change also introduces a closenb(2) system
call extension, allowing the caller to bypass blocking SO_LINGER close
behavior.  This extension allows UDS to avoid blocking on closing the
last reference to an in-flight file descriptor, in an atomic fashion.
The extension is currently part of libsys, but there is no reason why
userland would not be allowed to make this call, so it is deliberately
not protected from use by userland.

Change-Id: Iec77d6665232110346180017fc1300b1614910b7
2017-03-09 23:39:50 +00:00
David van Moolenbroek
181fb1b2b5 RS: add infrastructure for mapping socket drivers
This patch introduces the first piece of support for the concept of
"socket drivers": services that implement one or more socket protocol
families.  The latter are also known as "domains", as per the first
parameter of the socket(2) API.  More specifically, this patch adds
the basic infrastructure for specifying that a particular service is
the socket driver for a set of domains.

Unlike major number mappings for block and character drivers, socket
domain mappings are static.  For that reason, they are specified in
system.conf files, using the "domain" keyword.  Such a keyword is to
be followed by one or more protocol families, without their "PF_"
prefix.  For example, a service with the line "domain INET INET6;"
will be mapped as the socket driver responsible for the AF_INET and
AF_INET6 protocol families.

This patch implements only the infrastructure for creating such
mappings; the actual mapping will be implemented in VFS in a later
patch.  The infrastructure is implemented in service(8), RS, and VFS.

For now there is a hardcoded limit of eight domains per socket driver.
This may sound like a lot, but the upcoming new LWIP service will
already use four of those.  Also, it is allowed for a service to be
both a block/character driver and a socket driver at the same time,
which is a requirement for the new LWIP service.

Change-Id: I93352d488fc6c481e7079248082895d388c39f2d
2017-03-09 23:39:49 +00:00
David van Moolenbroek
a1c660069f libc: switch to NetBSD getifaddrs(3)
Change-Id: I698f7cef84506ebd7beed6974a232eb75ab853c3
2017-03-09 23:39:48 +00:00
David van Moolenbroek
c58da9fbc3 Rename MINIX service(8) to minix-service(8)
IMPORTANT: this change has a docs/UPDATING entry!

This rename is unfortunately necessary because NetBSD has decided to
create its own service(8) utility, and we will want to import theirs
as well.  The two can obviously not coexist.

Also move ours from /bin to /sbin, as it is a superuser-only utility.

Change-Id: Ic6e46ffb3a84b4747d2fdcb0d74e62dbea065039
2017-02-22 17:16:21 +00:00
David van Moolenbroek
5f6c420586 Retire env.h
This was a MINIX3-specific header file placed outside of the minix/
header subdirectory, with its definitions duplicated in the more
standard minix/sysutil.h header.

Also make env_prefix(3) take constant pointers.

Change-Id: I243c38eb38e24eb98f0c0dddf7f340e7fec255f4
2017-02-16 10:22:27 +00:00
David van Moolenbroek
44fdeb7a62 libc: more poll(3) wrapper fixes
- POLLRDBAND is reported by select(2) as errorfd, not readfd;
- POLLERR is not the same as errorfd of select(2);
- flags that are not requested should not be returned.

Change-Id: I9cb3c2c260ead5a2852a2fbbc10280c2b5b0dff9
2017-02-16 10:18:26 +00:00
David van Moolenbroek
6dd801ffb8 libaudiodriver: buffer accounting bugfix
This fixes #191.

Change-Id: I93752161e466b2d03bbcb0838705aab9619c05c8
2016-12-28 13:12:58 +00:00
David van Moolenbroek
192c3a97f6 libaudiodriver: readd callback to drv_int_sum()
The callback, which was dropped in commit git-842c4ed, allows drivers
to fetch the interrupt status once and save it locally for subsequent
calls to drv_int().

This fixes #190 .

Change-Id: I83918656f637e716f60e9f4c19f1498f761d3b52
2016-12-28 13:08:23 +00:00
David van Moolenbroek
1bb466dd36 libc: make posix_spawn(3) clean up child on failure
Change-Id: I39a321f23326485fca789e5792a57532d1036716
2016-12-28 13:05:55 +00:00
rlfnb
fee60e45e6 introducing libacpi
Change-Id: I0808545fefaefc9a8fc8d1101bd85b676467fea0
2016-10-29 17:08:10 +02:00
David van Moolenbroek
f221d2ce48 RMIB: add support for vector copy-out
Change-Id: I9e9b4b8d6eed39fdb511c6bd2a375ddf898064a5
2016-10-24 12:10:34 +00:00
David van Moolenbroek
6956dd2b51 libc: bugfixes for minix's poll(3) wrapper
- clear "revents" fields even when the call times out;
- do not call FD_ISSET with a negative file descriptor number.

Change-Id: I7aeaae79e73e39aed127a75495ea08256b18c182
2016-10-18 14:18:41 +02:00
David van Moolenbroek
3ac58492b3 Add LLVM GCOV coverage support
With this patch, it is now possible to generate coverage information
for MINIX3 system services with LLVM.  In particular, the system can
be built with MKCOVERAGE=yes, either with a native "make build" or
with crosscompilation.  Either way, MKCOVERAGE=yes will build the
MINIX3 system services with coverage profiling support, generating a
.gcno file for each source module.  After a reboot it is possible to
obtain runtime coverage data (.gcda files) for individual system
services using gcov-pull(8).  The combination of the .gcno and .gcda
files can then be inspected with llvm-cov(1).

For reasons documented in minix.gcov.mk, only system service program
modules are supported for now; system service libraries (libsys etc.)
are not included.  Userland programs are not affected by MKCOVERAGE.

The heart of this patch is the libsys code that writes data generated
by the LLVM coverage hooks into a serialized format using the routines
we already had for GCC GCOV.  Unfortunately, the new llvm_gcov.c code
is LLVM ABI dependent, and may therefore have to be updated later when
we upgrade LLVM.  The current implementation should support all LLVM
versions 3.x with x >= 4.

The rest of this patch is mostly a light cleanup of our existing GCOV
infrastructure, with as most visible change that gcov-pull(8) now
takes a service label string rather than a PID number.

Change-Id: I6de055359d3d2b3f53e426f3fffb17af7877261f
2016-09-24 22:18:31 +00:00
Jean-Baptiste Boric
6afe26749a Enable building with MKDEBUG and MKDEBUGLIB
The way these options work is by creating files that contain debugging
symbols and stashing them in a dedicated set. The minix-debug set has
been created for this purpose, but it will probably have to be refined
since it has been tested only with the default options with an i386
cross-build.

LSC: Amended to support many combination of MKDEBUG, MKDEBUGLIB, with
     and without X11, for both intel and arm.

Change-Id: I2901952e8229938f9ac79c8656484acf704ccd9b
2016-09-10 12:42:45 +02:00
rlfnb
3c71801e8e made PAE enabled by accident and fixed libmagicrt.h
Change-Id: Ia09d8b9a68aa2e1522d66ed93453de9d7d802cb2
2016-08-06 10:52:09 +02:00
David van Moolenbroek
815afbad33 libmthread: resolve memory leaks on exception path
If libmthread runs into a memory allocation failure while attempting
to enlarge its thread pool, it does not free up any preliminary
allocations made so far.

Reported by dcb314.

This closes #152.

Change-Id: Ib882a4544e4802a0eb0a53446b43997876cde633
2016-08-05 11:17:30 +00:00
David van Moolenbroek
1ccb488d84 libsys: fix format specifier in RMIB code
Reported by dcb314.

This closes #141.

Change-Id: I26011870891f5ba22844c335af6081ee9f05c12c
2016-08-05 11:16:43 +00:00
David van Moolenbroek
232819dd49 VFS: store process suspension state as union
Previously, VFS would use various subsets of a number of fproc
structure fields to store state when the process is blocked
(suspended) for various reasons.  As a result, there was a fair
amount of abuse of fields, hidden state, and confusion as to
which fields were used with which suspension states.

Instead, the suspension state is now split into per-state
structures, which are then stored in a union.  Each of the union's
structures should be accessed only right before, during, and right
after the fp_blocked_on field is set to the corresponding blocking
type.  As a result, it is now very clear which fields are in use
at which times, and we even save a bit of memory as a side effect.

Change-Id: I5c24e353b6cb0c32eb41c70f89c5cfb23f6c93df
2016-08-05 11:15:15 +00:00
David van Moolenbroek
cfd712b424 Various timer improvements
Now that clock_t is an unsigned value, we can also allow the system
uptime to wrap.  Essentially, instead of using (a <= b) to see if time
a occurs no later than time b, we use (b - a <= CLOCK_MAX / 2).  The
latter value does not exist, so instead we add TMRDIFF_MAX for that
purpose.

We must therefore also avoid using values like 0 and LONG_MAX as
special values for absolute times.  This patch extends the libtimers
interface so that it no longer uses 0 to indicate "no timeout".
Similarly, TMR_NEVER is now used as special value only when
otherwise a relative time difference would be used.  A minix_timer
structure is now considered in use when it has a watchdog function set,
rather than when the absolute expiry time is not TMR_NEVER.  A few new
macros in <minix/timers.h> help with timer comparison and obtaining
properties from a minix_timer structure.

This patch also eliminates the union of timer arguments, instead using
the only union element that is only used (the integer).  This prevents
potential problems with e.g. live update.  The watchdog function
prototype is changed to pass in the argument value rather than a
pointer to the timer structure, since obtaining the argument value was
the only current use of the timer structure anyway.  The result is a
somewhat friendlier timers API.

The VFS select code required a few more invasive changes to restrict
the timer value to the new maximum, effectively matching the timer
code in PM.  As a side effect, select(2) has been changed to reject
invalid timeout values.  That required a change to the test set, which
relied on the previous, erroneous behavior.

Finally, while we're rewriting significant chunks of the timer code
anyway, also covert it to KNF and add a few more explanatory comments.

Change-Id: Id43165c3fbb140b32b90be2cca7f68dd646ea72e
2016-08-05 11:12:44 +00:00
rlfnb
bf609e1012 Preparations for PAE support
Change-Id: I9a7c9ed9f803e2e26d745d14149a8aec64ab0c75
2016-08-04 09:24:42 +02:00
Antoine Leca
55a5a21b97 Clean up traces from long-gone sys_int86 interface
Change-Id: I773ab67342be852cc74f76b4fcbb338c9cb11f7e
2016-07-20 19:00:44 +02:00
David van Moolenbroek
534584945c IPC: use RMIB to handle kern.ipc sysctl subtree
With this patch, the IPC service is changed to use the new RMIB
facility to register and handle the "kern.ipc" sysctl subtree itself.
The subtree was previously handled by the MIB service directly.  This
change improves locality of handling: especially the
kern.ipc.sysvipc_info node has some peculiarities specific to the IPC
service and is therefore better handled there.  Also, since the IPC
service is essentially optional to the system, this rearrangement
yields a cleaner situation when the IPC service is not running: in
that case, the MIB service will expose a few basic kern.ipc nodes
indicating that no SysV IPC facilities are present.  Those nodes will
be overridden through RMIB when the IPC service is running.

It should be easier to add the remaining (from NetBSD) kern.ipc nodes
as well now.

Test88 is extended with a new subtest that verifies that sysctl-based
information retrieval for semaphore sets works as expected.

Change-Id: I6b7730e85305b64cfd8418c0cc56bde64b22c584
2016-06-18 12:47:24 +00:00
David van Moolenbroek
6f3e0bcd3d MIB/libsys: support for remote MIB (RMIB) subtrees
Most of the nodes in the general sysctl tree will be managed directly
by the MIB service, which obtains the necessary information as needed.
However, in certain cases, it makes more sense to let another service
manage a part of the sysctl tree itself, in order to avoid replicating
part of that other service in the MIB service.  This patch adds the
basic support for such delegation: remote services may now register
their own subtrees within the full sysctl tree with the MIB service,
which will then forward any sysctl(2) requests on such subtrees to the
remote services.

The system works much like mounting a file system, but in addition to
support for shadowing an existing node, the MIB service also supports
creating temporary mount point nodes.  Each have their own use cases.
A remote "kern.ipc" would use the former, because even when such a
subtree were not mounted, userland would still expect some of its
children to exist and return default values.  A remote "net.inet"
would use the latter, as there is no reason to precreate nodes for all
possible supported networking protocols in the MIB "net" subtree.

A standard remote MIB (RMIB) implementation is provided for services
that wish to make use of this functionality.  It is essentially a
simplified and somewhat more lightweight version of the MIB service's
internals, and works more or less the same from a programmer's point
of view.  The most important difference is the "rmib" prefix instead
of the "mib" prefix.  Documentation will hopefully follow later.

Overall, the RMIB functionality should not be used lightly, for
several reasons.  First, despite being more lightweight than the MIB
service, the RMIB module still adds substantially to the code
footprint of the containing service.  Second, the RMIB protocol not
only adds extra IPC for sysctl(2), but has also not been optimized for
performance in other ways.  Third, and most importantly, the RMIB
implementation also several limitations.  The main limitation is that
remote MIB subtrees must be fully static.  Not only may the user not
create or destroy nodes, the service itself may not either, as this
would clash with the simplified remote node versioning system and
the cached subtree root node child counts.  Other limitations exist,
such as the fact that the root of a remote subtree may only be a
node-type node, and a stricter limit on the highest node identifier
of any child in this subtree root (currently 4095).

The current implementation was born out of necessity, and therefore
it leaves several improvements to future work.  Most importantly,
support for exit and crash notification is missing, primarily in the
MIB service.  This means that remote subtrees may not be cleaned up
immediately, but instead only when the MIB service attempts to talk
to the dead remote service.  In addition, if the MIB service itself
crashes, re-registration of remote subtrees is currently left up to
the individual RMIB users.  Finally, the MIB service uses synchronous
(sendrec-based) calls to the remote services, which while convenient
may cause cascading service hangs.  The underlying protocol is ready
for conversion to an asynchronous implementation already, though.

A new test set, testrmib.sh, tests the basic RMIB functionality.  To
this end it uses a test service, rmibtest, and also reuses part of
the existing test87 MIB service test.

Change-Id: I3378fe04f2e090ab231705bde7e13d6289a9183e
2016-06-18 12:46:59 +00:00
David van Moolenbroek
53d2fa057e Do not hide the MSG_NOSIGNAL flag
Instead, filter it in libc for old networking implementations, as
those do not support sending SIGPIPE to user processes anyway.  This
change allows newer socket drivers to implement the flag as per the
specification.

Change-Id: I423bdf28ca60f024a344d0a73e2eab38f1b269da
2016-06-18 12:20:48 +00:00
David van Moolenbroek
91d0779762 libchardriver: make some pointers constant
Change-Id: Ieabc6962d10f9f6e0db7807fd24add212bcfc148
2016-06-18 12:20:45 +00:00
David van Moolenbroek
dc2c582f36 Correct bad assignments in various conditions
Reported by dcb314.

This fixes #128, #129, #130, #131, #132, #133.

Change-Id: I284d6dd87fba7c5775bea22d04412d685a2ab027
2016-06-17 20:07:55 +00:00
David van Moolenbroek
ee0384891a MFS: do not crash when reading superblock fails
While MFS failing to do I/O on a block is generally fatal, reading
the superblock at mount time is an exception: this case may occur
when the given partition is too small to contain the superblock.
Therefore, MFS should not crash or even report anything in this
case, but rather refuse to mount cleanly.

This fixes #121.

Change-Id: I11326b48922a0e0ebefecbb8eec7c428f985f2b3
2016-06-17 19:48:20 +00:00
David van Moolenbroek
6c7e614940 SEF: identity transfer only after controlled crash
Transparent (endpoint-preserving) restarts with identity transfer
are meant to exercise the crash recovery system only.  After *real*
crashes, such restarts are useless at best and dangerous at worst,
because no state integrity can be guaranteed afterwards.  Thus,
except after a controlled crash, it is best not to perform such
restarts at all.  This patch changes SEF such that identity transfer
is successful only if the old process was the subject of a crash
induced through "service fi".  As a result, testrelpol.sh should
continue to be able to use identity transfers for testing purposes,
but any real crash will be handled more appropriately.

This fixes #126.

Change-Id: Idc17ac7b3dfee05098529cb889ac835a0cd03ef0
2016-06-17 18:19:25 +00:00
Jean-Baptiste Boric
135965dc20 Revert "libutil: add getmaxpartitions() implementation"
This reverts commit 22ad44d6a9fa80d47806bf1897394569b6c15b8a.

With the MIB service implemented, this hack is no longer necessary.

Change-Id: Ic969c2dcecd6fc9ce283d1dda6518796869875e3
2016-03-13 19:51:58 +00:00
David van Moolenbroek
7ecc6a9247 libc: enable all functionality in net/
Some functions in lib/libc/net were disabled on MINIX3 only, but with
a few added header files they build just fine, even though some of
them rely on system functionality that has not yet been implemented.
Since the functionality is unlikely to be used in practice (because
it typically requires the use of protocol families that themselves are
not yet supported, such as IPv6), already enabling it right now helps
in building packages that rely on the functionality being present at
compile time, while not posing any practical risk of breaking the same
packages at run time.

Change-Id: Idee8e3963c9e300bde9575429f0e77b0565acaef
2016-03-13 16:03:39 +00:00
David van Moolenbroek
84ed480ef7 libc: fix local from-source upgrades
Commit git-c38dbb9 inadvertently broke local MINIX3-on-MINIX3 builds,
since its libc changes relied on VFS being upgraded already as well.
As a result, after installing the new libc, networking ceased to work,
leading to curl(1) failing later on in the build process.  This patch
introduces transitional code that is necessary for the build process
to complete, after which it is obsolete again.

Change-Id: I93bf29c01d228e3d7efc7b01befeff682954f54d
2016-03-09 12:11:57 +00:00
David van Moolenbroek
c38dbb97aa Prepare for switch to native BSD socket API
Currently, the BSD socket API is implemented in libc, translating the
API calls to character driver operations underneath.  This approach
has several issues:

- it is inefficient, as most character driver operations are specific
  to the socket type, thus requiring that each operation start by
  bruteforcing the socket protocol family and type of the given file
  descriptor using several system calls;
- it requires that libc itself be changed every time system support
  for a new protocol is added;
- various parts of the libc implementations violate the asynchronous
  signal safety POSIX requirements.

In order to resolve all these issues at once, the plan is to turn the
BSD socket calls into system calls, thus making the BSD socket API the
"native" ABI, removing the complexity from libc and instead letting
VFS deal with the socket calls.

The overall change is going to break all networking functionality. In
order to smoothen the transition, this patch introduces the fifteen
new BSD socket system calls, and makes libc try these first before
falling back on the old behavior.  For now, the VFS implementations of
the new calls fail such that libc will always use the fallback cases.
Later on, when we introduce the actual implementation of the native
BSD socket calls, all statically linked programs will automatically
use the new ABI, thus limiting actual application breakage.

In other words: by itself, this patch does nothing, except add a bit
of transitional overhead that will disappear in the future.  The
largest part of the patch is concerned with adding full support for
the new BSD socket system calls to trace(1) - this early addition has
the advantage of making system call tracing output of several socket
calls much more readable already.

Both the system call interfaces and the trace(1) support have already
been tested using code that will be committed later on.

Change-Id: I3460812be50c78be662d857f9d3d6840f3ca917f
2016-02-23 14:34:05 +00:00
David van Moolenbroek
0df28c9fa4 libc: reorganize vector I/O wrappers
The reorganization allows other libc system call wrappers (namely,
sendmsg and recvmsg) to perform I/O vector coalescing as well.

Change-Id: I116b48a6db39439053280ee805e0dcbdaec667a3
2016-02-22 23:24:47 +00:00
David van Moolenbroek
c33d6ef392 VFS: start off cleanup of pipe2 IPC message
There is no reason to use a single message for nonoverlapping requests
and replies combined, and in fact splitting them out allows reuse of
messages and avoids various problems with field layouts.  Since the
upcoming socketpair(2) system call will be using the same reply as
pipe2(2), split up the single message used for the latter.  In order
to keep the used parts of messages at the front, start a transitional
phase to move the pipe(2) flags field to the front of its request.

Change-Id: If3f1c3d348ec7e27b7f5b7147ce1b9ef490dfab9
2016-02-22 23:23:02 +00:00
David van Moolenbroek
17580212b4 libc: check raw IP socket type before using it
Previously, the libc sendto(3) and recvfrom(3) implementations would
blindly assume that any unrecognized socket is a raw-IP socket.  This
is not only inconsistent but also messes with returned error codes.

Change-Id: Id0328f04ea8ca0968a4e8636bc441caa0c3579b7
2016-02-22 23:21:05 +00:00
Lionel Sambuc
86b583c518 Fix usage of parenthesis in Makefiles
While BSD make support both $() and ${} around variables, the NetBSD
source tree uses only ${} by convention.

Imported software is left as is, and sometimes $() is used when the
containing Makefile/Makefile fragment is used both by GNU make and BSD
make, as it can happen for the tools, and other parts as well which are
compiled using the host make tool.

Change-Id: Ic7d480812fde53e7e3e95275a30a3b720c95cc15
2016-02-07 19:17:44 +01:00
David van Moolenbroek
373b793920 Basic statistical profiling fixes
At least it works again now.  Sprofalyze should be made aware of the
kernel information page, though (i.e., /proc/ipcvecs).

Change-Id: Id4e5f6417ad152607c4e53b323b6f65ea4b10c6e
2016-01-28 13:09:35 +01:00
Lionel Sambuc
e1cdaee106 Fix ARM noassert builds -g/-O{s/0/1/2/3}
Also fixes ARM assert build -O3.

Change-Id: I52bda91308ecfa0e8b23c4140c38c49347cc10f7
2016-01-24 18:56:38 +01:00
Lionel Sambuc
125832e358 Improve libexec debug output
Change-Id: I6a82d3eeed263e813852549fac015b82832c5c29
2016-01-16 14:04:28 +01:00
David van Moolenbroek
10b7016b5a Fix soft faults in FSes resulting in partial I/O
In order to resolve page faults on file-mapped pages, VM may need to
communicate (through VFS) with a file system.  The file system must
therefore not be the one to cause, and thus end up being blocked on,
such page faults.  To resolve this potential deadlock, the safecopy
system was previously extended with the CPF_TRY flag, which causes the
kernel to return EFAULT to the caller of a safecopy function upon
getting a pagefault, bypassing VM and thus avoiding the loop.  VFS was
extended to repeat relevant file system calls that returned EFAULT,
after resolving the page fault, to keep these soft faults from being
exposed to applications.

However, general UNIX I/O semantics dictate that if an I/O transfer
partially succeeded before running into a failure, the partial result
is to be returned.  Proper file system implementations may therefore
end up returning partial success rather than the EFAULT code resulting
from a soft fault.  Since VFS does not get the EFAULT code in this
case, it does not know that a soft fault occurred, and thus does not
repeat the call either.  The end result is that an application may get
partial I/O results (e.g., a short read(2)) even on regular files.
Applications cannot reasonably be expected to deal with this.

Due to the fact that most of the current file system implementations
do not implement proper partial-failure semantics, this problem is not
yet widespread.  In fact, it has only occurred on direct block device
I/O so far.  However, the next generation of file system services will
be implementing proper I/O semantics, thus exacerbating the problem.

To remedy this situation, this patch changes the CPF_TRY semantics:
whenever the kernel experiences a soft fault during a safecopy call,
in addition to returning FAULT, the kernel also stores a mark in the
grant created with CPF_TRY.  Instead of testing on EFAULT, VFS checks
whether the grant was marked, as part of revoking the grant.  If the
grant was indeed marked by the kernel, VFS repeats the file system
operation, regardless of its initial return value.  Thus, the EFAULT
code now only serves to make the file system fail the call faster.

The approach is currently supported for both direct and magic grants,
but is used only with magic grants - arguably the only case where it
makes sense.  Indirect grants should not have CPF_TRY set; in a chain
of indirect grants, the original grant is marked, as it should be.
In order to avoid potential SMP issues, the mark stored in the grant
is its grant identifier, so as to discard outdated kernel writes.
Whether this is necessary or effective remains to be evaluated.

This patch also cleans up the grant structure a bit, removing reserved
space and thus making the structure slightly smaller.  The structure
is used internally between system services only, so there is no need
for binary compatibility.

Change-Id: I6bb3990dce67a80146d954546075ceda4d6567f8
2016-01-16 14:04:21 +01:00
David van Moolenbroek
efc775b4c8 libsys: use linked list for free grants
With this change, obtaining an existing free grant is no longer an
operation of O(n) complexity.  As a result, the now-deprecated
getgrant/setgrant part of the grants API also no longer has a
performance advantage.

Change-Id: Ic19308a76924c6242f9784244a6b3600e561e0fe
2016-01-16 14:04:20 +01:00
David van Moolenbroek
ca779acd72 Add sequence numbers to grant IDs
The memory grant identifier for safecopies now includes a sequence
number in its upper bits, to prevent accidental reuse of a grant ID
after revocation and subsequent reallocation.  This should increase
overall system robustness by a tiny amount, and possibly help catch
bugs in system services early on.  For now, the lower 20 bits of the
grant ID are used as grant table slot index (thus allowing for up to
a million grants per process), and the next 11 bits of the (signed
32-bit) grant ID are used to store the per-slot sequence number.  As
grant IDs are never exposed to userland, the split can be changed
later on without breaking the userland ABI.

Change-Id: Ic34be27ff2a45db0ea5db037a24eef9efcd9ca40
2016-01-16 14:04:19 +01:00
David van Moolenbroek
2f09e77b82 MIB: add support for System V IPC information node
The kernel.ipc.sysvipc_info node is the gateway from NetBSD ipcs(1)
and ipcrm(1) to the IPC server, and thus necessary for a clean
import of these two utilities.  The MIB service implementation uses
the preexisting (Linux-specific) information calls on the IPC server
to obtain the information.

Change-Id: I85d1e193162d6b689f114764254dd7f314d2cfa0
2016-01-16 14:04:12 +01:00
David van Moolenbroek
910831cb5c PM: generic process event publish/subscribe system
Now that there are services other than PM and VFS that implement
userland system calls directly, these services may need to know about
events related to user processes.  In particular, signal delivery may
have to interrupt blocking system calls, and certain cleanup tasks may
have to be performed after a user process exits.

This patch aims to implement a generic, lasting solution for this
problem, by allowing services to subscribe to "signal delivered"
and/or "process exit" events from PM.  PM publishes such events by
sending messages to its subscribed services, which must then reply an
acknowledgment message.

For now, only the two aforementioned events are implemented, and only
the IPC service makes use of the process event facility.

The new process event publish/subscribe system replaces the previous
VM notify-sig/watch-exit/query-exit system, which was unsound: 1) it
allowed subscription to events from individual processes, and suffered
from fundamental race conditions as a result; 2) it relied on "not too
many" processes making use of the IPC server functionality in order to
avoid loss of notifications.  In addition, it had the "ipc" process
name hardcoded, did not distinguish between signal delivery and exits,
and added a roundtrip to VM for all events from all processes.

Change-Id: I75ebad4bc54e646c6433f473294cb4003b2c3430
2016-01-16 14:04:10 +01:00