This commit adds a new TCP/IP service to MINIX 3. As its core, the
service uses the lwIP TCP/IP stack for maintenance reasons. The
service aims to be compatible with NetBSD userland, including its
low-level network management utilities. It also aims to support
modern features such as IPv6. In summary, the new LWIP service has
support for the following main features:
- TCP, UDP, RAW sockets with mostly standard BSD API semantics;
- IPv6 support: host mode (complete) and router mode (partial);
- most of the standard BSD API socket options (SO_);
- all of the standard BSD API message flags (MSG_);
- the most used protocol-specific socket and control options;
- a default loopback interface and the ability to create one more;
- configuration-free ethernet interfaces and driver tracking;
- queuing and multiple concurrent requests to each ethernet driver;
- standard ioctl(2)-based BSD interface management;
- radix tree backed, destination-based routing;
- routing sockets for standard BSD route reporting and management;
- multicast traffic and multicast group membership tracking;
- Berkeley Packet Filter (BPF) devices;
- standard and custom sysctl(7) nodes for many internals;
- a slab allocation based, hybrid static/dynamic memory pool model.
Many of its modules come with fairly elaborate comments that cover
many aspects of what is going on. The service is primarily a socket
driver built on top of the libsockdriver library, but for BPF devices
it is at the same time also a character driver.
Change-Id: Ib0c02736234b21143915e5fcc0fda8fe408f046f
This commit (temporarily) leaves MINIX 3 without a TCP/IP service.
Thanks go out to Philip Homburg for providing this TCP/IP stack in the
first place. It has served MINIX well for a long time.
Change-Id: I0e3eb6fe64204081e4e3c2b9d6e6bd642f121973
This new implementation of the UDS service is built on top of the
libsockevent library. It thereby inherits all the advantages that
libsockevent brings. However, the fundamental restructuring
required for that change also paved the way for resolution of a
number of other important open issues with the old UDS code. Most
importantly, the rewrite brings the behavior of the service much
closer to POSIX compliance and NetBSD compatibility. These are the
most important changes:
- due to the use of libsockevent, UDS now supports multiple suspending
calls per socket and a large number of standard socket flags and
options;
- socket address matching is now based on <device,inode> lookups
instead of canonized path names, and socket addresses are no longer
altered either due to canonization or at connect time;
- the socket state machine is now well defined, most importantly
resolving the erroneous reset-on-EOF semantics of the old UDS, but
also allowing socket reuse;
- sockets are now connected before being accepted instead of being
held in connecting state, unless the LOCAL_CONNWAIT option is set
on either the connecting or the listening socket;
- connect(2) on datagram sockets is now supported (needed by syslog),
and proper datagram socket disconnect notification is provided;
- the receive queue now supports segmentation, associating ancillary
data (in-flight file descriptors and credentials) with each segment
instead of being kept fully separately; this is a POSIX requirement
(and needed by tmux);
- as part of the segmentation support, the receive queue can now hold
as many packets as can fit, instead of one;
- in addition to the flags supported by libsockevent, the MSG_PEEK,
MSG_WAITALL, MSG_CMSG_CLOEXEC, MSG_TRUNC, and MSG_CTRUNC send and
receive flags are now supported;
- the SO_PASSCRED and SO_PEERCRED socket options are replaced by
LOCAL_CREDS and LOCAL_PEEREID respectively, now following NetBSD
semantics and allowing use of NetBSD libc's getpeereid(3);
- memory usage is reduced by about 250 KB due to centralized in-flight
file descriptor tracking, with a limit of OPEN_MAX total rather than
of OPEN_MAX per socket;
- memory usage is reduced by another ~50 KB due to removal of state
redundancy, despite the fact that socket path names may now be up to
253 bytes rather than the previous 104 bytes;
- compared to the old UDS, there is now very little direct indexing on
the static array of sockets, thus allowing dynamic allocation of
sockets more easily in the future;
- the UDS service now has RMIB support for the net.local sysctl tree,
implementing preliminary support for NetBSD netstat(1).
Change-Id: I4a9b6fe4aaeef0edf2547eee894e6c14403fcb32
The service-only getepinfo(2) PM call returns information about a
given endpoint. This patch extends that call so that it returns
enough information to allow correctly filling a sockcred structure.
A new getsockcred(3) function is added to libsys to fill an actual
sockcred structure with the obtained information. However, for the
caller's convenience, the groups list is kept separate.
Change-Id: I9f1a6d1a221c77eabaa3498ff4ec9a5fb922e4fd
This patch prepares for moving of the creation of socket files on the
file system from the libc bind(2) stub into the UDS service. This
change is necessary for the socket type agnostic libc implementation.
The change is not yet activated - the code that is not yet used is
enclosed in "#if NOT_YET" blocks. The activation needs to be atomic
with UDS's switch to libsockdriver; otherwise, user applications may
break.
As part of the change, various UDS bind(2) semantics are changed to
match the POSIX standard and other operating systems. In
implementation terms, the service-only VFS API checkperms(2) is
renamed to socketpath(2), and extended with a new subcall which
creates a new socket file. An extension to test56 checks the new
bind(2) semantics of UDS, although most new tests are still disabled
until activation as well.
Finally, as further preparation for a more structural redesign of the
UDS service, also return the <device,inode> number pair for the
created or checked file name, and make returning the canonized path
name optional.
Change-Id: I892d04b3301d4b911bdc571632ddde65fb747a8a
The flag is supported only when copying out file descriptors (i.e.
COPYFD_TO). It will be used by UDS to support MSG_CMSG_CLOEXEC.
Change-Id: I46bfd04b5f28e22ec48938e43e42f78d3931220d
This patch stops a socket driver from using copyfd(2) to copy in a
file descriptor that is a reference to a socket owned by that socket
driver, returning EDEADLK instead. In effect, this will stop deadlock
and resource exhaustion issues with UDS once it has been converted to
a socket driver. See the comment in the patch for details.
Change-Id: I5728a405eabda207725618231a6ff7be2d517146
This change effectively adds the VFS side of support for the SO_LINGER
socket option, by allowing file descriptor close operations to be
suspended (and later resumed) by socket drivers. Currently, support
is limited to the close(2) system call--in all other cases where file
descriptors are closed (dup2, close-on-exec, process exit..), the
close operation still completes instantly. As a general policy, the
close(2) return value will always indicate that the file descriptor
has been closed: either 0, or -1 with errno set to EINPROGRESS. The
latter error may be thrown only when a suspended close is interrupted
by a signal.
As necessary for UDS, this change also introduces a closenb(2) system
call extension, allowing the caller to bypass blocking SO_LINGER close
behavior. This extension allows UDS to avoid blocking on closing the
last reference to an in-flight file descriptor, in an atomic fashion.
The extension is currently part of libsys, but there is no reason why
userland would not be allowed to make this call, so it is deliberately
not protected from use by userland.
Change-Id: Iec77d6665232110346180017fc1300b1614910b7
If a select(2) call was issued on a file descriptor for which the file
pointer was closed due to invalidation (FILP_CLOSED), typically as the
result of a character/socket driver dying, the call would previously
return with an error: EINTR upon call entry or EIO on invalidation at
at a later time. Especially the former could severely confuse
applications, which would assume the call was interrupted by a signal,
restart the select call and immediately get EINTR again, ad infinitum.
This patch changes the select(2) semantics such that for closed filps,
the file descriptor is returned as readable and/or writable (depending
on the requested operations), as such letting the entire select call
finish successfully. Applications will then typically attempt to read
from and/or write to the file descriptor, resulting in an I/O error
that they should generally be better equipped to handle.
This patch also fixes a potential problem with returning early from a
select(2) call if a bad file descriptor is given: previously, in such
cases not all actions taken so far would be undone; now they are.
Change-Id: Ia6581f8789473a8a6c200852fccf552691a17025
This patch adds the implementation of the BSD socket system calls
which have been introduced in an earlier patch. At the same time, it
adds support for communication with socket drivers, using a new
"socket device" (SDEV_) protocol. These two parts, implemented in
socket.c and sdev.c respectively, form the upper and lower halves of
the new BSD socket support in VFS. New mapping functionality for
socket domains and drivers is added as well, implemented in smap.c.
The rest of the changes mainly facilitate the separation of character
and socket driver calls, and do not make any fundamental alterations.
For example, while this patch changes VFS's select.c rather heavily,
the new select logic for socket drivers is the exact same as for
character drivers; the changes mainly separate the driver type
specific parts from the generic select logic further than before.
Change-Id: I2f13084dd3c8d3a68bfc69da0621120c8291f707
This patch introduces the first piece of support for the concept of
"socket drivers": services that implement one or more socket protocol
families. The latter are also known as "domains", as per the first
parameter of the socket(2) API. More specifically, this patch adds
the basic infrastructure for specifying that a particular service is
the socket driver for a set of domains.
Unlike major number mappings for block and character drivers, socket
domain mappings are static. For that reason, they are specified in
system.conf files, using the "domain" keyword. Such a keyword is to
be followed by one or more protocol families, without their "PF_"
prefix. For example, a service with the line "domain INET INET6;"
will be mapped as the socket driver responsible for the AF_INET and
AF_INET6 protocol families.
This patch implements only the infrastructure for creating such
mappings; the actual mapping will be implemented in VFS in a later
patch. The infrastructure is implemented in service(8), RS, and VFS.
For now there is a hardcoded limit of eight domains per socket driver.
This may sound like a lot, but the upcoming new LWIP service will
already use four of those. Also, it is allowed for a service to be
both a block/character driver and a socket driver at the same time,
which is a requirement for the new LWIP service.
Change-Id: I93352d488fc6c481e7079248082895d388c39f2d
After processing certain asynchronous requests from VFS, VM would send
an asynchronous reply without supplying the AMF_NOREPLY flag. As a
result, this asynchronous reply could be taken as the result of an
ipc_sendrec() call, causing the entire VM/VFS communication to become
desynchronized. The end result was a deadlock-induced panic during a
later request.
This bug was exposed because of the higher-than-usual concurrency
level in the NetBSD rc scripts. The fix consists of properly setting
the AMF_NOREPLY flag for asynchronous replies.
Change-Id: Iafafe2fdd67f212ecbf27a53862cefba2e4cf7e8
IMPORTANT: this change has a docs/UPDATING entry!
This change is a long overdue switch-over from the old MINIX set of
user and group accounts to the NetBSD set. This switch-over is
increasingly important now that we are importing more and more
utilities from NetBSD, several of which expect various user accounts
to exist. By switching over in one go, we save ourselves various
headaches in the long run, even if the switch-over itself is a bit
painful for existing MINIX users.
The newly imported master.passwd and group files have three exceptions
compared to their NetBSD originals:
1. There is a custom "service" account for MINIX 3 services. This
account is used to limit run-time privileges of various system
services, and is not used for any files on disk. Its user ID may
be changed later, but should always correspond to whatever the
SERVICE_UID definition is set to.
2. The user "bin" has its shell set to /bin/sh, instead of NetBSD's
/sbin/nologin. The reason for this is that the test set in
/usr/tests/minix-posix will not be able to run otherwise.
3. The group "operator" has been set to group ID 0, to match its old
value. This tweak is purely for transitioning purposes: as of
writing, pkgsrc packages are still using root:operator as owner and
group for most installed files. Sometime later, we can change back
"operator" to group ID 5 without breaking anything, because it does
not appear that this group name is used for anything important.
Change-Id: I689bcfff4cf7ba85c27d1ae579057fa3f8019c68
This was a MINIX3-specific header file placed outside of the minix/
header subdirectory, with its definitions duplicated in the more
standard minix/sysutil.h header.
Also make env_prefix(3) take constant pointers.
Change-Id: I243c38eb38e24eb98f0c0dddf7f340e7fec255f4
As of change git-87c599d, when processing CLOCK notifications, PM no
longer set the current process pointer 'mp'. That pointer is however
used when delivering signals through check_sig(), to see whether the
current process may deliver a signal to the target process. As a
result, delivering SIGALARM signals used a previous pointer in these
checks, causing alarm signals not to be delivered in some cases.
This patch ensures that alarm signals are again delivered with PM as
current process.
Change-Id: I94ccbe8b71289df0e1d6d67928e55297bbc28360
With this patch, it is now possible to generate coverage information
for MINIX3 system services with LLVM. In particular, the system can
be built with MKCOVERAGE=yes, either with a native "make build" or
with crosscompilation. Either way, MKCOVERAGE=yes will build the
MINIX3 system services with coverage profiling support, generating a
.gcno file for each source module. After a reboot it is possible to
obtain runtime coverage data (.gcda files) for individual system
services using gcov-pull(8). The combination of the .gcno and .gcda
files can then be inspected with llvm-cov(1).
For reasons documented in minix.gcov.mk, only system service program
modules are supported for now; system service libraries (libsys etc.)
are not included. Userland programs are not affected by MKCOVERAGE.
The heart of this patch is the libsys code that writes data generated
by the LLVM coverage hooks into a serialized format using the routines
we already had for GCC GCOV. Unfortunately, the new llvm_gcov.c code
is LLVM ABI dependent, and may therefore have to be updated later when
we upgrade LLVM. The current implementation should support all LLVM
versions 3.x with x >= 4.
The rest of this patch is mostly a light cleanup of our existing GCOV
infrastructure, with as most visible change that gcov-pull(8) now
takes a service label string rather than a PID number.
Change-Id: I6de055359d3d2b3f53e426f3fffb17af7877261f
All functions prefixed with bdev_ are moved into bdev.c, and those
prefixed with cdev_ are now in cdev.c. The code in both files are
converted to KNF. The little (IOCTL-related) code left in device.c
is also cleaned up but should probably be moved into other existing
source files. This is left to a future patch. In general, VFS is
long overdue for a source code rebalancing, and the patch here is
only a step in the right direction.
Change-Id: I2fb25734b5778b44f2ff6d2ce331a8e2146e20b0
Previously, VFS would use various subsets of a number of fproc
structure fields to store state when the process is blocked
(suspended) for various reasons. As a result, there was a fair
amount of abuse of fields, hidden state, and confusion as to
which fields were used with which suspension states.
Instead, the suspension state is now split into per-state
structures, which are then stored in a union. Each of the union's
structures should be accessed only right before, during, and right
after the fp_blocked_on field is set to the corresponding blocking
type. As a result, it is now very clear which fields are in use
at which times, and we even save a bit of memory as a side effect.
Change-Id: I5c24e353b6cb0c32eb41c70f89c5cfb23f6c93df
Any attempt to use open(2) to open a socket file now fails with
EOPNOTSUPP, as is common and in the process of being standardized.
The behavior and error code is now tested in test56.
Any attempt to open a file of which the type is not known to VFS
(e.g., as a result of bogus file system contents) now fails with EIO.
For now, this is a safety feature, to prevent VFS tripping over such
types in unchecked cases. In the future, a proper VFS code audit
should determine whether we can lift this restriction again, although
it does not seem particularly useful to be able to open files of
unknown types anyway. Another error code may be assigned to this case
later, too.
Change-Id: Ib4cb4341eec954f0448fe469ecf28bd78edebde2
By now it has become clear that the VFS select code has an unusually
high concentration of bugs, and there is no indication that any form
of convergence to a bug-free state is in sight. Thus, for now, it
may be helpful to be able to dump the contents of the select tables
in order to track down any bugs in the future. Hopefully that will
allow the next bugs to be resolved slightly after than before.
The debug dump can be triggered with "svrctl vfs get print_select".
Change-Id: Ia826746dce0f065d7f3b46aa9047945067b8263d
A select query could deadlock if..
- it was querying a character or socket device that, at the start of
the select query, was not known to be ready for the requested
operations;
- this device could not be checked immediately, due to another ongoing
query to the same character or socket driver;
- the select query had a timer that triggered before the device could
be checked, thereby changing the select query to non-blocking.
In this situation, a missing flag check would cause the select code to
conclude erroneously that the operations which it flagged for later,
were satisfied. At the same time, the same flag remained set, so that
the select query would continue to wait for that device. This
resulted in a deadlock. The same bug could most likely be triggered
through other scenarios that were even less likely to occur.
This patch fixes the race condition and puts in a hopefully slightly
more informative comment for the affected block of code.
In practice, the bug could be triggered fairly reliably by generating
lots of output in tmux.
Change-Id: I1c909255dcf552e6c7cef08b0cf5cbc41294b99c
Now that clock_t is an unsigned value, we can also allow the system
uptime to wrap. Essentially, instead of using (a <= b) to see if time
a occurs no later than time b, we use (b - a <= CLOCK_MAX / 2). The
latter value does not exist, so instead we add TMRDIFF_MAX for that
purpose.
We must therefore also avoid using values like 0 and LONG_MAX as
special values for absolute times. This patch extends the libtimers
interface so that it no longer uses 0 to indicate "no timeout".
Similarly, TMR_NEVER is now used as special value only when
otherwise a relative time difference would be used. A minix_timer
structure is now considered in use when it has a watchdog function set,
rather than when the absolute expiry time is not TMR_NEVER. A few new
macros in <minix/timers.h> help with timer comparison and obtaining
properties from a minix_timer structure.
This patch also eliminates the union of timer arguments, instead using
the only union element that is only used (the integer). This prevents
potential problems with e.g. live update. The watchdog function
prototype is changed to pass in the argument value rather than a
pointer to the timer structure, since obtaining the argument value was
the only current use of the timer structure anyway. The result is a
somewhat friendlier timers API.
The VFS select code required a few more invasive changes to restrict
the timer value to the new maximum, effectively matching the timer
code in PM. As a side effect, select(2) has been changed to reject
invalid timeout values. That required a change to the test set, which
relied on the previous, erroneous behavior.
Finally, while we're rewriting significant chunks of the timer code
anyway, also covert it to KNF and add a few more explanatory comments.
Change-Id: Id43165c3fbb140b32b90be2cca7f68dd646ea72e
Aditionally this removes all trailing whitespaces in pm server code
using: sed -i 's/[[:space:]]*$//' *.c
Change-Id: Ie44162fd56cd7042f4f0cc7bd7314b17ea128761
With this patch, the IPC service is changed to use the new RMIB
facility to register and handle the "kern.ipc" sysctl subtree itself.
The subtree was previously handled by the MIB service directly. This
change improves locality of handling: especially the
kern.ipc.sysvipc_info node has some peculiarities specific to the IPC
service and is therefore better handled there. Also, since the IPC
service is essentially optional to the system, this rearrangement
yields a cleaner situation when the IPC service is not running: in
that case, the MIB service will expose a few basic kern.ipc nodes
indicating that no SysV IPC facilities are present. Those nodes will
be overridden through RMIB when the IPC service is running.
It should be easier to add the remaining (from NetBSD) kern.ipc nodes
as well now.
Test88 is extended with a new subtest that verifies that sysctl-based
information retrieval for semaphore sets works as expected.
Change-Id: I6b7730e85305b64cfd8418c0cc56bde64b22c584
Most of the nodes in the general sysctl tree will be managed directly
by the MIB service, which obtains the necessary information as needed.
However, in certain cases, it makes more sense to let another service
manage a part of the sysctl tree itself, in order to avoid replicating
part of that other service in the MIB service. This patch adds the
basic support for such delegation: remote services may now register
their own subtrees within the full sysctl tree with the MIB service,
which will then forward any sysctl(2) requests on such subtrees to the
remote services.
The system works much like mounting a file system, but in addition to
support for shadowing an existing node, the MIB service also supports
creating temporary mount point nodes. Each have their own use cases.
A remote "kern.ipc" would use the former, because even when such a
subtree were not mounted, userland would still expect some of its
children to exist and return default values. A remote "net.inet"
would use the latter, as there is no reason to precreate nodes for all
possible supported networking protocols in the MIB "net" subtree.
A standard remote MIB (RMIB) implementation is provided for services
that wish to make use of this functionality. It is essentially a
simplified and somewhat more lightweight version of the MIB service's
internals, and works more or less the same from a programmer's point
of view. The most important difference is the "rmib" prefix instead
of the "mib" prefix. Documentation will hopefully follow later.
Overall, the RMIB functionality should not be used lightly, for
several reasons. First, despite being more lightweight than the MIB
service, the RMIB module still adds substantially to the code
footprint of the containing service. Second, the RMIB protocol not
only adds extra IPC for sysctl(2), but has also not been optimized for
performance in other ways. Third, and most importantly, the RMIB
implementation also several limitations. The main limitation is that
remote MIB subtrees must be fully static. Not only may the user not
create or destroy nodes, the service itself may not either, as this
would clash with the simplified remote node versioning system and
the cached subtree root node child counts. Other limitations exist,
such as the fact that the root of a remote subtree may only be a
node-type node, and a stricter limit on the highest node identifier
of any child in this subtree root (currently 4095).
The current implementation was born out of necessity, and therefore
it leaves several improvements to future work. Most importantly,
support for exit and crash notification is missing, primarily in the
MIB service. This means that remote subtrees may not be cleaned up
immediately, but instead only when the MIB service attempts to talk
to the dead remote service. In addition, if the MIB service itself
crashes, re-registration of remote subtrees is currently left up to
the individual RMIB users. Finally, the MIB service uses synchronous
(sendrec-based) calls to the remote services, which while convenient
may cause cascading service hangs. The underlying protocol is ready
for conversion to an asynchronous implementation already, though.
A new test set, testrmib.sh, tests the basic RMIB functionality. To
this end it uses a test service, rmibtest, and also reuses part of
the existing test87 MIB service test.
Change-Id: I3378fe04f2e090ab231705bde7e13d6289a9183e
The new asserts from git-29e004d exposed an issue in how VFS handles
aborting file system (FS) requests that are queued for a FS (as
opposed to sent to it) when that FS crashes. In that scenario, the
queued worker has its w_task set to NONE, because there is no ongoing
communication. However, worker_stop() is called on it regardless,
which used to abort the request only if w_task was not set to NONE,
leading to an improperly aborted request, a warning, and a VFS crash a
bit later. This patch changes worker_stop() so that w_task need not
be set to a valid endpoint for FS requests to be properly aborted.
Change-Id: Ib73db285e689ae4742b15cba26137bf340bc303b
This patch aims to synchronize the basic process user and group ID
management, as well as the set[ug]id(2) and sete[ug]id(2) behavior,
with NetBSD. As it turns out, the main issue was missing support for
saved user and group IDs. This support is now added.
Since NetBSD's userland, which we are importing, may rely on NetBSD
specifics when it comes to security, we choose not to deviate from
NetBSD's behavior in any way here. A new test, test89, verifies the
correct behavior - it has been confirmed to pass on NetBSD as is.
Change-Id: I023935546d97ed01ffd8090f7793d336cceb0f4a
Currently, the BSD socket API is implemented in libc, translating the
API calls to character driver operations underneath. This approach
has several issues:
- it is inefficient, as most character driver operations are specific
to the socket type, thus requiring that each operation start by
bruteforcing the socket protocol family and type of the given file
descriptor using several system calls;
- it requires that libc itself be changed every time system support
for a new protocol is added;
- various parts of the libc implementations violate the asynchronous
signal safety POSIX requirements.
In order to resolve all these issues at once, the plan is to turn the
BSD socket calls into system calls, thus making the BSD socket API the
"native" ABI, removing the complexity from libc and instead letting
VFS deal with the socket calls.
The overall change is going to break all networking functionality. In
order to smoothen the transition, this patch introduces the fifteen
new BSD socket system calls, and makes libc try these first before
falling back on the old behavior. For now, the VFS implementations of
the new calls fail such that libc will always use the fallback cases.
Later on, when we introduce the actual implementation of the native
BSD socket calls, all statically linked programs will automatically
use the new ABI, thus limiting actual application breakage.
In other words: by itself, this patch does nothing, except add a bit
of transitional overhead that will disappear in the future. The
largest part of the patch is concerned with adding full support for
the new BSD socket system calls to trace(1) - this early addition has
the advantage of making system call tracing output of several socket
calls much more readable already.
Both the system call interfaces and the trace(1) support have already
been tested using code that will be committed later on.
Change-Id: I3460812be50c78be662d857f9d3d6840f3ca917f
There is no reason to use a single message for nonoverlapping requests
and replies combined, and in fact splitting them out allows reuse of
messages and avoids various problems with field layouts. Since the
upcoming socketpair(2) system call will be using the same reply as
pipe2(2), split up the single message used for the latter. In order
to keep the used parts of messages at the front, start a transitional
phase to move the pipe(2) flags field to the front of its request.
Change-Id: If3f1c3d348ec7e27b7f5b7147ce1b9ef490dfab9
In order to resolve page faults on file-mapped pages, VM may need to
communicate (through VFS) with a file system. The file system must
therefore not be the one to cause, and thus end up being blocked on,
such page faults. To resolve this potential deadlock, the safecopy
system was previously extended with the CPF_TRY flag, which causes the
kernel to return EFAULT to the caller of a safecopy function upon
getting a pagefault, bypassing VM and thus avoiding the loop. VFS was
extended to repeat relevant file system calls that returned EFAULT,
after resolving the page fault, to keep these soft faults from being
exposed to applications.
However, general UNIX I/O semantics dictate that if an I/O transfer
partially succeeded before running into a failure, the partial result
is to be returned. Proper file system implementations may therefore
end up returning partial success rather than the EFAULT code resulting
from a soft fault. Since VFS does not get the EFAULT code in this
case, it does not know that a soft fault occurred, and thus does not
repeat the call either. The end result is that an application may get
partial I/O results (e.g., a short read(2)) even on regular files.
Applications cannot reasonably be expected to deal with this.
Due to the fact that most of the current file system implementations
do not implement proper partial-failure semantics, this problem is not
yet widespread. In fact, it has only occurred on direct block device
I/O so far. However, the next generation of file system services will
be implementing proper I/O semantics, thus exacerbating the problem.
To remedy this situation, this patch changes the CPF_TRY semantics:
whenever the kernel experiences a soft fault during a safecopy call,
in addition to returning FAULT, the kernel also stores a mark in the
grant created with CPF_TRY. Instead of testing on EFAULT, VFS checks
whether the grant was marked, as part of revoking the grant. If the
grant was indeed marked by the kernel, VFS repeats the file system
operation, regardless of its initial return value. Thus, the EFAULT
code now only serves to make the file system fail the call faster.
The approach is currently supported for both direct and magic grants,
but is used only with magic grants - arguably the only case where it
makes sense. Indirect grants should not have CPF_TRY set; in a chain
of indirect grants, the original grant is marked, as it should be.
In order to avoid potential SMP issues, the mark stored in the grant
is its grant identifier, so as to discard outdated kernel writes.
Whether this is necessary or effective remains to be evaluated.
This patch also cleans up the grant structure a bit, removing reserved
space and thus making the structure slightly smaller. The structure
is used internally between system services only, so there is no need
for binary compatibility.
Change-Id: I6bb3990dce67a80146d954546075ceda4d6567f8
The kernel.ipc.sysvipc_info node is the gateway from NetBSD ipcs(1)
and ipcrm(1) to the IPC server, and thus necessary for a clean
import of these two utilities. The MIB service implementation uses
the preexisting (Linux-specific) information calls on the IPC server
to obtain the information.
Change-Id: I85d1e193162d6b689f114764254dd7f314d2cfa0
As mentioned in previous patches, services may not subscribe to
process events from specific processes only, since this results in
race conditions. However, the IPC server can safely turn on and off
its entire subscription based on whether any System V IPC semaphores
(and, in the future, message queues) are allocated at all. Since
the System V IPC facilities are not so commonly used, this removes
the extra round trip from PM to the IPC server and back for caught
signals and process exits in the common case.
Change-Id: I937259034872be32f4e26ab99270f4d475ff6134
- rewrite the semop(2) implementation so that it now conforms to the
specification, including atomicity, support for blocking more than
once, range checks, but also basic fairness support;
- fix permissions checking;
- fix missing time adjustments;
- fix off-by-one errors and other bugs;
- do not allocate dynamic memory for GETALL/SETALL;
- add test88, which properly tests the semaphore functionality.
Change-Id: I85f0d3408c0d6bba41cfb4c91a34c8b46b2a5959
Now that there are services other than PM and VFS that implement
userland system calls directly, these services may need to know about
events related to user processes. In particular, signal delivery may
have to interrupt blocking system calls, and certain cleanup tasks may
have to be performed after a user process exits.
This patch aims to implement a generic, lasting solution for this
problem, by allowing services to subscribe to "signal delivered"
and/or "process exit" events from PM. PM publishes such events by
sending messages to its subscribed services, which must then reply an
acknowledgment message.
For now, only the two aforementioned events are implemented, and only
the IPC service makes use of the process event facility.
The new process event publish/subscribe system replaces the previous
VM notify-sig/watch-exit/query-exit system, which was unsound: 1) it
allowed subscription to events from individual processes, and suffered
from fundamental race conditions as a result; 2) it relied on "not too
many" processes making use of the IPC server functionality in order to
avoid loss of notifications. In addition, it had the "ipc" process
name hardcoded, did not distinguish between signal delivery and exits,
and added a roundtrip to VM for all events from all processes.
Change-Id: I75ebad4bc54e646c6433f473294cb4003b2c3430
Closer to KNF, better coding practices, more similar to other
services, no more global variables, a few more comments, that
kind of stuff. No major functional changes.
Change-Id: I6e8f53bfafd6f41e92031fba76c40a31d2107a8e
- switch to the NetBSD identifier system; it is not only better, but
also required for porting NetBSD ipcs(1) and ipcrm(1); however, it
requires that slots not be moved, and that results in some changes;
- synchronize some other things with NetBSD: where keys are kept, as
well as various non-permission mode flags;
- fix semctl(2) vararg retrieval and message field type;
- use SUSPEND instead of weird reply exceptions in the call table;
- fix several memory leaks and at least one missing permission check;
- improve the atomicity of semop(2) by a small amount, even though
its atomicity is still broken at a fundamental level;
- use the new cheaper way to retrieve the current time;
- resolve all level-5 LLVM warnings.
Change-Id: I0c47aacde478b23bb77d628384aeab855a22fdbf
Specifically, add support for the IPC_INFO, SEM_INFO, and SEM_STAT
semctl(2) operations, similar to how information about shared memory
is already exposed as well. The MINIX3 ipcs(1) utility already had
support for these operations, and can now actually use them, too.
Change-Id: Ice5a02e729bf6df6aa8fab76e854808adc04dae3
- About 80% of PM's process table consisted of per-signal sigaction
structures. This is information not used by the MIB service, and
can safely be stored outside the main process table.
- The MIB service does not need most of the VFS process table, so VFS
now generates a "light" version of its table upon request, with just
the fields used by the MIB service.
The result is a size reduction of the MIB service of about 700KB.
Change-Id: I79fe7239361fbfb45286af8e86a10aed4c2d2be7
Instead of pulling in process tables itself, ProcFS now queries the
MIB service for process information. This reduces ProcFS's memory
usage by about 1MB. The change does have two negative consequences.
First, getting all the original /proc/<pid>/psinfo fields filled in
would take a lot of extra effort. Since the only program that uses
those files at all is mtop(1), we reformat psinfo to expose only the
information used by mtop(1). This means that with this patch, older
copies of MINIX3 ps and top will cease to work.
Second, since both MIB and ProcFS update their own view of the
process list only once per clock tick, ProcFS' view may now be
outdated by up to two clock ticks. This is unlikely to pose a
problem in practice.
Change-Id: Iaa6b60450c8fb52d092962394d33d08bd638bc01
Now that uname(3) uses sysctl(2), we no longer need sysuname(2).
Backward compatibility is retained for old statically linked
binaries for a short while.
Also remove the now-obsolete MINIX3-specific "arch" field from the
utsname structure. While this is an ABI break at the libc level,
it should pose no problems in practice, because:
- statically linked programs (i.e., all of the base system) are not
affected, as they will use headers synchronized with libc;
- the structure is getting smaller, thus, older dynamically linked
programs (typically in pkgsrc) using the new libc will end up with
garbage in the "arch" field, but it is unlikely they will use this
field anyway, since it was specific to MINIX3;
- new dynamically linked programs using an old libc could end up with
memory corruption, but this is not a scenario that is expected to
occur in the first place - certainly not with programs from pkgsrc.
Change-Id: I29c76576f509feacc8f996f0bd353ca8961d4917
PM uses its own process table entry as source for kernel signals,
and temporarily changes its own process group to make the signals
arrive at the right processes. However, the value is never reset,
with as result that the temporary value shows up in ps(1) output.
Change-Id: Ib7f635b2cf1958055123736dfd58c26530632785
