With this patch, it is now possible to generate coverage information
for MINIX3 system services with LLVM. In particular, the system can
be built with MKCOVERAGE=yes, either with a native "make build" or
with crosscompilation. Either way, MKCOVERAGE=yes will build the
MINIX3 system services with coverage profiling support, generating a
.gcno file for each source module. After a reboot it is possible to
obtain runtime coverage data (.gcda files) for individual system
services using gcov-pull(8). The combination of the .gcno and .gcda
files can then be inspected with llvm-cov(1).
For reasons documented in minix.gcov.mk, only system service program
modules are supported for now; system service libraries (libsys etc.)
are not included. Userland programs are not affected by MKCOVERAGE.
The heart of this patch is the libsys code that writes data generated
by the LLVM coverage hooks into a serialized format using the routines
we already had for GCC GCOV. Unfortunately, the new llvm_gcov.c code
is LLVM ABI dependent, and may therefore have to be updated later when
we upgrade LLVM. The current implementation should support all LLVM
versions 3.x with x >= 4.
The rest of this patch is mostly a light cleanup of our existing GCOV
infrastructure, with as most visible change that gcov-pull(8) now
takes a service label string rather than a PID number.
Change-Id: I6de055359d3d2b3f53e426f3fffb17af7877261f
The way these options work is by creating files that contain debugging
symbols and stashing them in a dedicated set. The minix-debug set has
been created for this purpose, but it will probably have to be refined
since it has been tested only with the default options with an i386
cross-build.
LSC: Amended to support many combination of MKDEBUG, MKDEBUGLIB, with
and without X11, for both intel and arm.
Change-Id: I2901952e8229938f9ac79c8656484acf704ccd9b
Now that clock_t is an unsigned value, we can also allow the system
uptime to wrap. Essentially, instead of using (a <= b) to see if time
a occurs no later than time b, we use (b - a <= CLOCK_MAX / 2). The
latter value does not exist, so instead we add TMRDIFF_MAX for that
purpose.
We must therefore also avoid using values like 0 and LONG_MAX as
special values for absolute times. This patch extends the libtimers
interface so that it no longer uses 0 to indicate "no timeout".
Similarly, TMR_NEVER is now used as special value only when
otherwise a relative time difference would be used. A minix_timer
structure is now considered in use when it has a watchdog function set,
rather than when the absolute expiry time is not TMR_NEVER. A few new
macros in <minix/timers.h> help with timer comparison and obtaining
properties from a minix_timer structure.
This patch also eliminates the union of timer arguments, instead using
the only union element that is only used (the integer). This prevents
potential problems with e.g. live update. The watchdog function
prototype is changed to pass in the argument value rather than a
pointer to the timer structure, since obtaining the argument value was
the only current use of the timer structure anyway. The result is a
somewhat friendlier timers API.
The VFS select code required a few more invasive changes to restrict
the timer value to the new maximum, effectively matching the timer
code in PM. As a side effect, select(2) has been changed to reject
invalid timeout values. That required a change to the test set, which
relied on the previous, erroneous behavior.
Finally, while we're rewriting significant chunks of the timer code
anyway, also covert it to KNF and add a few more explanatory comments.
Change-Id: Id43165c3fbb140b32b90be2cca7f68dd646ea72e
Most of the nodes in the general sysctl tree will be managed directly
by the MIB service, which obtains the necessary information as needed.
However, in certain cases, it makes more sense to let another service
manage a part of the sysctl tree itself, in order to avoid replicating
part of that other service in the MIB service. This patch adds the
basic support for such delegation: remote services may now register
their own subtrees within the full sysctl tree with the MIB service,
which will then forward any sysctl(2) requests on such subtrees to the
remote services.
The system works much like mounting a file system, but in addition to
support for shadowing an existing node, the MIB service also supports
creating temporary mount point nodes. Each have their own use cases.
A remote "kern.ipc" would use the former, because even when such a
subtree were not mounted, userland would still expect some of its
children to exist and return default values. A remote "net.inet"
would use the latter, as there is no reason to precreate nodes for all
possible supported networking protocols in the MIB "net" subtree.
A standard remote MIB (RMIB) implementation is provided for services
that wish to make use of this functionality. It is essentially a
simplified and somewhat more lightweight version of the MIB service's
internals, and works more or less the same from a programmer's point
of view. The most important difference is the "rmib" prefix instead
of the "mib" prefix. Documentation will hopefully follow later.
Overall, the RMIB functionality should not be used lightly, for
several reasons. First, despite being more lightweight than the MIB
service, the RMIB module still adds substantially to the code
footprint of the containing service. Second, the RMIB protocol not
only adds extra IPC for sysctl(2), but has also not been optimized for
performance in other ways. Third, and most importantly, the RMIB
implementation also several limitations. The main limitation is that
remote MIB subtrees must be fully static. Not only may the user not
create or destroy nodes, the service itself may not either, as this
would clash with the simplified remote node versioning system and
the cached subtree root node child counts. Other limitations exist,
such as the fact that the root of a remote subtree may only be a
node-type node, and a stricter limit on the highest node identifier
of any child in this subtree root (currently 4095).
The current implementation was born out of necessity, and therefore
it leaves several improvements to future work. Most importantly,
support for exit and crash notification is missing, primarily in the
MIB service. This means that remote subtrees may not be cleaned up
immediately, but instead only when the MIB service attempts to talk
to the dead remote service. In addition, if the MIB service itself
crashes, re-registration of remote subtrees is currently left up to
the individual RMIB users. Finally, the MIB service uses synchronous
(sendrec-based) calls to the remote services, which while convenient
may cause cascading service hangs. The underlying protocol is ready
for conversion to an asynchronous implementation already, though.
A new test set, testrmib.sh, tests the basic RMIB functionality. To
this end it uses a test service, rmibtest, and also reuses part of
the existing test87 MIB service test.
Change-Id: I3378fe04f2e090ab231705bde7e13d6289a9183e
Transparent (endpoint-preserving) restarts with identity transfer
are meant to exercise the crash recovery system only. After *real*
crashes, such restarts are useless at best and dangerous at worst,
because no state integrity can be guaranteed afterwards. Thus,
except after a controlled crash, it is best not to perform such
restarts at all. This patch changes SEF such that identity transfer
is successful only if the old process was the subject of a crash
induced through "service fi". As a result, testrelpol.sh should
continue to be able to use identity transfers for testing purposes,
but any real crash will be handled more appropriately.
This fixes#126.
Change-Id: Idc17ac7b3dfee05098529cb889ac835a0cd03ef0
At least it works again now. Sprofalyze should be made aware of the
kernel information page, though (i.e., /proc/ipcvecs).
Change-Id: Id4e5f6417ad152607c4e53b323b6f65ea4b10c6e
In order to resolve page faults on file-mapped pages, VM may need to
communicate (through VFS) with a file system. The file system must
therefore not be the one to cause, and thus end up being blocked on,
such page faults. To resolve this potential deadlock, the safecopy
system was previously extended with the CPF_TRY flag, which causes the
kernel to return EFAULT to the caller of a safecopy function upon
getting a pagefault, bypassing VM and thus avoiding the loop. VFS was
extended to repeat relevant file system calls that returned EFAULT,
after resolving the page fault, to keep these soft faults from being
exposed to applications.
However, general UNIX I/O semantics dictate that if an I/O transfer
partially succeeded before running into a failure, the partial result
is to be returned. Proper file system implementations may therefore
end up returning partial success rather than the EFAULT code resulting
from a soft fault. Since VFS does not get the EFAULT code in this
case, it does not know that a soft fault occurred, and thus does not
repeat the call either. The end result is that an application may get
partial I/O results (e.g., a short read(2)) even on regular files.
Applications cannot reasonably be expected to deal with this.
Due to the fact that most of the current file system implementations
do not implement proper partial-failure semantics, this problem is not
yet widespread. In fact, it has only occurred on direct block device
I/O so far. However, the next generation of file system services will
be implementing proper I/O semantics, thus exacerbating the problem.
To remedy this situation, this patch changes the CPF_TRY semantics:
whenever the kernel experiences a soft fault during a safecopy call,
in addition to returning FAULT, the kernel also stores a mark in the
grant created with CPF_TRY. Instead of testing on EFAULT, VFS checks
whether the grant was marked, as part of revoking the grant. If the
grant was indeed marked by the kernel, VFS repeats the file system
operation, regardless of its initial return value. Thus, the EFAULT
code now only serves to make the file system fail the call faster.
The approach is currently supported for both direct and magic grants,
but is used only with magic grants - arguably the only case where it
makes sense. Indirect grants should not have CPF_TRY set; in a chain
of indirect grants, the original grant is marked, as it should be.
In order to avoid potential SMP issues, the mark stored in the grant
is its grant identifier, so as to discard outdated kernel writes.
Whether this is necessary or effective remains to be evaluated.
This patch also cleans up the grant structure a bit, removing reserved
space and thus making the structure slightly smaller. The structure
is used internally between system services only, so there is no need
for binary compatibility.
Change-Id: I6bb3990dce67a80146d954546075ceda4d6567f8
With this change, obtaining an existing free grant is no longer an
operation of O(n) complexity. As a result, the now-deprecated
getgrant/setgrant part of the grants API also no longer has a
performance advantage.
Change-Id: Ic19308a76924c6242f9784244a6b3600e561e0fe
The memory grant identifier for safecopies now includes a sequence
number in its upper bits, to prevent accidental reuse of a grant ID
after revocation and subsequent reallocation. This should increase
overall system robustness by a tiny amount, and possibly help catch
bugs in system services early on. For now, the lower 20 bits of the
grant ID are used as grant table slot index (thus allowing for up to
a million grants per process), and the next 11 bits of the (signed
32-bit) grant ID are used to store the per-slot sequence number. As
grant IDs are never exposed to userland, the split can be changed
later on without breaking the userland ABI.
Change-Id: Ic34be27ff2a45db0ea5db037a24eef9efcd9ca40
Now that there are services other than PM and VFS that implement
userland system calls directly, these services may need to know about
events related to user processes. In particular, signal delivery may
have to interrupt blocking system calls, and certain cleanup tasks may
have to be performed after a user process exits.
This patch aims to implement a generic, lasting solution for this
problem, by allowing services to subscribe to "signal delivered"
and/or "process exit" events from PM. PM publishes such events by
sending messages to its subscribed services, which must then reply an
acknowledgment message.
For now, only the two aforementioned events are implemented, and only
the IPC service makes use of the process event facility.
The new process event publish/subscribe system replaces the previous
VM notify-sig/watch-exit/query-exit system, which was unsound: 1) it
allowed subscription to events from individual processes, and suffered
from fundamental race conditions as a result; 2) it relied on "not too
many" processes making use of the IPC server functionality in order to
avoid loss of notifications. In addition, it had the "ipc" process
name hardcoded, did not distinguish between signal delivery and exits,
and added a roundtrip to VM for all events from all processes.
Change-Id: I75ebad4bc54e646c6433f473294cb4003b2c3430
This functionality is required for BSD top(1), as exposed through
the CTL_KERN KERN_CP_TIME sysctl(2) call. The idea is that the
overall time spent in the system is divided into five categories.
While NetBSD uses a separate category for the kernel ("system") and
interrupts, we redefine "system" to mean userspace system services
and "interrupts" to mean time spent in the kernel, thereby providing
the same categories as MINIX3's own top(1), while adding the "nice"
category which, like on NetBSD, is used for time spent by processes
with a priority lowered by the system administrator.
Change-Id: I2114148d1e07d9635055ceca7b163f337c53c43a
- the userland call is now made to PM only, and PM relays the call to
other servers as appropriate; this is an ABI change that will
ultimately allow us to add proper support for wait3() and the like;
for the moment there is backward compatibility;
- the getrusage-specific kernel subcall has been removed, as it
provided only redundant functionality, and did not provide the means
to be extended correctly in the future - namely, allowing the kernel
to return different values depending on whether resource usage of
the caller (self) or its children was requested;
- VM is now told whether resource usage of the caller (self) or its
children is requested, and it refrains from filling in wrong values
for information it does not have;
- VM now uses the correct unit for the ru_maxrss values;
- VFS is cut out of the loop entirely, since it does not provide any
values at the moment; a comment explains how it should be readded.
Change-Id: I27b0f488437dec3d8e784721c67b03f2f853120f
This change serves to reduce the clutter inside the top-level kerninfo
structure, and allows other ARM-specific values to be added on the
kernel page in one place.
Change-Id: I36a6aada9dbd1230b25014728be675d389088667
Please note that this information is for use by system services only!
The clock facility is not ready to be used directly by userland, and
thus, this kernel page extension is NOT part of the userland ABI.
For service programmers' convenience, change the prototype of the
getticks(3) to return the uptime clock value directly, since the call
can no longer fail.
Correct the sys_times(2) reply message to use the right field type
for the boot time.
Restructure the kernel internals a bit so as to have all the clock
stuff closer together.
Change-Id: Ifc050b7bd253aecbe46e3bd7d7cc75bd86e45555
Instead of importing an external _minix_kerninfo variable, any code
using the shared kernel page should now call get_minix_kerninfo(3).
Since this is the only logical name for such a function, rename the
previous get_minix_kerninfo call to ipc_minix_kerninfo.
Change-Id: I2e424b6fb55aa55d3da850187f1f7a0b7cbbf910
- do not allow live update for request and protocol free states if
there are any worker threads that have pending or active work;
- destroy all worker threads before such live updates and recreate
them afterwards, because transferring (the contents of) the
thread stacks is not an option at this time;
- recreate worker threads in the new instance only if they were
shut down before the state transfer, by letting RS provide the
original preparation state as initialization information.
Change-Id: I846225f5b7281f19e69175485f2c88a4b4891dc2
Due to changed VM internals, more elaborate preparation is required
before a live update with multiple components including VM can take
place. This patch adds the essential preparation infrastructure to
VM and adapts RS to make use of it. As a side effect, it is no
longer necessary to supply RS as the last component (if at all)
during the set-up of a multicomponent live update operation.
Change-Id: If069fd3f93f96f9d5433998e4615f861465ef448
The 'memory' service has holes in its data section, which causes
problems during state transfer. Since VM cannot handle page faults
during a multicomponent-with-VM live update, the state transfer must
ensure that no page faults occur during copying. Therefore, we now
query VM about the regions to copy, thus skipping holes. While the
solution is not ideal, it is sufficiently generic that it can be used
for the data section state transfer of all processes, and possibly
for state transfer of other regions in the future as well.
Change-Id: I2a71383a18643ebd36956c396fbd22c8fd137202
VM used to call sendrec to send a boot-time RS_INIT reply to RS, but
RS could run into a pagefault at the same time, thus spawning a
message to VM, resulting in a deadlock. We resolve this situation by
making VM acknowledge RS_INIT asynchronously at boot time, while
retaining the synchronous sendrec for subsequent RS_INIT responses.
Change-Id: I3cb72d7f8d6b9bfdc59a85958ada739c37fa3bde
The following services have been updated to support stateful restarts:
- Drivers: tty
- Filesystems: isofs, mfs, pfs, libvtreefs-based file servers
- System servers: tty, ds, pm, vfs, vm
Change-Id: Ie84baa3ba1774047b3ae519808fe4116928edabb
This patch employs one solution to resolve two independent but related
issues. Both issues are the result of one fundamental aspect of the
way VM's memory mapping works: VM uses its cache to map in blocks for
memory-mapped file regions, and for blocks already in the VM cache, VM
does not go to the file system before mapping them in. To preserve
consistency between the FS and VM caches, VM relies on being informed
about all updates to file contents through the block cache. The two
issues are both the result of VM not being properly informed about
such updates:
1. Once a file system provides libminixfs with an inode association
(inode number + inode offset) for a disk block, this association
is not broken until a new inode association is provided for it.
If a block is freed and reallocated as a metadata (non-inode)
block, its old association is maintained, and may be supplied to
VM's secondary cache. Due to reuse of inodes, it is possible
that the same inode association becomes valid for an actual file
block again. In that case, when that new file is memory-mapped,
under certain circumstances, VM may end up using the metadata
block to satisfy a page fault on the file, due to the stale inode
association. The result is a corrupted memory mapping, with the
application seeing data other than the current file contents
mapped in at the file block.
2. When a hole is created in a file, the underlying block is freed
from the device, but VM is not informed of this update, and thus,
if VM's cache contains the block with its previous inode
association, this block will remain there. As a result, if an
application subsequently memory-maps the file, VM will map in the
old block at the position of the hole, rather than an all-zeroes
block. Thus, again, the result is a corrupted memory mapping.
This patch resolves both issues by making the file system inform the
minixfs library about blocks being freed, so that libminixfs can
break the inode association for that block, both in its own cache and
in the VM cache. Since libminixfs does not know whether VM has the
block in its cache or not, it makes a call to VM for each block being
freed. Thus, this change introduces more calls to VM, but it solves
the correctness issues at hand; optimizations may be introduced
later. On the upside, all freed blocks are now marked as clean,
which should result in fewer blocks being written back to the device,
and the blocks are removed from the caches entirely, which should
result in slightly better cache usage.
This patch is necessary but not sufficient to resolve the situation
with respect to memory mapping of file holes in general. Therefore,
this patch extends test 74 with a (rather particular but effective)
test for the first issue, but not yet with a test for the second one.
This fixes#90.
Change-Id: Iad8b134d2f88a884f15d3fc303e463280749c467
This barrier ensures that all fields of an asynchronously sent
message are properly initialized before the message is marked as
valid.
Change-Id: I7b9590c11c4e040c8f992f1dd2581e54201bf214
Previously, there was a tiny chance that tickdelay(3) would return
early or that it would fail to reinstate a previous alarm.
- sys_setalarm(2) now returns TMR_NEVER instead of 0 for the time
left if no previous alarm was set;
- sys_setalarm(2) now also returns the current time, to allow the
caller to determine whether it got an alarm notification for the
alarm it set or for a previous alarm that has just gone off;
- tickdelay(3) now makes use of these facilities.
Change-Id: Id4f8fe19a61ca8574f43131964e6f0317f613f49
