In order to resolve page faults on file-mapped pages, VM may need to
communicate (through VFS) with a file system. The file system must
therefore not be the one to cause, and thus end up being blocked on,
such page faults. To resolve this potential deadlock, the safecopy
system was previously extended with the CPF_TRY flag, which causes the
kernel to return EFAULT to the caller of a safecopy function upon
getting a pagefault, bypassing VM and thus avoiding the loop. VFS was
extended to repeat relevant file system calls that returned EFAULT,
after resolving the page fault, to keep these soft faults from being
exposed to applications.
However, general UNIX I/O semantics dictate that if an I/O transfer
partially succeeded before running into a failure, the partial result
is to be returned. Proper file system implementations may therefore
end up returning partial success rather than the EFAULT code resulting
from a soft fault. Since VFS does not get the EFAULT code in this
case, it does not know that a soft fault occurred, and thus does not
repeat the call either. The end result is that an application may get
partial I/O results (e.g., a short read(2)) even on regular files.
Applications cannot reasonably be expected to deal with this.
Due to the fact that most of the current file system implementations
do not implement proper partial-failure semantics, this problem is not
yet widespread. In fact, it has only occurred on direct block device
I/O so far. However, the next generation of file system services will
be implementing proper I/O semantics, thus exacerbating the problem.
To remedy this situation, this patch changes the CPF_TRY semantics:
whenever the kernel experiences a soft fault during a safecopy call,
in addition to returning FAULT, the kernel also stores a mark in the
grant created with CPF_TRY. Instead of testing on EFAULT, VFS checks
whether the grant was marked, as part of revoking the grant. If the
grant was indeed marked by the kernel, VFS repeats the file system
operation, regardless of its initial return value. Thus, the EFAULT
code now only serves to make the file system fail the call faster.
The approach is currently supported for both direct and magic grants,
but is used only with magic grants - arguably the only case where it
makes sense. Indirect grants should not have CPF_TRY set; in a chain
of indirect grants, the original grant is marked, as it should be.
In order to avoid potential SMP issues, the mark stored in the grant
is its grant identifier, so as to discard outdated kernel writes.
Whether this is necessary or effective remains to be evaluated.
This patch also cleans up the grant structure a bit, removing reserved
space and thus making the structure slightly smaller. The structure
is used internally between system services only, so there is no need
for binary compatibility.
Change-Id: I6bb3990dce67a80146d954546075ceda4d6567f8
With this change, obtaining an existing free grant is no longer an
operation of O(n) complexity. As a result, the now-deprecated
getgrant/setgrant part of the grants API also no longer has a
performance advantage.
Change-Id: Ic19308a76924c6242f9784244a6b3600e561e0fe
The memory grant identifier for safecopies now includes a sequence
number in its upper bits, to prevent accidental reuse of a grant ID
after revocation and subsequent reallocation. This should increase
overall system robustness by a tiny amount, and possibly help catch
bugs in system services early on. For now, the lower 20 bits of the
grant ID are used as grant table slot index (thus allowing for up to
a million grants per process), and the next 11 bits of the (signed
32-bit) grant ID are used to store the per-slot sequence number. As
grant IDs are never exposed to userland, the split can be changed
later on without breaking the userland ABI.
Change-Id: Ic34be27ff2a45db0ea5db037a24eef9efcd9ca40
The kernel.ipc.sysvipc_info node is the gateway from NetBSD ipcs(1)
and ipcrm(1) to the IPC server, and thus necessary for a clean
import of these two utilities. The MIB service implementation uses
the preexisting (Linux-specific) information calls on the IPC server
to obtain the information.
Change-Id: I85d1e193162d6b689f114764254dd7f314d2cfa0
Now that there are services other than PM and VFS that implement
userland system calls directly, these services may need to know about
events related to user processes. In particular, signal delivery may
have to interrupt blocking system calls, and certain cleanup tasks may
have to be performed after a user process exits.
This patch aims to implement a generic, lasting solution for this
problem, by allowing services to subscribe to "signal delivered"
and/or "process exit" events from PM. PM publishes such events by
sending messages to its subscribed services, which must then reply an
acknowledgment message.
For now, only the two aforementioned events are implemented, and only
the IPC service makes use of the process event facility.
The new process event publish/subscribe system replaces the previous
VM notify-sig/watch-exit/query-exit system, which was unsound: 1) it
allowed subscription to events from individual processes, and suffered
from fundamental race conditions as a result; 2) it relied on "not too
many" processes making use of the IPC server functionality in order to
avoid loss of notifications. In addition, it had the "ipc" process
name hardcoded, did not distinguish between signal delivery and exits,
and added a roundtrip to VM for all events from all processes.
Change-Id: I75ebad4bc54e646c6433f473294cb4003b2c3430
- switch to the NetBSD identifier system; it is not only better, but
also required for porting NetBSD ipcs(1) and ipcrm(1); however, it
requires that slots not be moved, and that results in some changes;
- synchronize some other things with NetBSD: where keys are kept, as
well as various non-permission mode flags;
- fix semctl(2) vararg retrieval and message field type;
- use SUSPEND instead of weird reply exceptions in the call table;
- fix several memory leaks and at least one missing permission check;
- improve the atomicity of semop(2) by a small amount, even though
its atomicity is still broken at a fundamental level;
- use the new cheaper way to retrieve the current time;
- resolve all level-5 LLVM warnings.
Change-Id: I0c47aacde478b23bb77d628384aeab855a22fdbf
Instead of pulling in process tables itself, ProcFS now queries the
MIB service for process information. This reduces ProcFS's memory
usage by about 1MB. The change does have two negative consequences.
First, getting all the original /proc/<pid>/psinfo fields filled in
would take a lot of extra effort. Since the only program that uses
those files at all is mtop(1), we reformat psinfo to expose only the
information used by mtop(1). This means that with this patch, older
copies of MINIX3 ps and top will cease to work.
Second, since both MIB and ProcFS update their own view of the
process list only once per clock tick, ProcFS' view may now be
outdated by up to two clock ticks. This is unlikely to pose a
problem in practice.
Change-Id: Iaa6b60450c8fb52d092962394d33d08bd638bc01
Now that uname(3) uses sysctl(2), we no longer need sysuname(2).
Backward compatibility is retained for old statically linked
binaries for a short while.
Also remove the now-obsolete MINIX3-specific "arch" field from the
utsname structure. While this is an ABI break at the libc level,
it should pose no problems in practice, because:
- statically linked programs (i.e., all of the base system) are not
affected, as they will use headers synchronized with libc;
- the structure is getting smaller, thus, older dynamically linked
programs (typically in pkgsrc) using the new libc will end up with
garbage in the "arch" field, but it is unlikely they will use this
field anyway, since it was specific to MINIX3;
- new dynamically linked programs using an old libc could end up with
memory corruption, but this is not a scenario that is expected to
occur in the first place - certainly not with programs from pkgsrc.
Change-Id: I29c76576f509feacc8f996f0bd353ca8961d4917
This functionality is required for BSD top(1), as exposed through
the CTL_KERN KERN_CP_TIME sysctl(2) call. The idea is that the
overall time spent in the system is divided into five categories.
While NetBSD uses a separate category for the kernel ("system") and
interrupts, we redefine "system" to mean userspace system services
and "interrupts" to mean time spent in the kernel, thereby providing
the same categories as MINIX3's own top(1), while adding the "nice"
category which, like on NetBSD, is used for time spent by processes
with a priority lowered by the system administrator.
Change-Id: I2114148d1e07d9635055ceca7b163f337c53c43a
The new MIB service implements the sysctl(2) system call which, as
we adopt more NetBSD code, is an increasingly important part of the
operating system API. The system call is implemented in the new
service rather than as part of an existing service, because it will
eventually call into many other services in order to gather data,
similar to ProcFS. Since the sysctl(2) functionality is used even
by init(8), the MIB service is added to the boot image.
MIB stands for Management Information Base, and the MIB service
should be seen as a knowledge base of management information.
The MIB service implementation of the sysctl(2) interface is fairly
complete; it incorporates support for both static and dynamic nodes
and imitates many NetBSD-specific quirks expected by userland. The
patch also adds trace(1) support for the new system call, and adds
a new test, test87, which tests the fundamental operation of the
MIB service rather thoroughly.
Change-Id: I4766b410b25e94e9cd4affb72244112c2910ff67
The magic runtime library is now built as part of the regular build, if
the MKMAGIC=yes flag is passed to the build system. The library has
been renamed from "magic" to "magicrt" to resolve a name clash with BSD
file(1)'s libmagic. All its level-5 LLVM warnings have been resolved.
The final library, "libmagicrt.bcc", is now stored in the destination
library directory rather than in the source tree.
Change-Id: Iebd4b93a2cafbb59f95d938ad1edb8b4f6e729f6
This brings our tree to NetBSD 7.0, as found on -current on the
10-10-2015.
This updates:
- LLVM to 3.6.1
- GCC to GCC 5.1
- Replace minix/commands/zdump with usr.bin/zdump
- external/bsd/libelf has moved to /external/bsd/elftoolchain/
- Import ctwm
- Drop sprintf from libminc
Change-Id: I149836ac18e9326be9353958bab9b266efb056f0
The CD now boots directly from the ISO 9660 filesystem instead of using
MBR partitioning with Minix file systems. This saves some space on the
CD and reduces memory requirements by some unknown amount as the root
ramdisk is completely eliminated.
The x86 hard drive image creation is also rewritten in the same
fashion.
The setup is modified to be more NetBSD-like (unpacking sets
tarballs instead of blindly copying the CD contents). Splitting MINIX
into sets is done in another commit due to it being a nightmare to
rebase.
Since MINIX lacks union mounts for now, a bunch of ramdisks are
generated at run-time to make parts of the filesystem writeable for the
CD. This solution isn't ideal, but it's enough for an installation CD.
Change-Id: Icbd9cca4dafebf7b42c345b107a17679a622d5cd
This patch adds support for the wait4 system call, and with that the
wait3 call as well. The implementation is absolutely minimal: only
user and system times of the exited child are returned (with all other
rusage fields left zero), and there is no support for tracers. Still,
this should cover the main use cases of wait4.
Change-Id: I7a04589a8423a23990ab39aa38e85d535556743a
- the userland call is now made to PM only, and PM relays the call to
other servers as appropriate; this is an ABI change that will
ultimately allow us to add proper support for wait3() and the like;
for the moment there is backward compatibility;
- the getrusage-specific kernel subcall has been removed, as it
provided only redundant functionality, and did not provide the means
to be extended correctly in the future - namely, allowing the kernel
to return different values depending on whether resource usage of
the caller (self) or its children was requested;
- VM is now told whether resource usage of the caller (self) or its
children is requested, and it refrains from filling in wrong values
for information it does not have;
- VM now uses the correct unit for the ru_maxrss values;
- VFS is cut out of the loop entirely, since it does not provide any
values at the moment; a comment explains how it should be readded.
Change-Id: I27b0f488437dec3d8e784721c67b03f2f853120f
- move MINIX3-specific files into minix/lib/libpuffs;
- resynchronize the remaining files with NetBSD code;
- remove a few unnecessary changes;
- put remaining MINIX3-specific changes in __minix blocks;
- sort out the source files being linked at all.
The result is that libpuffs now successfully links against FUSE
file system programs again. It can successfully mount, perform
some of the most basic operations, and unmount the file system.
Change-Id: Ieac220f7ad8c4d1fa293abda81967e8045be0bb4
Various generic file IOCTL calls should be processed by VFS rather
than individual drivers. For this reason, we rewrite them to use
fcntl(2) instead.
Change-Id: I38a5f3c7b21943a897114a51678a800f7e7f0a77
Currently, the userland ABI uses a single field ('user_sp') far
into the very large 'kinfo' structure on the shared kernel
information page. This precludes us from modifying or getting
rid of 'kinfo' in the future without breaking userland. This
patch adds a separate 'kuserinfo' structure to the kernel
information page, with only information that is part of the
userland ABI, in an extensible manner. Userland now uses this
field if it is present, and falls back to the old field if not.
Change-Id: Ib7b24b53a440f40a2edc28cdfa48447ac2179288
This change serves to reduce the clutter inside the top-level kerninfo
structure, and allows other ARM-specific values to be added on the
kernel page in one place.
Change-Id: I36a6aada9dbd1230b25014728be675d389088667
Please note that this information is for use by system services only!
The clock facility is not ready to be used directly by userland, and
thus, this kernel page extension is NOT part of the userland ABI.
For service programmers' convenience, change the prototype of the
getticks(3) to return the uptime clock value directly, since the call
can no longer fail.
Correct the sys_times(2) reply message to use the right field type
for the boot time.
Restructure the kernel internals a bit so as to have all the clock
stuff closer together.
Change-Id: Ifc050b7bd253aecbe46e3bd7d7cc75bd86e45555
Instead of importing an external _minix_kerninfo variable, any code
using the shared kernel page should now call get_minix_kerninfo(3).
Since this is the only logical name for such a function, rename the
previous get_minix_kerninfo call to ipc_minix_kerninfo.
Change-Id: I2e424b6fb55aa55d3da850187f1f7a0b7cbbf910
- do not allow live update for request and protocol free states if
there are any worker threads that have pending or active work;
- destroy all worker threads before such live updates and recreate
them afterwards, because transferring (the contents of) the
thread stacks is not an option at this time;
- recreate worker threads in the new instance only if they were
shut down before the state transfer, by letting RS provide the
original preparation state as initialization information.
Change-Id: I846225f5b7281f19e69175485f2c88a4b4891dc2
Since the heap is reconstructed upon state transfer, the old malloc
state is discarded. In order to avoid state transfer errors, we can
and in fact must discard the internal state of the malloc
implementation. This patch achieves this by using the sectionify
pass to mark the variables in the libminc malloc object as state that
must be skipped during state transfer.
Change-Id: Ie330f582c8bd45f37a878ea41fa0f9d4a18045e1
Due to changed VM internals, more elaborate preparation is required
before a live update with multiple components including VM can take
place. This patch adds the essential preparation infrastructure to
VM and adapts RS to make use of it. As a side effect, it is no
longer necessary to supply RS as the last component (if at all)
during the set-up of a multicomponent live update operation.
Change-Id: If069fd3f93f96f9d5433998e4615f861465ef448
The 'memory' service has holes in its data section, which causes
problems during state transfer. Since VM cannot handle page faults
during a multicomponent-with-VM live update, the state transfer must
ensure that no page faults occur during copying. Therefore, we now
query VM about the regions to copy, thus skipping holes. While the
solution is not ideal, it is sufficiently generic that it can be used
for the data section state transfer of all processes, and possibly
for state transfer of other regions in the future as well.
Change-Id: I2a71383a18643ebd36956c396fbd22c8fd137202
Edited by David van Moolenbroek to deallocate the guard page as well.
Note that while the new approach is better in theory (previously, the
hole could end up being filled by another allocated page), guard page
protection is now broken in practice, because VM does not support
setting specific page permissions (in this case, PROT_NONE).
Change-Id: I882624f5d152d3ebe82fca649cbad85aa4931780
VM used to call sendrec to send a boot-time RS_INIT reply to RS, but
RS could run into a pagefault at the same time, thus spawning a
message to VM, resulting in a deadlock. We resolve this situation by
making VM acknowledge RS_INIT asynchronously at boot time, while
retaining the synchronous sendrec for subsequent RS_INIT responses.
Change-Id: I3cb72d7f8d6b9bfdc59a85958ada739c37fa3bde
The following services have been updated to support stateful restarts:
- Drivers: tty
- Filesystems: isofs, mfs, pfs, libvtreefs-based file servers
- System servers: tty, ds, pm, vfs, vm
Change-Id: Ie84baa3ba1774047b3ae519808fe4116928edabb
