The kernel.ipc.sysvipc_info node is the gateway from NetBSD ipcs(1)
and ipcrm(1) to the IPC server, and thus necessary for a clean
import of these two utilities. The MIB service implementation uses
the preexisting (Linux-specific) information calls on the IPC server
to obtain the information.
Change-Id: I85d1e193162d6b689f114764254dd7f314d2cfa0
As mentioned in previous patches, services may not subscribe to
process events from specific processes only, since this results in
race conditions. However, the IPC server can safely turn on and off
its entire subscription based on whether any System V IPC semaphores
(and, in the future, message queues) are allocated at all. Since
the System V IPC facilities are not so commonly used, this removes
the extra round trip from PM to the IPC server and back for caught
signals and process exits in the common case.
Change-Id: I937259034872be32f4e26ab99270f4d475ff6134
- rewrite the semop(2) implementation so that it now conforms to the
specification, including atomicity, support for blocking more than
once, range checks, but also basic fairness support;
- fix permissions checking;
- fix missing time adjustments;
- fix off-by-one errors and other bugs;
- do not allocate dynamic memory for GETALL/SETALL;
- add test88, which properly tests the semaphore functionality.
Change-Id: I85f0d3408c0d6bba41cfb4c91a34c8b46b2a5959
Now that there are services other than PM and VFS that implement
userland system calls directly, these services may need to know about
events related to user processes. In particular, signal delivery may
have to interrupt blocking system calls, and certain cleanup tasks may
have to be performed after a user process exits.
This patch aims to implement a generic, lasting solution for this
problem, by allowing services to subscribe to "signal delivered"
and/or "process exit" events from PM. PM publishes such events by
sending messages to its subscribed services, which must then reply an
acknowledgment message.
For now, only the two aforementioned events are implemented, and only
the IPC service makes use of the process event facility.
The new process event publish/subscribe system replaces the previous
VM notify-sig/watch-exit/query-exit system, which was unsound: 1) it
allowed subscription to events from individual processes, and suffered
from fundamental race conditions as a result; 2) it relied on "not too
many" processes making use of the IPC server functionality in order to
avoid loss of notifications. In addition, it had the "ipc" process
name hardcoded, did not distinguish between signal delivery and exits,
and added a roundtrip to VM for all events from all processes.
Change-Id: I75ebad4bc54e646c6433f473294cb4003b2c3430
Closer to KNF, better coding practices, more similar to other
services, no more global variables, a few more comments, that
kind of stuff. No major functional changes.
Change-Id: I6e8f53bfafd6f41e92031fba76c40a31d2107a8e
- switch to the NetBSD identifier system; it is not only better, but
also required for porting NetBSD ipcs(1) and ipcrm(1); however, it
requires that slots not be moved, and that results in some changes;
- synchronize some other things with NetBSD: where keys are kept, as
well as various non-permission mode flags;
- fix semctl(2) vararg retrieval and message field type;
- use SUSPEND instead of weird reply exceptions in the call table;
- fix several memory leaks and at least one missing permission check;
- improve the atomicity of semop(2) by a small amount, even though
its atomicity is still broken at a fundamental level;
- use the new cheaper way to retrieve the current time;
- resolve all level-5 LLVM warnings.
Change-Id: I0c47aacde478b23bb77d628384aeab855a22fdbf
Specifically, add support for the IPC_INFO, SEM_INFO, and SEM_STAT
semctl(2) operations, similar to how information about shared memory
is already exposed as well. The MINIX3 ipcs(1) utility already had
support for these operations, and can now actually use them, too.
Change-Id: Ice5a02e729bf6df6aa8fab76e854808adc04dae3
- About 80% of PM's process table consisted of per-signal sigaction
structures. This is information not used by the MIB service, and
can safely be stored outside the main process table.
- The MIB service does not need most of the VFS process table, so VFS
now generates a "light" version of its table upon request, with just
the fields used by the MIB service.
The result is a size reduction of the MIB service of about 700KB.
Change-Id: I79fe7239361fbfb45286af8e86a10aed4c2d2be7
Instead of pulling in process tables itself, ProcFS now queries the
MIB service for process information. This reduces ProcFS's memory
usage by about 1MB. The change does have two negative consequences.
First, getting all the original /proc/<pid>/psinfo fields filled in
would take a lot of extra effort. Since the only program that uses
those files at all is mtop(1), we reformat psinfo to expose only the
information used by mtop(1). This means that with this patch, older
copies of MINIX3 ps and top will cease to work.
Second, since both MIB and ProcFS update their own view of the
process list only once per clock tick, ProcFS' view may now be
outdated by up to two clock ticks. This is unlikely to pose a
problem in practice.
Change-Id: Iaa6b60450c8fb52d092962394d33d08bd638bc01
Now that uname(3) uses sysctl(2), we no longer need sysuname(2).
Backward compatibility is retained for old statically linked
binaries for a short while.
Also remove the now-obsolete MINIX3-specific "arch" field from the
utsname structure. While this is an ABI break at the libc level,
it should pose no problems in practice, because:
- statically linked programs (i.e., all of the base system) are not
affected, as they will use headers synchronized with libc;
- the structure is getting smaller, thus, older dynamically linked
programs (typically in pkgsrc) using the new libc will end up with
garbage in the "arch" field, but it is unlikely they will use this
field anyway, since it was specific to MINIX3;
- new dynamically linked programs using an old libc could end up with
memory corruption, but this is not a scenario that is expected to
occur in the first place - certainly not with programs from pkgsrc.
Change-Id: I29c76576f509feacc8f996f0bd353ca8961d4917
PM uses its own process table entry as source for kernel signals,
and temporarily changes its own process group to make the signals
arrive at the right processes. However, the value is never reset,
with as result that the temporary value shows up in ps(1) output.
Change-Id: Ib7f635b2cf1958055123736dfd58c26530632785
So far, VM reported only the number of bytes actually allocated to
each process. This patch adds two additional fields: the sum of the
byte sizes of all the virtual address ranges in the process, and that
number minus the part of the process stack that is not actually
mapped in. Unfortunately, we have to guess where the process stack
is, so the second field is not necessarily accurate.
Change-Id: If9e07c20e8588bc3e11601ec79bdcebc06eba6ee
This functionality is required for BSD top(1), as exposed through
the CTL_KERN KERN_CP_TIME sysctl(2) call. The idea is that the
overall time spent in the system is divided into five categories.
While NetBSD uses a separate category for the kernel ("system") and
interrupts, we redefine "system" to mean userspace system services
and "interrupts" to mean time spent in the kernel, thereby providing
the same categories as MINIX3's own top(1), while adding the "nice"
category which, like on NetBSD, is used for time spent by processes
with a priority lowered by the system administrator.
Change-Id: I2114148d1e07d9635055ceca7b163f337c53c43a
The new MIB service implements the sysctl(2) system call which, as
we adopt more NetBSD code, is an increasingly important part of the
operating system API. The system call is implemented in the new
service rather than as part of an existing service, because it will
eventually call into many other services in order to gather data,
similar to ProcFS. Since the sysctl(2) functionality is used even
by init(8), the MIB service is added to the boot image.
MIB stands for Management Information Base, and the MIB service
should be seen as a knowledge base of management information.
The MIB service implementation of the sysctl(2) interface is fairly
complete; it incorporates support for both static and dynamic nodes
and imitates many NetBSD-specific quirks expected by userland. The
patch also adds trace(1) support for the new system call, and adds
a new test, test87, which tests the fundamental operation of the
MIB service rather thoroughly.
Change-Id: I4766b410b25e94e9cd4affb72244112c2910ff67
Magic instrumentation is now performed on all system services if the
system is built with MKMAGIC=yes, which implies MKBITCODE=yes.
Change-Id: I9d1233650188b7532a9356b720fb68d5f8248939
Until now, the program name of a service was always the file name
(without directory) of the service binary. The program name is used
to, among other things, find the corresponding system.conf entry.
With ASR moving to a situation where all rerandomized service binaries
are stored in a single directory, this can no longer be maintained.
Instead, the service(8) command can now be instructed to override the
service program name, using its new -progname option.
Change-Id: I981e9b35232c88048d8804ec5eca58d1e4a5db82
This brings our tree to NetBSD 7.0, as found on -current on the
10-10-2015.
This updates:
- LLVM to 3.6.1
- GCC to GCC 5.1
- Replace minix/commands/zdump with usr.bin/zdump
- external/bsd/libelf has moved to /external/bsd/elftoolchain/
- Import ctwm
- Drop sprintf from libminc
Change-Id: I149836ac18e9326be9353958bab9b266efb056f0
This patch adds support for the wait4 system call, and with that the
wait3 call as well. The implementation is absolutely minimal: only
user and system times of the exited child are returned (with all other
rusage fields left zero), and there is no support for tracers. Still,
this should cover the main use cases of wait4.
Change-Id: I7a04589a8423a23990ab39aa38e85d535556743a
- the userland call is now made to PM only, and PM relays the call to
other servers as appropriate; this is an ABI change that will
ultimately allow us to add proper support for wait3() and the like;
for the moment there is backward compatibility;
- the getrusage-specific kernel subcall has been removed, as it
provided only redundant functionality, and did not provide the means
to be extended correctly in the future - namely, allowing the kernel
to return different values depending on whether resource usage of
the caller (self) or its children was requested;
- VM is now told whether resource usage of the caller (self) or its
children is requested, and it refrains from filling in wrong values
for information it does not have;
- VM now uses the correct unit for the ru_maxrss values;
- VFS is cut out of the loop entirely, since it does not provide any
values at the moment; a comment explains how it should be readded.
Change-Id: I27b0f488437dec3d8e784721c67b03f2f853120f
The current values were both inaccurate (especially for dynamically
linked executables) and using the wrong unit (bytes, instead of
kilobytes times ticks-of-execution). For now we are better off not
populating these fields at all.
Change-Id: I195a8fa8db909e64a833eec25f59c9ee0b89bdc5
POSIX states that times() and getrusage() should only return child
user and system times of terminated children for which wait*() has
returned their PIDs.
Change-Id: I38e19ad71543a3b91e944bef8e4e1bd903de51bf
Currently, the userland ABI uses a single field ('user_sp') far
into the very large 'kinfo' structure on the shared kernel
information page. This precludes us from modifying or getting
rid of 'kinfo' in the future without breaking userland. This
patch adds a separate 'kuserinfo' structure to the kernel
information page, with only information that is part of the
userland ABI, in an extensible manner. Userland now uses this
field if it is present, and falls back to the old field if not.
Change-Id: Ib7b24b53a440f40a2edc28cdfa48447ac2179288
Please note that this information is for use by system services only!
The clock facility is not ready to be used directly by userland, and
thus, this kernel page extension is NOT part of the userland ABI.
For service programmers' convenience, change the prototype of the
getticks(3) to return the uptime clock value directly, since the call
can no longer fail.
Correct the sys_times(2) reply message to use the right field type
for the boot time.
Restructure the kernel internals a bit so as to have all the clock
stuff closer together.
Change-Id: Ifc050b7bd253aecbe46e3bd7d7cc75bd86e45555
Instead of importing an external _minix_kerninfo variable, any code
using the shared kernel page should now call get_minix_kerninfo(3).
Since this is the only logical name for such a function, rename the
previous get_minix_kerninfo call to ipc_minix_kerninfo.
Change-Id: I2e424b6fb55aa55d3da850187f1f7a0b7cbbf910
This commits adds a basic infrastructure to support Address Space
Randomization (ASR). In a nutshell, using the already imported ASR
LLVM pass, multiple versions can be generated for the same system
service, each with a randomized, different address space layout.
Combined with the magic instrumentation for state transfer, a system
service can be live updated into another ASR-randomized version at
runtime, thus providing live rerandomization.
Since MINIX3 is not yet capable of running LLVM linker passes, the
ASR-randomized service binaries have to be pregenerated during
crosscompilation. These pregenerated binaries can then be cycled
through at runtime. This patch provides the basic proof-of-concept
infrastructure for both these parts.
In order to support pregeneration, the clientctl host script has
been extended with a "buildasr" command. It is to be used after
building the entire system with bitcode and magic support, and will
produce a given number of ASR-randomized versions of all system
services. These services are placed in /usr/service/asr in the
image that is generated as final step by the "buildasr" command.
In order to support runtime updating, a new update_asr(8) command
has been added to MINIX3. This command attempts to live-update the
running system services into their next ASR-randomized versions.
For now, this command is not run automatically, and thus must be
invoked manually.
Technical notes:
- For various reasons, magic instrumentation is x86-only for now,
and ASR functionality is therefore to be used on x86 only as well.
- The ASR-randomized binaries are placed in numbered subdirectories
so as not to have to change their actual program names, which are
assumed to be static in various places (system.conf, procfs).
- The root partition is typically too small to contain all the
produced binaries, which is why we introduce /usr/service. There
is a symlink from /service/asr to /usr/service/asr for no other
reason than to let userland continue to assume that all services
are reachable through /service.
- The ASR count field (r_asr_count/ASRcount) maintained by RS is not
used within RS in any way; it is only passed through procfs to
userland in order to allow update_asr(8) to keep track of which
version is currently loaded without having to maintain own state.
- Ideally, pre-instrumentation linking of a service would remove all
its randomized versions. Currently, the user is assumed not to
perform ASR instrumentation and then recompile system services
without performing ASR instrumentation again, as the randomized
binaries included in the image would then be stale. This aspect
has to be improved later.
- Various other issues are flagged in the comments of the various
parts of this patch.
Change-Id: I093ad57f31c18305591f64b2d491272288aa0937
Lack of alignment causes minix_stack_fill to produce an incorrect
frame layout, subsequently resulting in a crash of the started
process. For now, we assume that the other callers of
minix_stack_fill do get an aligned buffer through sbrk(3), but this
may have to be changed later as well.
Change-Id: I1575bd62b050749d1a1aae7417310c91713462c3
The libexec ELF parser expects to be given a word-aligned buffer,
but the ASR pass may cause VM and VFS to pass it an arbitrarily
aligned buffer, causing libexec to refuse loading the executable.
This patch aligns the buffers explicitly.
Change-Id: Ic2d5fd3a8f204c3e4f000cffdb7ac71c8339257a
- do not allow live update for request and protocol free states if
there are any worker threads that have pending or active work;
- destroy all worker threads before such live updates and recreate
them afterwards, because transferring (the contents of) the
thread stacks is not an option at this time;
- recreate worker threads in the new instance only if they were
shut down before the state transfer, by letting RS provide the
original preparation state as initialization information.
Change-Id: I846225f5b7281f19e69175485f2c88a4b4891dc2
The bitcode file given to the instrumentation pass does not include
certain weak symbols, in particular regcomp and regfree, which are
required to be visible to the magic pass for state transfer to work
correctly. This patch forces DS to make the calls using their actual
symbol names (with leading underscore), thus resolving the issue, but
this issue should really be solved in a cleaner and more generic way.
Change-Id: Iebee4341cc30ddabcf7593afb5c49d41c0839863
That way, these pages are transferred during live update, as they
should. This resolves a mfs crash after a number of live updates.
Change-Id: Ia53bec2692b2114c29b96a453beb0f915f56453a
This patch changes the VM makefile to specify that the magic pass is
to skip memory function instrumentation, and to transfer the data
variables of the malloc code (thus overriding the exception we made
for all other system services). We add two magic pass flags to
achieve this. Since the magic pass is a big bowl of spaghetti code,
ignoring whitespace changes while viewing this patch is recommended.
Change-Id: I5ab83b23d8437b37c44dea99537bc202469c9df6
Due to changed VM internals, more elaborate preparation is required
before a live update with multiple components including VM can take
place. This patch adds the essential preparation infrastructure to
VM and adapts RS to make use of it. As a side effect, it is no
longer necessary to supply RS as the last component (if at all)
during the set-up of a multicomponent live update operation.
Change-Id: If069fd3f93f96f9d5433998e4615f861465ef448
During live update, the new instance of VM may make changes that,
after a rollback, have to be undone by the old instance of VM, in
particular because both instances share (read-write) all dynamically
allocated pages.
Change-Id: I2bcfa8e627ca6084b1991e0af7cccecc683894a2
A missing check to see whether the range being transferred is sane
(with a starting address lower than an ending address) caused extra
memory to be marked erroneously as copy-on-write for some processes,
ultimately resulting in pagefaults on the stack during live update
rollback.
Change-Id: I1516b509b485379606d8df05b8a0f514896a0f19
If the stack is not mapped at the VM_DATATOP (e.g. booted with
ac_layout = 1), there might be some more regions hiding above
the stack. We also have to transfer those.
Change-Id: Idf3b94a36fcec8a10ace2f6dffe816faf0a88f60
. make sure the priv id etc is maintained so
future privctl talk about the right thing
. solves broken IPC after update
Change-Id: I17ed0212c22d634e6db1e80f8dcb2fb8bffe82c6
The 'memory' service has holes in its data section, which causes
problems during state transfer. Since VM cannot handle page faults
during a multicomponent-with-VM live update, the state transfer must
ensure that no page faults occur during copying. Therefore, we now
query VM about the regions to copy, thus skipping holes. While the
solution is not ideal, it is sufficiently generic that it can be used
for the data section state transfer of all processes, and possibly
for state transfer of other regions in the future as well.
Change-Id: I2a71383a18643ebd36956c396fbd22c8fd137202
Two bugs fixed wrt vm restartability.
. make sure pagetable data is only allocated
using dynamic data instead of static spare pages
(bootstrap pages). They are needed for bootstrap
but now repeat some of the initialization so only
dynamic data remains. This solves the problem of
physical addresses changing (as static pages are
re-allocated for the new instance) after update.
. pt_ptalloc has to be specified in bytes instead of
pde slot numbers. leaving pt_pt NULL causes mapping
transfers to fail because NULL happens to be mapped in
then and updates then happen there.
. added some sanity checks against the above happening.
The new state is that VM can update many times, but the system
isn't fully reliable afterwards yet.
Change-Id: I7313602c740cdae8590589132291116ed921aed7
Allow extra space for in-band metadata when allocating cache blocks.
Edited by David van Moolenbroek: since this effectively halves the
potential size of the typical file system cache, do this only when
compiling with instrumentation.
Change-Id: I0840af6420899ede2d5bb7539e79c0a456b5128d
