6969 Commits

Author SHA1 Message Date
David van Moolenbroek
a3975fbc35 Kernel: disable assert in prefetch abort handler
For a reason currently unknown to us, the qemu-linaro emulator
sometimes produces a Prefetch Abort exception with a fault location
(IFAR) rather different from the location of the instruction being
executed (LR corrected by 4).  So far it has been observed in the
__udivmodsi4 routine of various processes, where the fault address is
for the first byte of the next page after the current instruction,
which itself is 44-64 bytes away from the start of that next page.
The affected instruction does not perform any sort of memory access.

Short of debugging qemu-linaro itself, we have no choice but to
disable the assert that previously went off in case the IFAR and
corrected LR are not equal.  Since we have not yet observed this case
on actual hardware, the kernel prints a warning when detecting such a
mismatch for the first time.  For the qemu-linaro case, the kernel's
actual page fault handling logic already handles this strange case
just fine.

Change-Id: Ibd19e624149ab4e68bfe75b918ec1554b825a431
2016-03-09 12:11:23 +00:00
David van Moolenbroek
28aa92fd90 rc: start syslogd with -s (no UDP sockets)
Change-Id: I80f46c1c525d2573614676cdd612e284c16be481
2016-02-29 16:28:16 +00:00
David van Moolenbroek
7399f63e53 3c90x: add support for 3c905B 100BaseTX
Tested and found working.  No driver changes required.

Change-Id: Ib05e7540a6264f784b6342d6e84c4e11423b2ec9
2016-02-27 15:37:20 +00:00
David van Moolenbroek
a617090dc0 setup.sh: fix various issues
- fix the reinstallation (preserve-/home) option;
- remove support for just reinstalling the bootloader, as the main
  purpose of this option (allowing an upgrade from the old MINIX
  boot monitor to the NetBSD bootloader) is no longer needed and was
  already broken;
- do not try to copy over /etc/motd.install: it no longer exists.

This resolves issue 106.

Change-Id: Iad3805d86c4806d725f9b285c2d4378670790f78
2016-02-26 19:45:55 +00:00
David van Moolenbroek
26d958c71e VFS: remove unused variable in worker.c
Change-Id: Ife41d292ab50a36c75dc28b682684095654bfcf2
2016-02-26 19:28:50 +00:00
David van Moolenbroek
c38dbb97aa Prepare for switch to native BSD socket API
Currently, the BSD socket API is implemented in libc, translating the
API calls to character driver operations underneath.  This approach
has several issues:

- it is inefficient, as most character driver operations are specific
  to the socket type, thus requiring that each operation start by
  bruteforcing the socket protocol family and type of the given file
  descriptor using several system calls;
- it requires that libc itself be changed every time system support
  for a new protocol is added;
- various parts of the libc implementations violate the asynchronous
  signal safety POSIX requirements.

In order to resolve all these issues at once, the plan is to turn the
BSD socket calls into system calls, thus making the BSD socket API the
"native" ABI, removing the complexity from libc and instead letting
VFS deal with the socket calls.

The overall change is going to break all networking functionality. In
order to smoothen the transition, this patch introduces the fifteen
new BSD socket system calls, and makes libc try these first before
falling back on the old behavior.  For now, the VFS implementations of
the new calls fail such that libc will always use the fallback cases.
Later on, when we introduce the actual implementation of the native
BSD socket calls, all statically linked programs will automatically
use the new ABI, thus limiting actual application breakage.

In other words: by itself, this patch does nothing, except add a bit
of transitional overhead that will disappear in the future.  The
largest part of the patch is concerned with adding full support for
the new BSD socket system calls to trace(1) - this early addition has
the advantage of making system call tracing output of several socket
calls much more readable already.

Both the system call interfaces and the trace(1) support have already
been tested using code that will be committed later on.

Change-Id: I3460812be50c78be662d857f9d3d6840f3ca917f
2016-02-23 14:34:05 +00:00
David van Moolenbroek
0df28c9fa4 libc: reorganize vector I/O wrappers
The reorganization allows other libc system call wrappers (namely,
sendmsg and recvmsg) to perform I/O vector coalescing as well.

Change-Id: I116b48a6db39439053280ee805e0dcbdaec667a3
2016-02-22 23:24:47 +00:00
David van Moolenbroek
c33d6ef392 VFS: start off cleanup of pipe2 IPC message
There is no reason to use a single message for nonoverlapping requests
and replies combined, and in fact splitting them out allows reuse of
messages and avoids various problems with field layouts.  Since the
upcoming socketpair(2) system call will be using the same reply as
pipe2(2), split up the single message used for the latter.  In order
to keep the used parts of messages at the front, start a transitional
phase to move the pipe(2) flags field to the front of its request.

Change-Id: If3f1c3d348ec7e27b7f5b7147ce1b9ef490dfab9
2016-02-22 23:23:02 +00:00
David van Moolenbroek
17580212b4 libc: check raw IP socket type before using it
Previously, the libc sendto(3) and recvfrom(3) implementations would
blindly assume that any unrecognized socket is a raw-IP socket.  This
is not only inconsistent but also messes with returned error codes.

Change-Id: Id0328f04ea8ca0968a4e8636bc441caa0c3579b7
2016-02-22 23:21:05 +00:00
Lionel Sambuc
5ef5b27fc1 Cross-compilation fixes:
- GCC >4.9:
     Newer toolchains trigger more warnings, which by default are
   treated as errors. Disable this while building the tools as this
   will be a recurring problem in the future.

   close #105

 - FreeBSD: Fix some fetch.sh scripts which fails as FreeBSD's patch
   fails to patch two .info files. We ignore this for the time being.

Change-Id: Ic669281db6c41005119ea8f76f78b5ec60e1b386
2016-02-08 13:05:55 +01:00
rlfnb
eaf726b35c ifdef for switching RTS/CTS off
Change-Id: Iccc4b151b2047539dfd79ce0ff1381c0579539e9
2016-02-07 20:37:08 +01:00
Lionel Sambuc
86b583c518 Fix usage of parenthesis in Makefiles
While BSD make support both $() and ${} around variables, the NetBSD
source tree uses only ${} by convention.

Imported software is left as is, and sometimes $() is used when the
containing Makefile/Makefile fragment is used both by GNU make and BSD
make, as it can happen for the tools, and other parts as well which are
compiled using the host make tool.

Change-Id: Ic7d480812fde53e7e3e95275a30a3b720c95cc15
2016-02-07 19:17:44 +01:00
rlfnb
59ca1d7542 moved service configuration out of system.conf
Change-Id: I996c0e700b087c5130ac6480ae39e1eefa771eff
2016-02-07 19:17:43 +01:00
Thomas Cort
b88949904f mail: close fp after use
Add missing `fclose(fp)`

Issue #109

Change-Id: I0908ede77cba18336c399c5e16771cb6650e6404
2016-02-07 09:50:08 -05:00
David van Moolenbroek
373b793920 Basic statistical profiling fixes
At least it works again now.  Sprofalyze should be made aware of the
kernel information page, though (i.e., /proc/ipcvecs).

Change-Id: Id4e5f6417ad152607c4e53b323b6f65ea4b10c6e
2016-01-28 13:09:35 +01:00
David van Moolenbroek
1d80a87694 bootloader: do not try to autoload NetBSD modules
The NetBSD bootloader attempts to load NetBSD kernel modules for
"unusual" file systems.  We do not support NetBSD kernel modules,
and thus, the bootloader gives us warnings about not being able
to load them, in particular when booting CD images.  This patch
disables the NetBSD file system module autoload feature.

Change-Id: I55fce53b4bb0282b7d8a005192200fe466312f62
2016-01-25 19:29:11 +01:00
David van Moolenbroek
4f5713cc0a tests: another fix
- test80: resolve race condition resulting from unintended send

Change-Id: Id029d679a3903de0f712a15d4756952dbc36070c
2016-01-25 19:07:40 +01:00
David van Moolenbroek
99e8768deb tests: fixes
- test3: support running the test set from a pseudoterminal;
- test60: fix number conversion bug that caused chmod errors;
- test65: remove nonworking package installation instructions;
- testisofs: work around failure due to having a timezone set;
- testisofs: exclude extra RR_MOVED directory from output.

Change-Id: Ibfcc631de7e2f4da46bac3ad9de8d7c7cd7a6189
2016-01-25 14:26:20 +01:00
Lionel Sambuc
e1cdaee106 Fix ARM noassert builds -g/-O{s/0/1/2/3}
Also fixes ARM assert build -O3.

Change-Id: I52bda91308ecfa0e8b23c4140c38c49347cc10f7
2016-01-24 18:56:38 +01:00
Lionel Sambuc
6a76619b7e Bumping version to 3.4.0
Change-Id: Ia5630d7409cf7042853847ee29866e2db3514c91
2016-01-22 12:24:28 +01:00
Lionel Sambuc
39508cdda5 setup.sh: clean up
- Fix unmounting order of slices.
 - Update pkgin pre-installation commands, make them as generic as
   possible.

Change-Id: Ifaa4021fed048facca8d2a170aa65491feb37702
2016-01-22 12:24:22 +01:00
Lionel Sambuc
6c380635c9 Server packages: Make sure /usr/pkg/var/run exists
If this directory doesn't exist, pid files are not created, which create
issues when shutting down or rebooting.

Change-Id: I52dddb57aca4368b1775606e22818fba99d05bf6
2016-01-22 12:24:03 +01:00
Lionel Sambuc
fb4fbf7a0c Fix X11 forwarding: sendmsg/recvmsg only supported over UDS
Change-Id: Id9dc5517ec09382010a556fbabc6dccc77942378
2016-01-22 12:23:47 +01:00
Lionel Sambuc
125832e358 Improve libexec debug output
Change-Id: I6a82d3eeed263e813852549fac015b82832c5c29
2016-01-16 14:04:28 +01:00
Lionel Sambuc
ace2de0ad1 Reduce sysident patches
Change-Id: I4499a294e74ed69648afc33bff5f750f3d1aa019
2016-01-16 14:04:27 +01:00
Gerhard Poul
d578d57bfe Fix typo in comment
Change-Id: Iefc0f11ac81da1d681785530e33a4668d8614de9
2016-01-16 14:04:26 +01:00
Lionel Sambuc
6e48120e2b Fix umount order in setup.sh
Change-Id: Ife0c36b0fdc70e4b7ae86278d0df8d6d00d19df4
2016-01-16 14:04:26 +01:00
Lionel Sambuc
226d301a62 tests: add most kyua kernel tests
Change-Id: I2da22559ee3b0208945e21312e7721a32e629d27
2016-01-16 14:04:25 +01:00
Lionel Sambuc
b3bac6e0e7 ld.elf_so: sysctl support uncommented
Change-Id: I253f3b4a2ef8c5b3fb332eb98a08d328a9591c52
2016-01-16 14:04:24 +01:00
Lionel Sambuc
2988757685 ld.elf_so: Drop fallback routine
Change-Id: Ia2236146c3302ba992aa6fea0fc9e9699fe9e3cd
2016-01-16 14:04:23 +01:00
Thomas Cort
ef31660ff2 mined: fix buffer overflow in input()
input() is used to accept filenames when saving, regular
expressions when searching, and other input. It writes
the characters into buffers such as file and exp_buf and
others which are of length LINE_LEN.

To prevent writing beyond the end of the intended buffer,
truncate the input at LINE_LEN - 1 and ensure that the
string is NULL terminated.

Change-Id: I142baa8cfae38bdd7fa648d86559d6d9b8e7a7fd
2016-01-16 14:04:22 +01:00
David van Moolenbroek
10b7016b5a Fix soft faults in FSes resulting in partial I/O
In order to resolve page faults on file-mapped pages, VM may need to
communicate (through VFS) with a file system.  The file system must
therefore not be the one to cause, and thus end up being blocked on,
such page faults.  To resolve this potential deadlock, the safecopy
system was previously extended with the CPF_TRY flag, which causes the
kernel to return EFAULT to the caller of a safecopy function upon
getting a pagefault, bypassing VM and thus avoiding the loop.  VFS was
extended to repeat relevant file system calls that returned EFAULT,
after resolving the page fault, to keep these soft faults from being
exposed to applications.

However, general UNIX I/O semantics dictate that if an I/O transfer
partially succeeded before running into a failure, the partial result
is to be returned.  Proper file system implementations may therefore
end up returning partial success rather than the EFAULT code resulting
from a soft fault.  Since VFS does not get the EFAULT code in this
case, it does not know that a soft fault occurred, and thus does not
repeat the call either.  The end result is that an application may get
partial I/O results (e.g., a short read(2)) even on regular files.
Applications cannot reasonably be expected to deal with this.

Due to the fact that most of the current file system implementations
do not implement proper partial-failure semantics, this problem is not
yet widespread.  In fact, it has only occurred on direct block device
I/O so far.  However, the next generation of file system services will
be implementing proper I/O semantics, thus exacerbating the problem.

To remedy this situation, this patch changes the CPF_TRY semantics:
whenever the kernel experiences a soft fault during a safecopy call,
in addition to returning FAULT, the kernel also stores a mark in the
grant created with CPF_TRY.  Instead of testing on EFAULT, VFS checks
whether the grant was marked, as part of revoking the grant.  If the
grant was indeed marked by the kernel, VFS repeats the file system
operation, regardless of its initial return value.  Thus, the EFAULT
code now only serves to make the file system fail the call faster.

The approach is currently supported for both direct and magic grants,
but is used only with magic grants - arguably the only case where it
makes sense.  Indirect grants should not have CPF_TRY set; in a chain
of indirect grants, the original grant is marked, as it should be.
In order to avoid potential SMP issues, the mark stored in the grant
is its grant identifier, so as to discard outdated kernel writes.
Whether this is necessary or effective remains to be evaluated.

This patch also cleans up the grant structure a bit, removing reserved
space and thus making the structure slightly smaller.  The structure
is used internally between system services only, so there is no need
for binary compatibility.

Change-Id: I6bb3990dce67a80146d954546075ceda4d6567f8
2016-01-16 14:04:21 +01:00
David van Moolenbroek
efc775b4c8 libsys: use linked list for free grants
With this change, obtaining an existing free grant is no longer an
operation of O(n) complexity.  As a result, the now-deprecated
getgrant/setgrant part of the grants API also no longer has a
performance advantage.

Change-Id: Ic19308a76924c6242f9784244a6b3600e561e0fe
2016-01-16 14:04:20 +01:00
David van Moolenbroek
ca779acd72 Add sequence numbers to grant IDs
The memory grant identifier for safecopies now includes a sequence
number in its upper bits, to prevent accidental reuse of a grant ID
after revocation and subsequent reallocation.  This should increase
overall system robustness by a tiny amount, and possibly help catch
bugs in system services early on.  For now, the lower 20 bits of the
grant ID are used as grant table slot index (thus allowing for up to
a million grants per process), and the next 11 bits of the (signed
32-bit) grant ID are used to store the per-slot sequence number.  As
grant IDs are never exposed to userland, the split can be changed
later on without breaking the userland ABI.

Change-Id: Ic34be27ff2a45db0ea5db037a24eef9efcd9ca40
2016-01-16 14:04:19 +01:00
David van Moolenbroek
59f1f7ecdd Kernel: apply x86 copy overflow check to ARM
Apply the x86 overflow check from git-d09f72c to ARM code as well.
Not just stack traces, but also system services can trigger this
case, possibly as a result of being handed bad pointers by userland,
ending in a kernel panic.

Change-Id: Ib817e8b682fafec8edb486a094319ad11eda7081
2016-01-16 14:04:19 +01:00
Lionel Sambuc
f12160c14d Kernel: synchronize i386 and ARM clock code
Change-Id: Ie5c4653299e47aaefc9d35a8af491ad0b2eab1ab
2016-01-16 14:04:18 +01:00
Dr. Florian Grätz
6077d1ad24 Kernel: convert K&R functions to ANSI
Changed all K&R style functions to ANSI-style declarations within the
kernel directory. The code compiles and aparently works for i386. For
arm my toolchain does not work, but I have changed the code with great
care. Also, the make command fails for the test suite. Therefore, I
strongly recommand to review the code with care.

Edited by David van Moolenbroek to convert really all K&R functions.

Change-Id: I58cde797d36f4caa9c72db4e4dc27d8545ab8866
2016-01-16 14:04:17 +01:00
rlfnb
d8c1e15ae6 e1000: add i82583V 1000baseT Ethernet
added device id for the i82583V NIC. Tested successful on bare metal.

Change-Id: I486ea60a52a16075a1da8ffa0280884a407acc30
2016-01-16 14:04:16 +01:00
David van Moolenbroek
b58e161ccb trace(1): resolve all level-5 LLVM warnings
Change-Id: If5ffe97eb0b15387b1ab674657879e13f58fb27e
2016-01-16 14:04:15 +01:00
David van Moolenbroek
0502421f1b Import NetBSD ipcrm(1)
A few MINIX3-specific changes are necessary due to the fact that we
are missing the System V IPC message queue system calls.

Change-Id: Idd252984be9df69618cef79bcf6c676cbf915d85
2016-01-16 14:04:14 +01:00
David van Moolenbroek
7b09d0426a Import NetBSD ipcs(1)
The old ipcs(1) utility had a dubious license at best.

Change-Id: Ic4c0a24f04c94cc77a43658f6c939d138461bd6a
2016-01-16 14:04:13 +01:00
David van Moolenbroek
2f09e77b82 MIB: add support for System V IPC information node
The kernel.ipc.sysvipc_info node is the gateway from NetBSD ipcs(1)
and ipcrm(1) to the IPC server, and thus necessary for a clean
import of these two utilities.  The MIB service implementation uses
the preexisting (Linux-specific) information calls on the IPC server
to obtain the information.

Change-Id: I85d1e193162d6b689f114764254dd7f314d2cfa0
2016-01-16 14:04:12 +01:00
David van Moolenbroek
5b1db95619 IPC server: subscribe to process events on demand
As mentioned in previous patches, services may not subscribe to
process events from specific processes only, since this results in
race conditions.  However, the IPC server can safely turn on and off
its entire subscription based on whether any System V IPC semaphores
(and, in the future, message queues) are allocated at all.  Since
the System V IPC facilities are not so commonly used, this removes
the extra round trip from PM to the IPC server and back for caught
signals and process exits in the common case.

Change-Id: I937259034872be32f4e26ab99270f4d475ff6134
2016-01-16 14:04:11 +01:00
David van Moolenbroek
56dc79cea0 IPC server: major fixes, test set for semaphores
- rewrite the semop(2) implementation so that it now conforms to the
  specification, including atomicity, support for blocking more than
  once, range checks, but also basic fairness support;
- fix permissions checking;
- fix missing time adjustments;
- fix off-by-one errors and other bugs;
- do not allocate dynamic memory for GETALL/SETALL;
- add test88, which properly tests the semaphore functionality.

Change-Id: I85f0d3408c0d6bba41cfb4c91a34c8b46b2a5959
2016-01-16 14:04:11 +01:00
David van Moolenbroek
910831cb5c PM: generic process event publish/subscribe system
Now that there are services other than PM and VFS that implement
userland system calls directly, these services may need to know about
events related to user processes.  In particular, signal delivery may
have to interrupt blocking system calls, and certain cleanup tasks may
have to be performed after a user process exits.

This patch aims to implement a generic, lasting solution for this
problem, by allowing services to subscribe to "signal delivered"
and/or "process exit" events from PM.  PM publishes such events by
sending messages to its subscribed services, which must then reply an
acknowledgment message.

For now, only the two aforementioned events are implemented, and only
the IPC service makes use of the process event facility.

The new process event publish/subscribe system replaces the previous
VM notify-sig/watch-exit/query-exit system, which was unsound: 1) it
allowed subscription to events from individual processes, and suffered
from fundamental race conditions as a result; 2) it relied on "not too
many" processes making use of the IPC server functionality in order to
avoid loss of notifications.  In addition, it had the "ipc" process
name hardcoded, did not distinguish between signal delivery and exits,
and added a roundtrip to VM for all events from all processes.

Change-Id: I75ebad4bc54e646c6433f473294cb4003b2c3430
2016-01-16 14:04:10 +01:00
David van Moolenbroek
0baafa0ef4 IPC server: restyle
Closer to KNF, better coding practices, more similar to other
services, no more global variables, a few more comments, that
kind of stuff.  No major functional changes.

Change-Id: I6e8f53bfafd6f41e92031fba76c40a31d2107a8e
2016-01-16 14:03:55 +01:00
David van Moolenbroek
4d272e5a97 IPC server: NetBSD sync, general improvements
- switch to the NetBSD identifier system; it is not only better, but
  also required for porting NetBSD ipcs(1) and ipcrm(1); however, it
  requires that slots not be moved, and that results in some changes;
- synchronize some other things with NetBSD: where keys are kept, as
  well as various non-permission mode flags;
- fix semctl(2) vararg retrieval and message field type;
- use SUSPEND instead of weird reply exceptions in the call table;
- fix several memory leaks and at least one missing permission check;
- improve the atomicity of semop(2) by a small amount, even though
  its atomicity is still broken at a fundamental level;
- use the new cheaper way to retrieve the current time;
- resolve all level-5 LLVM warnings.

Change-Id: I0c47aacde478b23bb77d628384aeab855a22fdbf
2016-01-16 13:58:47 +01:00
David van Moolenbroek
58c1923c9f IPC server: expose semaphore information
Specifically, add support for the IPC_INFO, SEM_INFO, and SEM_STAT
semctl(2) operations, similar to how information about shared memory
is already exposed as well.  The MINIX3 ipcs(1) utility already had
support for these operations, and can now actually use them, too.

Change-Id: Ice5a02e729bf6df6aa8fab76e854808adc04dae3
2016-01-13 20:32:56 +01:00
David van Moolenbroek
9d796422b1 Import NetBSD pagesize(1)
Change-Id: I6a749cba5c6dabb3daea8c5bad3e94be342833d7
2016-01-13 20:32:55 +01:00
David van Moolenbroek
6ad322a932 MIB: slim process tables to reduce memory usage
- About 80% of PM's process table consisted of per-signal sigaction
  structures.  This is information not used by the MIB service, and
  can safely be stored outside the main process table.

- The MIB service does not need most of the VFS process table, so VFS
  now generates a "light" version of its table upon request, with just
  the fields used by the MIB service.

The result is a size reduction of the MIB service of about 700KB.

Change-Id: I79fe7239361fbfb45286af8e86a10aed4c2d2be7
2016-01-13 20:32:55 +01:00