David's TODO list

This is the to-do/wish list of David van Moolenbroek, who believes that all of these projects are either downright necessary or incredibly helpful for the MINIX3 project. It is not necessarily exhaustive, final, accurate, or in any way to be taken as indicative of what may happen in the future. Interested contributors are free to adopt any of these projects as their own and/or communicate with David about doing so.

• reason: the current libc implementation is not atomic and thus dangerous, aside from violating POSIX
• note: there are going to be other calls to which this applies, but dup2 is a rather obvious case

• reason: various places use either SUSPEND or EDONTREPLY, with the same meaning
• reason: SUSPEND is not defined in errno.h and thus may end up overlapping with existing error codes

• reason: needed for devfs, devmgr, and an improvement for procfs
• reason: requested by zentrum
• complication: processes issuing “cd ..” from any of those file systems while remounting
• note: we are rapidly moving to a situation where procfs no longer needs to be on the ramdisk anyway

• reason: a first step towards NetBSD convergence regarding block device access
• reason: better performance and less skew by avoiding going through a file system
• complication: the need for temporary buffers for I/O
• subsequent project: continuous block tracing through blocking character driver IOCTLs
• related project: take file systems out of the raw block device communication path altogether

• reason: as is, library code cannot be shared between system services and userland
• complication: making a script that does the conversion for all people currently doing system development

• reason: improved debugging
• complication: it seems that the current ARM compilation process does not generate stackframes at all

• reason: the current mess for select is actually bothering userland
• complication: there are a lot of POSIX requirements to be met here, and we know they're not all tested
• subsequent project: allow PFS to expand pipe sizes as memory is available

• reason: current duplication of hairy processing code
• complication: dealing with blocking calls; dealing with differing data structures
• complication: the console using a different level of interpretation than eg pty
• complication: there is effectively no test set for most of this
• note: it is unclear whether the library should cover part of the character driver processing
• note: this project interacts with the job control project
• subsequent project: allow multiple concurrent blocking calls to TTY

• reason: right now, the process isn't stopped on signal receipt, so its address space may still change
• complication: the process must not be able to ignore debugger signals
• note: there was at least one other major implementation issue

• reason: there is currently no advanced one and some basic features would often be good enough
• reason: printed stack traces are currently unusable with dynamically linked binaries
• complication: it's easy without support for symbols and disassembly, but also not very useful

• reason: missing system call, relatively easy to resolve
• complication: the behavior of the fch*at functions depends on the opened directory's O_SEARCH flag
• complication: must make sure that symbolic link resolution is not testing against the mode

• reason: the current limit is lower than it could be
• complication: this requires MFS to announce 64-bit support to VFS
• note: implementing a MFSv4 (through libfsgraph or otherwise) would obviate this point

• reason: the current approach requires extra IPC and extra state to be kept in the file system
• reason: seek behavior can be combined with posix_fadvise/madvise to provide hints

• reason: having to allocate a huge buffer in libc is far from ideal and ruins any benefits of using vectors
• complication: IOV_MAX is 1024 as per NetBSD, so allocating one grant per element is definitely not an option
• complication: the solution will have to work even for vector copy operations
• complication: for future kernel threading support, the system (eg VFS) will have to make a copy of the vector
• note: probably best implemented as a new grant type (_VMAGIC ?) and kernel support, maybe with a fast lookup table
• note: most of this could then be limited to the kernel and libsys safecopy code (which does mallocs as it is)
• note: now depends on BSD socket API project

• reason: applications such as tmux use it and may simply expect it to work
• note: this probably involves putting the request message in the sigframe and have the kernel pretend a resend
• complication: this should not get in the way of system call tracing

• reason: needed to support memory-mapped files, and save half of the cache memory, on isofs
• reason: needed for file systems such as FAT
• note: this will require decoupling file system blocks from I/O blocks
• note: it would be useful to have an interface that makes memory mapping of files as easy as possible for this case

• reason: these are essential interfaces for other programmers
• complication: we haven't settled whether these should be wiki pages, manual pages, or yet something else

• reason: services can do way too much
• reason: many more services should not be running as root
• complication: it's going to be hard to figure out which service needs what
• note: as part of this, it should be decided whether having one single service UID is tenable

• reason: expected by various parts of NetBSD userland
• reason: most executable text areas are currently writable, which is absolutely ridiculous for a “secure” OS
• complication: VM does not seem to have anything in the way of implementing this

• reason: the whole system can easily die if a service pagefaults while all memory is in use
• reason: requested by zentrum
• note: the easiest way is to have a pool with memory reserved for handling service pagefaults
• note: this may best be done in combination with a pool with memory reserved for contiguous memory

• reason: this is going to be increasingly necessary to have MINIX3 installed alongside other OSes
• complication: this affects not only libblockdriver but quite some of the boot process as well
• complication: it is unclear how /dev node naming, numbering should work, and whether devfs is needed
• note: need more discussion with antoineL about this first

• reason: (lots of reasons, TBD)
• complication: (lots of complications, TBD)

• reason: missing functionality
• complication: tricky in many ways, may end up doubling sem.c code size
• complication: needs an extensive test set to match (adding to test88 or separate)

• reason: ipcs/ipcrm expect this to be supported
• note: this will require that the process-is-blocked-on table be split from sem.c

• reason: where needed, this kind of stuff should now be handled by the MIB service
• reason: there are rare cases where recompiling procfs is not desirable
• complication: especially for /proc/services this will require moving a lot into MIB

• reason: essentially the same role is now fulfilled by the MIB service, making procfs somewhat redundant
• reason: we want to allow footprint reduction in general
• complication: not all of the information is available through the MIB service as it is
• complication: /etc/mtab is currently a symlink to /proc/mounts, this needs to be dealt with
• note: ideally, procfs should no longer be needed on the ramdisk

• reason: the current implementation is heavyweight (libc regex) and uses malloc through regcomp(3)
• reason: at the same time it does not fulfill needs of services, which want to have multiple independent subscriptions
• complication: this requires a proper look at all current DS publish/subscribe usage scenarios
• note: an example use case is RMIB, which wants to detect MIB-service restarts independent of the main code
• note: a solution based on multiple possible subscriptions to sets of service classes would probably suffice
• note: libsys should probably have a DS notification dispatcher, possibly as part of SEF
• note: this probably a good time to introduce a system-wide constant for service label sizes

• reason: a RMIB call to a deadlocked service may currently deadlock MIB, and with that the entire system
• complication: this may or may not require the MIB service to sign up for PM process events
• note: this depends on RMIB being notified about service deaths through DS, rather than via ipc_sendrec()

• problem: it is not possible to modify (= bump the version number of) already-mounted RMIB subtrees (e.g., net.interface)
• problem: it is not possible to mount RMIB subtrees using a name only (e.g. minix.lwip)
• problem: it is currently possible to mess up the MIB tree with bad name+id combos in RMIB mount requests

• reason: processes may currently be terminated while in an uninterruptible system call, possibly triggering poorly-tested scenarios in other services (safecopy failures, etc)
• complication: this will require changes to the PM signal state machine, with subtle side effects
• complication: involving all user-facing system services in exit notification is a performance problem
• note: the most obvious solution would be kernel support for notification (to PM) when a process system call has completed, comparable to SIGNDELAY, and a PREEXIT process state in PM
• note: ideally the same idea would be applied to signal handler invocation, because the current approach makes dangerous and already-incorrect(?) assumptions about “retreg” there

• reason: pselect(2) is required by dhcpcd(8) and various other parts of userland
• complication: pselect(2) is supposed to be atomic and thus cannot be implemented as a select(2) wrapper
• complication: a proper implementation will require a non-trivial extension to the PM/VFS protocol
• subsequent project: also implement paccept(2); this will require storing more call state in VFS

• status: IN PROGRESS
• reason: required by POSIX and generally desirable
• reason: allows for the introduction of various concepts currently expected by NetBSD userland
• complication: many fundamental issues remain to be resolved
• note: this is a new implementation that is conceptually based almost entirely on antoineL's work
• note: need more discussion with antoineL about this first

• status: PROTOTYPE READY, NEEDS TESTING
• reason: a first crash-consistent file system
• reason: start of the infrastructure for a reusable file system graph management layer (libfsgraph)
• complication: different file system will require different small exceptions
• complication: no neat way to model orphan management in the edge (directory) layer

• status: MERGED
• reason: libc should not need to test or track socket types, it violates the light-libc minix philosophy
• reason: the individual writes to implement sendto (etc) probably violate posix signal atomicity
• benefit: this allows for proper socket call support in trace(1)
• complication: this will break everything, require INET's retirement and a substantial UDS rewrite
• note: this is probably best done along with many other network stack related changes

• status: MERGED
• reason: this is causing problems with openssh server privilege separation, possibly openntpd (#61), possibly others
• complication: uid/gid management is so convoluted that a paper was published about it
• complication: lots of services other than PM and VFS (eg UDS, IPC..) are probably also doing the wrong thing

• status: MERGED
• reason: this is actually causing issues in installboot and who knows where else
• note: it seems that the best option is to have the kernel write a mark into the grant table upon failure
• note: cpf_revoke can be made to return the mark
• complication: for indirect grants, the original (magic) grant would have to be marked, not the last part of the chain

• status: MERGED
• reason: too easy to use a grant that has been revoked as it is
• note: a 16/16 split may limit the number of ongoing operations? then again, is that just a function of NR_PROCS?
• complication: must ensure that the own-address-space grant for live update is not negatively affected by this

• status: MERGED
• reason: ipcs(1) is currently not showing all available information
• note: this should be done as a MIB service extension now
• subsequent project: add support for those to trace(1) as well

• status: SUPERSEDED by MIB service
• reason: netbsd ps(1) is expected by /etc/rc.subr
• complication: having to deal with all fields that minix does not have
• complication: performance issues retrieving all this information from procfs
• note: ideally the same wrapper system would replace the one currently used by w(1)

• status: MERGED
• reason: tmux can currently not used by nonroot users
• complication: without devfs, we need a separate /dev/pts file system

• status: MERGED
• reason: performance optimization, but allows for functional simplification of time retrieval management
• complication: 64-bit support and atomic retrieval and all that
• subsequent project: removing lazy-time-update support from file systems

• status: MERGED
• reason: various userland programs need this, like time(1) and csh(1)
• complication: rusage information is scattered across services; getrusage calls into several
• complication: PM would have to copy in this information preemptively for zombies