Unix Domain Sockets

Student: Thomas Cort
Mentor: Thomas Veerman
SVN branch name: src.r7042.uds

The goal of this project is to implement Unix Domain Sockets in Minix3.

The basic implementation of unix domain sockets was merged into trunk on July 15, 2010. It includes updates to the header files, man pages, tests, pfs, vfs, rs, and libc. Unix domain sockets support these functions: accept(2), bind(2), close(2), connect(2), dup(2), dup2(2), getnucred(2), getpeereid(2), getpeername(2), getsockname(2), getsockopt(2), ioctl(2), listen(2), mknod(2), read(2), readv(2), recv(2), recvfrom(2), recvmsg(2), select(2), send(2), sendmsg(2), sendto(2), setsockopt(2), shutdown(2), socket(2), socketpair(2), write(2), writev(2). The following programs were also updated: fsck(1), fsck1(1), MAKEDEV(8), and DESCRIBE(8).

After the basic implementation of unix domain sockets was merged, some advanced features were implemented. Those advanced features are file descriptor passing (SCM_RIGHTS), credentials passing (SCM_CREDENTIALS), and adding permission checking and path canonicalization to VFS.

We want to test as many applications as possible that use Unix domain sockets. If you know of any not on the list, please add them.

This is the Google Summer of Code application period. During this time I developed a project proposal and completed the goat-sheep project. The goat-sheep project involved reading parts of the libc library, vfs server, pfs server, and inet server and answering questions. During this time I developed a test program on Linux to test the functionality and POSIX compliance of the unix domain sockets implementation. That program will eventually be used to test the implementation of unix domain sockets in minix.

This is the Google Summer of Code community bonding period. During this time I created accounts on the wiki and gforge, subscribed to the minix3 Google group and the Summer of Code Students Google group, began following @minix3 on twitter, and started hanging out on the #minix IRC channel. I setup a QEMU virtual machine and have it updated to the latest SVN trunk revision. I also did some studying. I read Dynamic Updates and Failure Resilience for the Minix File Server and VfsFsProtocol. I read and documented the PipeFS source code. I also reported bugs. I ||||ported GNU ed to minix to help familiarize myself with some of the differences between ACK & minix and GCC & Linux. I worked on improving my Unix Domain Sockets test program too. Specifically I added more comments, cleaned up some of the code, and made it integrate better with the existing test suite by using the functions defined in test/common.c.

I began the project by reading the specification / design document provided by my mentor. I implemented the header file changes and created stubs for sendmsg(2), recvmsg(2), and socketpair(2) in libc that I will fully implement later in the project. I implemented getpeereid(2) and getnucred(2) in libc and created man pages for the two functions. I also created skeleton man pages for the other socket functions that I will complete as go (for example, when I implement socket(2) for unix domain sockets I'll finish the socket(2) man page). Additionally, I imported my UDS test program. Then I spent some time reading about minix drivers and how to connect them to device files. I created /dev/uds and got it connected to PFS. I modified PFS to handle both file system messages and device messages. I modified socket(2) to open /dev/uds when a unix domain socket is needed. I added open and close functions in PFS to handle socket(2) and close(2). The open and close functions are partially complete and I will finish them by the next report. Also by the next report I hope to have several other socket functions implemented (bind(2), listen(2), accept(2), etc).

This week I finished implementing unix domain socket support for socket(2) and close(2). I implemented unix domain socket support for bind(2), listen(2), getpeername(2), and getsockname(2). I wrote documentation for those functions in the form of man pages. To support the implementation of those functions I made modifications which allow mknod(2) to create socket files. I also added an ioctl(2) handler to PFS. By the next report I hope to have connect(2) and accept(2) completed and begin work on read(2) and write(2).

This week I implemented socketpair(2) and shutdown(2). I also implemented part of connect(2) and did a tiny bit of accept(2). I spent some time reading about how processes are suspended while waiting for I/O. I began working on read(2) and write(2) for unix domain sockets in PFS. By the next report I hope to have connect(2) and accept(2) completed. I also hope to have read(2) and write(2) working for SOCK_STREAM unix domain sockets.

This week I spent my time working on accept(2)/connect(2) and read(2)/write(2). I had trouble getting the suspend and revive parts working properly. Revive didn't work as I expected, so I spent some time going down the wrong path. I'm back on track now and will hopefully have the four functions accept(2), connect(2), read(2), and write(2) completed by the next report. I also hope to start working on dup(2).

This week I finished accept(2) and connect(2). I got read(2), write(2), sendto(2), and recvfrom(2) working for SOCK_STREAM and SOCK_SEQPACKET unix domain sockets. I also implemented send(2) and recv(2). After some testing I discovered that dup(2) and dup2(2) are handled in the VFS and just work. With this week's work I was able to enable several more tests. I hope to complete the read/write/send*/recv* functions by the next report. I'd also like to get started on setsockopt(2) and getsockopt(2).

This week I finished read(2), write(2), sendto(2), recvfrom(2), setsockopt(2), and getsockopt(2). I updated the fsck and fsck1 commands to handle I_UNIX_SOCKET files so that they don't give “bad mode” errors on socket files. I also went through all of my code adding comments and cleaning up the rough spots. For next week I hope to finish select(2) and get a good start on sendmsg(2) and recvmsg(2). I will also look into compiling X11 with unix domain socket support.

This week I finished select(2) and a basic implementation of sendmsg(2) and recvmsg(2). Currently sending/receiving control data is unsupported by sendmsg(2) and recvmsg(2). After the unix domain sockets code is merged into trunk the plan is to start working on advanced features that use the control data part of msghdr (example: socket descriptor passing, etc). I also finished writing man pages for the socket functions this week. My test program, test56, has tests for every socket function, and the tests pass.

Towards the end of the week I started looking into trying out real world applications that use unix domain sockets. I tried installing the X11 package on my system with packman to see if X worked before attempting to compile it from source with UDS support. It didn't work, and it took down the whole system when I ran startx. I also looked at re-compiling python since it has unix sockets support and some unit tests. I got a linker error when linking libpython due to a problem with Berkley DB. I then tried compiling Berkley DB and got a linker error there too. I also tried the latest fltk-2.0.x so that I could try building dillo. I extracted the patch from bigports in svn and applied it to the new sources. I tried to compiling fltk and got a compiler error. I didn't look deeply into any of the errors I got because I was hoping to just find something that worked out of the box.

By the next status report I will update the PipeFS document to describe how unix domain sockets are implemented in PFS. I will also look into some of the errors I encountered this week, and I will begin working on some advanced features.

P.S. For fun I threw together a little application called udspong. It's an ncurses based 'screensaver' that features a bouncing ball. There is a process to draw the ball and a process to calculate the ball's next position. The two processes communicate over unix domain sockets.

My unix domain sockets implementation was merged into trunk this week

This week I tried troubleshooting my problems with X11. I couldn't get the built-in i810 video to work nor could I get a Trident PCI card to work. I tried both with their respective dirvers and the vesa driver. I assume it is either a bug or there was something it didn't like about my monitor / monitor settings. I didn't want to take away too much of my time from my minix development, so I gave up and installed minix inside a QEMU virtual machine on Linux with kvm enabled. I got X running and I'm ready to try re-compiling X with support for unix domain sockets next week.

This week I also updated my test program to be able to run as the 'bin' user. In addition, I changed the paths of the test files so that nothing outside the /usr/src/test directory is touched. It revealed a bug in the way credential passing was implemented. pfs was using the endpoint from the incoming device message (dev_m_in→IO_ENDPT) to identify the socket own. I found out that dev_m_in→IO_ENDPT isn't always the socket owner's endpoint. For example, the socket owner's endpoint is stored in dev_m_in→POSITION for IOCTLs.

By the next status report I will update the PipeFS document to describe how unix domain sockets are implemented in PFS, I will try compiling X with support for unix domain sockets, I will discuss the next steps with my mentor (implementing advanced features like passing file descriptors over unix domain sockets), and I will begin developing a test program in Linux to test the advanced features.

This week I added some additional tests to test56 to ensure that things were working properly. I also updated documentation on the wiki (PipeFS and VfsFsProtocol). I met with my mentor. The next steps for this project are testing packages that use unix domain sockets and implementing file descriptor passing.

I started testing applications that use unix domain sockets this week. A listing is at the top of the page. So far rsync works. Most of the other applications either use file descriptor passing or need features that minix doesn't have yet. By the next status report I hope to have a few packages that use unix domain sockets ported to minix and working. I also hope to get started on file descriptor passing. I found a BSD licensed library libancillary that implements file descriptor passing. It includes a nifty test program. I'll use it to test my implementation of file descriptor passing.

This week I did a little more porting work. Because of the pkgsrc GSoC project, the C library has been getting some new functions, like poll(2), which make porting a lot easier. I've also been studying the VFS. I added 5 functions to VFS that will be used for file descriptor passing. By the next status report I will add support for PFS to call those VFS functions. There are a few other changes I still need to make to the VFS. The goal is to have file descriptor passing working by the next status report.

This week I worked on PFS and VFS. I got the nested calling working so that PFS can call a predefined set of functions in VFS. The message protocol is documented above. The documentation will be moved to its own wiki page when code is merged into trunk. In check_perms() I have the permissions checking working, I got a start on canonicalizing the socket file path with the help of my mentor, and I made some updates to uds_bind() in pfs to call check_perms().

There is only a little over a week left in the Google Summer of Code program, and my goal is to finish the advanced UDS features. To finish I need to complete adding path canonicalization to VFS for check_perms(), add calls to check_perms() in a few places in pfs, make some libc changes, add functions in pfs to handle control data to implement file descriptor passing, add code to prevent an in-flight filp from being closed, and add some tests to test56.

This week I finished adding path canonicalization to VFS for check_perms(), I added calls to check_perms() in a few places in pfs, I made some libc changes, I added functions in pfs to handle control data to implement file descriptor and credentials passing, I added code to prevent an in-flight filp from being closed, and I added an SCM_CREDENTIALS test to test56.

There's only one day left in Summer of Code; the official end time is 19:00 UTC on August 16. On my last day I plan on adding tests to test56 to test file descriptor passing. With the remaining time I'll port/test applications that uses file descriptor passing.

Since Report 12, I've added a file descriptor passing test to test56. In the test there are two processes A and B. A opens a file, A passes the file descriptor to B, B writes a message to the file using the file descriptor from A, B closes the file, B let's A know it's done writing to the file, A seeks to the start of the file and reads the message, A confirms that B wrote the right message, and finally A closes the file.

Today I also ported pure-ftpd to minix3. It uses unix domain sockets mainly for file descriptor passing. I did some work on the wiki too (The PFS-VFS protocol).

Post-GSoC I plan on doing some more porting work. During GSoC I only had time to get 2 packages working on minix3 (pure-ftpd and rsync). I'll be starting with X11, postfix, and proftpd. I also plan on preparing and submitting patches for pure-ftpd.