There are many filesystems implementing FUSE API: ntfs-3g, fuse-unionfs, fuse-cddfs, etc. Main goal of this project is to make these filesystems available in MINIX 3 without any changes or at least with very minor changes (which can be sent to the upstream).
Week 1 (May 23 - May 29)
Checked ucontext implementation in MINIX. Example compiled with ACK segfaults because of bug in ACK (volatile is ignored). Since we decided to use gcc and later clang it's OK. Call context framework works fine with very minor changes (to be gone, when we switch to libc/lib* from NetBSD).
Framebuf stuff widely uses kevent(), which is currently unavailable. It is moved (discussed with Thomas) to the end of summer, because we will not use it in VFS-libpuffs messaging and it's not used in FUSE (PUFFS filesystems only).
Working on mounting (SEF initialization and READ_SUPER) I've decided to use function wrapper for main(). Argc and argv are unavaible in libpuffs, but they're required for env_setargs(). Also it should be a good idea to initialize SEF ASAP. Wrappers are supported by both gcc and clang. Kind of a hack, but in this case should be OK.
Week 2 (May 30 - Jun 5)
Done with mounting and control loop. Ported required components and cleaned unnecessary stuff. Like planed I will work on VFS↔PUFFS during next week. Unfortunately lost half of week because of unexpected deals at university. To be compensated during next week (or 2 weeks).
Week 3 (Jun 6 - Jun 12)
Fixed a bug in original libpuffs (caused submount failures in pnullfs tree) and sent a report/patch to NetBSD gnats. Also after rebuilding something in NetBSD I fall into serious troubles with pnullfs (several assertions failures), failed to found the reason and reverted to an old VM snapshot. But with that snapshot seems to work fine.
Worked on VFS↔PUFFS stuff. Mostly on lookup, stat, getdents. Started to test our libpuffs using pnullfs: it uses filesystem calls and current VFS doesn't allow nested fs calls except getsysinfo. To be discussed with Thomas.
Week 4 (Jun 13 - Jun 19)
Worked on VFS↔PUFFS stuff. Pnullfs (r/w) works fine and it is almost full functional. Should finish VFS↔PUFFS and pnullfs next week.
Week 5 (Jun 20 - Jun 26)
Basic VFS←→PUFFS glue is done (except bread/writes and flush, which to be added later, when we design caching). Pnullfs is tested and works fine except several bugs causing MINIX framework to fail.
Refuse is ported except very small piece of code.
Week 6 (Jun 27 - Jul 3)
Worked on testing and fixing libpuffs bugs. Also triaged several bugs in original libpuffs (patches to be submitted), mfs/ext2 glitch and small VFS glitch.
Started to work on new stat(). Almost done: changes are backward and forward compatible. There're few bugs to fix: test43 loops forever when “new_libc + new_vfs”, some tests linked with old libc have a strange issue with st_ctime (when run with new VFS). Also stat.h from gcc package has to be updated and nbsd_libc to be cleaned up (no xxx_stat simulation/conversion).
Week 7 (Jul 4 - Jul 10)
I have implemented new stat syscall. It is well tested, debugged and has initial review. Though Ben wants me to investigate if we can minimize minix-port.patch (nbsd_libc).
Also I separated minimal libc from libsys into libminc. Now drivers and servers can use libsys + libminc or libsys + libc (required for PUFFS/FUSE servers).
All code is rebased against recent master branch. fuse branch contains PUFFS/FUSE stuff only, my_base contains independent changes required for my project (new stat, libminc, fix for MFS/ext2 and “unlocked” nested calls support in VFS).
Libpuffs still requires some debugging and cleanup. With new stat librefuse works without any serious modifications (probably even vanilla with nbsd_libc). I'll update schedule according current state.
Week 8 (Jul 11 - Jul 17)
Worked on fixing libpuffs bugs (refuse is just fine). Now pnullfs passes most of /usr/src/test/Week 9 (Jul 18 - Jul 24) Fixed several bugs in libpuffs (related to librefuse). Librefuse works fine now. Started to work on ntfs-3g. Did additional tests in NetBSD: just noticed they link with libfuse, libperfuse and libpuffs (and not libpuffs + librefuse). Though everything still works when linked with librefuse, so hopefully we didn't get into trouble… ntfs-3g compiles almost from scratch. I stalled on a deadlock() between fs and vfs (reading from device). Week 10 (Jul 25 - Jul 31) ntfs-3g package is available. Though it doesn't work with current VFS and there are problems with preliminary version of AVFS. Also this week I did some fixes in libpuffs related to ntfs-3g and basic code cleanup. Code is pushed to the public repository. Week 11 (Aug 1 - Aug 7) Failed to make AVFS preview working with libpuffs. There're some issues with no obvious workarounds. Cache work is frozen until we finish discussing some new drawbacks. Week 12 (Aug 8 - Aug 14) No changes. Cache is either postponed or will not be implemented (depends on what will be decided). Week 13 (Aug 15 - Aug 21) - FINAL** Started to test ntfs-3g on AVFS. Works with minor issue (creating new files), to be fixed next week. Unionfs (2 lines in the code) requires few small MINIX changes, after that only Makefile has to be modified. Note that “FINAL” statement. It's about status reports only ===== Design ===== ==== FUSE overview ==== FUSE is a filesystem in userspace, thus it’s a natural fit for MINIX 3. It consists of * Kernel module (mainly VFS and page cache integration): fuse in linux or puffs in NetBSD. * Messaging interface used in communications between file server and kernel. * User level library: libfuse in Linux or libpuffs in NetBSD (includes librefuse to support fuse based filesystems). * User level filesystem implementation (file server). From developer’s point of view FUSE can be spitted into two logical parts: * A set of callbacks provided by user level fileserver to handle requests from the kernel. * A set of control flow routines (like fuse_main() from libfuse which takes full control over filesystem server). There are two callback interfaces: * A standard (high-level). It’s close to system calls and most file servers implement this interface. It mostly operates with file paths rather than inodes. * A low-level interface which is close to the kernel’s VFS. Just few file servers implement this interface. Libfuse has multi-thread support, but can work without threads (“single thread” mode). Developers of libpuffs avoid threads  and use continuations which has some advantages over threads (it’s much like threads but with explicit scheduling points). ====== NetBSD ====== It consists of * Kernel module (puffs). * libpuffs which is significantly different from FUSE and according its description is more flexible. Also there are some filesystems implemented on top of libpuffs, e.g. psshfs (puffs sshfs), dtfs (fully functional filesystem), puffs nullfs, sysctl. * librefuse on top of libpuffs as translation layer between FUSE file server and libpuffs. librefuse implements puffs callback interface. It supports FUSE high-level interface and uses it in its implementation. Librefuse has support for all revisions of FUSE (special translation layer with runtime decisions), thus fileservers written for both FUSE 24 and FUSE 26 can be compiled with librefuse. According to the experience of NetBSD folks there’re no common changes in FUSE filesystem servers to work with librefuse. Control flow is implemented by libpuffs. It supports both single threaded model (in this case it uses continuations) and multiple threads. librefuse initializes puffs callbacks structure with own functions (they use FUSE opts provided by fileserver) and then everything is handled by libpuffs code. ==== MINIX implementation discussion (design) ==== Most part of FUSE project just provides infrastructure to create an interface to the user-level filesystem server. It’s available in minix for free! Experiment with hello_fuse fileserver shows that FUSE file servers can be compiled with some minix library (I call it a glue server or a glue library) to be executed as MINIX 3 file servers. Current VFS-FS protocol fits very well, but probably minor changes (additions) will be required. Main differences between fuse file server (the glue library) and MFS/ext2: * Inode→path conversion will be required, since most part of FUSE API is about paths and VFS in MINIX operates on inodes. Dictionary or some f(inode) will do its job. * More internal function calls (e.g. stat() in fs_lookup()). * Probably some extra care when check for enter/leave mount during path parsing. * Support for different versions of FUSE API. * With single thread there is a chance of deadlocking. According to  umount call done from fileserver can trigger some handlers of the same fileserver which is already blocked on this syscall. I feel that this design is much better than “VFS →Own FUSE server→fileserver”. A good candidate to solve problems listed above is libpuffs + librefuse. We can port (partly reimplement) libpuffs, and use librefuse + filesystem servers without any modifications (or at least with very minor modifications). It has both advantages and disadvantages. Advantages: * Continuations and threads support. * No fear about deadlocks (according to paper describing puffs, solution for single thread model is not clean, but it works and nothing better can be provided). * We will not maintain compatibility with FUSE API (and different versions of this API). * Integration with pkgsrc: librefuse, fuse-afpfs-ng, fuse-archivemount, fuse-bindfs, fuse-cddfs, fuse-chironfs, fuse-cryptofs, fuse-curlftpfs, fuse-djmount, fuse-encfs, fuse-gmailfs, fuse-gphotofs, fuse-gstfs, fuse-httpfs, fuse-loggedfs, fuse-lzofs, fuse-mp3fs, fuse-ntfs-3g, fuse-obexfs, fuse-pod, fuse-svnfs, fuse-unionfs, fuse-wdfs, fuse-wikipediafs. * BSD license. Disadvantages: * Translation layer (librefuse) is not just pretty abstraction, but also a CPU (mainly) and memory consumer (shouldn’t be much). For almost every VFS operation librefuse tranlates data from the FUSE format to the PUFFS format. It’s significant for operations with metadata, but not for reading/writing, when we operate on buffers and possibly just on some small control structures. Libpuffs, librefuse and FUSE fileservers are written in C99, which is unsupported by ACK. Thus we will have to use gcc or clang (if it’s available) to have less (or no) changes in their code. ===== Schedule ===== Below is an approximate schedule, especially the part after first deadline. ==== Pre-Coding Period (Apr 25 - May 22) ==== learn puffs code and understand what can be ported and what should be reimplemented (now I can recall just obvious part: communication with kernel → comminication with MINIX’ VFS). If in time, start coding. ==== Week 1 (May 23 - May 29) ==== * Fix ucontext (required by CC framework) issue. * Port continuations (CC - call context) framework. Do some handy tests. * Port (minor changes) framebuf framework. ==== Week 2 (May 30 - Jun 5) ==== * Make everything else compiling. * Work on VFS-FS-PUFFS stuff. ==== Week 3 (Jun 6 - Jun 12) ==== Work on VFS-FS-PUFFS stuff. ==== Week 4 (Jun 13 - Jun 19) ==== Work on VFS-FS-PUFFS stuff. ==== Week 5 (Jun 20 - Jun 26) ==== Try libpuffs using pnullfs and work on issues. ==== Week 6 (Jun 27 - Jul 3) ==== Port librefuse. pkgsrc for libpuffs and librefuse. ==== Week 7 (Jul 4 - Jul 10) - MIDTERM ==== Reserved for issues. At this point we should have almost fully functional libpuffs and librefuse. To check implementation I’ll use hello_world and fusexmp fileservers. ==== Week 8 (Jul 11 - Jul 17) ==== Test libpuffs and fix remaining bugs. Test librefuse. ==== Week 9 (Jul 18 - Jul 24) ==== fuse-ntfs-3g ==== Week 10 (Jul 25 - Jul 31) ==== fuse-ntfs-3g ==== Week 11 (Aug 1 - Aug 7) ==== Cache for libpuffs. ==== Week 12 (Aug 8 - Aug 14) ==== Cache for libpuffs. ==== Week 13 (Aug 15 - Aug 21) - FINAL ==== Yet another “issues” week or fuse-unionfs and fuse-cddfs. libpuffs is properly working, fuse file servers can be used almost without modifications. fuse-ntfs-3g is well tested. ===== Resources ===== *  "puffs - Pass-to-Userspace Framework File System" presented at AsiaBSDCon 2007. *  "ReFUSE: Userspace FUSE Reimplementation Using puffs" presented at EuroBSDCon 2007.