P2P Research Institute Blog

Download MP3 audio of interview here.

A couple of weeks ago I interviewed Michael Stapelberg, head developer of the DustFS project. You can listen to a recording of this interview here. DustFS is an encrypted, distributed file system based on the BitTorrent protocol. DustFS’s major features are full encryption, authentication using certificates and usage of BitTorrent for fast distribution of files (accelerated by using so-called cache servers).

Michael is quite active in the Open Source community having contributed to a number of projects such as Enlightenment (e17), collectd and Bacula. Apart from the above Michael is also the author of mxallowd an anti-spam daemon for Linux/*BSD and is about to enter a Computer Science program at the University of Heidelberg in southern Germany.

According to Michael, the idea for DustFS came from a local Chaos Computer Club (CCC) meeting. In Germany, most of the DSL providers offer high downstream but low upstream bandwidth Internet connections. If you are a DSL customer in Germany and you want to be able to conveniently share large files with your friends, your options are essentially limited to the existing, well-known protocols such as FTP, NFS, Samba and AFS. The problem however is that all of these protocols require a central server - this factor combined with the limited upstream bandwidth in Germany conspire to make this sub-optimal for distributing large files amongst a group of friends. A distributed, P2P protocol has the advantage that one can download simultaneously from multiple peers, therefore effectively combining their upstream bandwidth, rather than being limited by the pipe of the central server - resulting in greatly improved download speed. Another issue to note is that most DSL connections in Germany do not have a static IPv4 address, therefore one must additionally deal with the problem of co-ordinating server addresses with your friends. P2P networks, on the other hand, are inherently distributed and self-configuring.

One solution to the above problem would be to build a torrent file for the data you want to share, set up a BitTorrent tracker and attempt to keep it private - however this is a lot of manual set up overhead and would be quite annoying to do every time you wanted to share a single file. The main aim of DustFS is to make sharing a file among a small group of friends in a secure and trustworthy fashion easy and painless.

As I mentioned already, DustFS is an encrypted FUSE file system built on top of the P2P Research Institute’s BitTorrent implementation Unworkable. Originally DustFS used Libtransmission as its BiTorrent implementation. However despite it’s name, Libtransmission is not in fact just a library.  Libtransmission provides a number of features not required by DustFS and at 17,000 lines of source code is quite large. Unworkable, on the other hand, is closer to 4,000 lines of code. To illustrate the difference this makes, the DustFS binary built with Libtransmission is 1.7 megabytes in size, as opposed to 700 kilobytes when built with Unworkable.

In Germany it is also quite cheap to rent servers which have reasonable upstream bandwidth allowances, a fact that DustFS plans to leverage to accelerate transfers. These rented servers could be used as intermediate cache servers for the DustFS network. The idea is that when you transfer a file over DustFS, a cache server kicks in and mirrors this file. If yet another user requests the same file, they download using the copy stored on the cache server - at a vastly greater rate then from a DSL peer. A cache server is stripped down DustFS node.  A DustFS cache node doesn’t need FUSE, and generally has a lower set of dependencies than a normal node.

While DustFS is still in early stages of development, it looks extremely promising. We think that Michael has struck on a very good idea. If his project can gain enough momentum to solve the non-trivial engineering problems, DustFS could revolutionise file-sharing among groups of friends. Imagine a world where everyone has an embedded, DustFS-capable device in their homes. Family videos, photos and so on would be easily shareable between trusted parties in a secure manner. Musicians could collaborate on tracks effortlessly without worry of their intellectual property being stolen. The potential is huge. We wish Michael and DustFS the best of luck and look forward to collaborating with him in the future.