Using fossil and venti as auxiliary storage on plan9.

For those that are not familiar, venti and fossil are disk file systems for the plan9 operating system. Venti acts as append only storage of data blocks, and fossil is a local copy of these blocks represented as a traditional file system. The most notable advantages to these systems is that venti has aggressive dedup and is append only. Data is addressed by its sha1 hash, and can never be deleted once stored. Fossil can sync data back to venti, generating a 'vac', this hash is a key to fossils 'window' in to the venti data, allowing it to construct local filesystem trees out of it. This means that essentially, venti archives all of the actual storage of the files, and vacs act as keys in to snapshots of that data that fossil can work with. Perhaps the most interesting part about fossil is that it is 'lazy loaded', meaning that data is pulled from venti only when absolutely necessary, making for the ability to create very small fossils for viewing previous vacs. Recently, I had thoughts this system would sound great for an auxiliary addition to my current plan9 setup. My thoughts were to store source code of mine and others(really just my $home/src) in to it, to both test and see if the benefits were worth. This post will kind of walk through my processes in creating configuration. This was done on 9ants5, but could be done an a normal 9front release if desired.

First though some history and thoughts on fossil and venti themselves. When used for a rootfs, it allows for multiple fossil installs based around one central venti. Since the information is dedup'd one fossil and 50 fossils use the same space on the venti server. This works well with the traditional idea of 9 being a 'grid' system, making the addition of new machines pratically free. The other great part is the snapshot capabilities, the snapshots themselves are ephimeral, and you can quite easily create a new fossil pointing to an existing snapshot. This would allow you, for example, to spin up a new machine/VM with a snapshot of last months code to test for comparisons. Because the whole system is snapshotted, you can tests with the context of the entire system. This is in contract to the idea of using modern VCS to manage individual projects, where it can be hard to truly rewind infrastructure to a specific point in time. Fossil even allows for the ability of trees, if you want to test a slight deviation you can use a base vac and create sperate branches of modifications, each getting their own new vac. With some simple tooling, this could be used to some very efficent version control systems(admitably, this currently does not exist to my knowledge). As for history, fossil was the original plan9 file system when it was first open sourced by Bell Labs. However, a nasty bug plauged it until 2012, leading it to give a bad impersonation, and eventual migration away from it. Since then a lot of experimentation and support has been done by the creator of 9ants, mycroftiv. One notable project is his 'spawngrid' system, which leverages ramfossils for on the demand containers.

For getting fossil+venti up and running pop open your disk(or file) in disk/edisk and create one large "plan9" partition. Then we need to divy this partition up in to our two venti partitions. disk/prep -a arenas -a isect /dev/sdE0/plan9 should do just fine. Next we need to write a config file for venti. The following should do:

; cat >/tmp/venti.conf <<. index main arenas /dev/sdE0/areans isect /dev/sdE0/isect mem 32m bcmem 48m icmem 64m httpaddr tcp!*!8000 addr tcp!*!17034 .

This points venti to our partitions and tells it to start listening and accept connections from any origin. Now to actually format the partitions and write our config to venti.

; venti/fmtarenas arenas /dev/sdE0/areans ; venti/fmtisect isect /dev/sdE0/isect ; venti/fmtindex /tmp/venti.conf ; venti/conf -w /dev/sdE0/areans /tmp/venti.conf

We should be all ready to go, let's start it up with
; venti/venti -c /tmp/venti.conf.
You can verify by looking at http://$host:8000/storage. Which should give you a summary of the available storage.

Now on to the more fun part, let's make a fossil for this venti. Since we don't really plan to have much data on this, we can use a file through ramfs to get started.

; ramfs ; dd -if /dev/zero -of /tmp/fossil -count 200000 #create a 200M file ; cat >/tmp/initfossil.conf <<. fsys main config /tmp/fossil fsys main open -AWPV fsys main create /active/adm adm sys d775 create /active/adm/users adm sys 664 users -w uname upas :upas uname adm +glenda uname upas +glenda srv -p fscons.newfs srv -A fossil.newfs . ; venti=$ventihost ; fossil/flfmt /tmp/fossil ; fossil/conf -w /tmp/fossil /tmp/initfossil.conf ; fossil/fossil -f /tmp/fossil

Now that it is up and running, lets connect to the fs console and do some additional configuring and take a snap shot.

; con /srv/fsons #ctl-\ + q quits fsys main uname $user :$user #add our user to fossil create /active/src $user $user d775 #add a directory for our user to write to sync # sync all of our changes snap # write to venti, when it is done you should get a vac # quit out of con # clean up ; kill fossil | rc ; unmount /tmp

Now you should be able to take advantage of the 9ants ramfossil script to access the data stored, just keep track of the vac, and remember that if you want changes to persist, you need to snap the filesystem and note the new vac. If you don't want to keep setting the $venti environment variable, add it as a value for your system under /lib/ndb/local just like $auth. You could add a script to your $home/lib/profile to test for /srv/ramfossil and start ramfossil if needed and then bind it over $home/src(what I did). You may also want to change the ramdisk fossil size by editing the ramdisk script itself, I found 1G to be fairly reasonable. If the fossil ever fills up, simply snap it to venti and start a new one from the newly generated vac. The lazy loading aspect should keep you covered for actually/writing 1G in one session.

For the sake of completeness, I will also walk through the processes of creating a persistent fossil filesystem. Start by using disk/edisk like before then

; ramfs ; cat >/tmp/fossil.conf <<. fsys main config /dev/sdG0/fossil fsys main open -c 3000 fsys main snaptime -a 0500 srv -p fscons srv -A fossil . ; disk/prep -a fossil /dev/sdG0/plan9 ; fossil/flfmt /dev/sdG0/fossil ; fossil/conf -w /dev/sdG0/fossil /tmp/fossil.conf ; venti=$myventi fossil/fossil -f /dev/sdG0/fossil ; unmount /tmp

This will set the fossil to snapshot at 5am to the venti, 9ants' fshalt scripts should handle snycing it when the system goes down(note it will not snapshot). Huge thanks to mycroftiv and his 9ants scripts for helping me figure my way through the install, setup processes, and keeping the fossil and venti system alive and usable for a standard user.