IncludeOS Filesystem API


#1

The new filesystem API is merged in dev now. Very early work, that is missing functionality like choosing extended partitions for mounting filesystems and random access file operations. The API is built to be read-only, supporting immutable infrastructure. VirtioBLK is not added yet, but the driver (currently memdisk) is replaceable. Memdisk is just an image that you attach to the service image via a binary section in the ELF header.

See: test_disk.cpp


#2

Nice Gonzo! Looking forward to trying it out :+1: One question; when you use the memdisk you wouldn’t really need the asynch callback-based workflow - but do you think the convenience of a classic, sequential interface, for memdisk only would be worth it?


#3

I guess so, but then you lose the ability to choose an arbitrary driver without changing anything else. Pros and cons.


#4

Right. And I guess callbacks will be immediate for memdisk. Let’s make a couple of tests to see exactly what the consequences of that will be, in terms of i.e. stack frames / exceptions.


#5

Memdisk is just an image that you attach to the service image via a binary section in the ELF header.

Useful! MirageOS’s crunch tool could also use the same ELF convention. Right now it either links in statically or requires a dynamic block device, so this is a good middle ground. I’ve opened a bug to track it in MirageOS as well.


#6

It does take a fair bit of time to link the whole thing together because of disk sizes, however, if the memdisk is just a FAT16 system of say 4-16mb then it should be fine. If its 100mb it will take a few seconds each time the service image is linked.

I am for the moment done with the API and some testing, and I will write some of my findings soon-ish.


#7

Filesystem, memdisk and IDE is merged into master now. It turns out our 28-bit PIO in bootloader sets a maximum size for how big the sections we can load are. So memdisk can’t support “big” images, and that is OK, but I had to fix up our IDE driver to support selecting slave for our images. Now we have FAT16, FAT32 (and FAT12 kinda works, I just havent really tested it, sorry!). There are decent tests for all relevant modules right now. The filesystem is read-only, intended primarily for immutable infrastructure and microservices.

This is what it looks like in the tests:
fat16.cpp
fat32.cpp

It’s the classic lambda spam due to async programming. Or I’ll have to stop myself from going too depth with the lambdas and go back to using functions with real names occasionally.

The filesystem is auto-mounted if no partition is specified, and from there the 3 (!) filesystem functions are available: ls, stat and read. They all come in variants of sync and async, so I guess in total there’s 6 commands. Now, I can’t think of any other filesystem commands to implement. Honest. So, did I miss any?