microvm

Thoughts on creating VMMs from Docker images

Dockerfiles are awesome There is so much software out there packaged as Docker images. Operating systems, SQL and NoSQL databases, reverse proxies, compilers, everything. Safe to say, most of that software available as Docker containers is built from the common file format - the Dockerfile. Dockerfiles are awesome. They are recipes for getting a bit of software functional. how have I been building Firecracker VMMs so far So far, all of my VMMs were built from Docker images using the following steps:
Read more

The jailer

A Firecracker release comes with two binaries - the firecracker and the jailer programs. The jailer brings even more isolation options to Firecracker by creating and securing a unique execution environment for each VMM. what can it do check the uniqueness and validity of the VMM id, maximum length of 64 characters, alphanumeric only assign NUMA node check the existence of the exec_file run the VMM as a specific user / group assign cgroups assign the VMM into a dedicated network namespace a VMM can be damonized what does it do This part comes from the jailer documentation1.
Read more

It's all about the the Iface name

Last night’s problem with the second VMM conflicting on the network layer with the first one was indeed the veth0 name hard coded in firectl. I’ve added the --veth-iface-name argument to firectl and I am now able to start multiple VMMs on a single bridge. sudo firectl \ --firecracker-binary=/usr/bin/firecracker \ --kernel=/firecracker/kernels/vmlinux-v5.8 \ --root-drive=/firecracker/filesystems/alpine-base-root.ext4 \ --cni-network=alpine \ --socket-path=/tmp/alpine.sock \ --ncpus=1 \ --memory=128 \ --veth-iface-name=vethalpine1 sudo firectl \ --firecracker-binary=/usr/bin/firecracker \ --kernel=/firecracker/kernels/vmlinux-v5.
Read more

Bridging the Firecracker network gap

Today I have looked at creating my own bridge networks for Firecracker VMMs. I already used CNI setups when evaluating the HashiCorp Nomad firecracker task driver1. Back then I incorrectly stated that Firecracker depends on certain CNI plugins. It doesn’t, it can take advantage of any CNI setup as long as the tc-redirect-tap is in the chained plugins. The Nomad task driver had some issues, briefly: every now and then, oddly, the task would never shut the VMM down and the only way to make the VMM gow down was to sudo kill nomad I tried updating the task driver to latest SDK version but I was not able to upgrade the Firecracker dependency past a specific commit, any version after that specific commit makes the VMM come up, the network setup to be there but the VMM is not reachable, really, really weird issue - reported it here So today, I took a different route.
Read more

Live resize Firecracker VMM drive

Towards the end of the Firecracker VMM with additional disks article1 I concluded that I didn’t know how to live resize an attached drive. It turns out it is possible and it’s very easy to do using the Firecracker VMM API. To launch the VMM with the API, I have to drop the --no-api argument (obviously) and use --api-sock with the path to the socket file. In a production system, I’d use a directory other than /tmp.
Read more

Firecracker VMM with additional disks

Before looking at the networking options, I have looked at adding extra drives to my Firecracker VMMs. Storing data on the root file system will not scale well long term. Additional disks will be a good solution to persist application specific data across reboots and upgrades. Create the disk on the host First, create an additional file system on the host: dd if=/dev/zero of="/firecracker/filesystems/alpine-vol2.ext4" bs=1M count=500 mkfs.ext4 "/firecracker/filesystems/alpine-vol2.ext4" Reconfigure the VMM Change the VMM drives configuration:
Read more

Launching Alpine Linux on Firecracker like a boss

The quest to launch an ETCD cluster on Firecracker starts here. In this post, I’m describing how I’ve built my initial Alpine 3.13 VMM with OpenSSH and a dedicated sudoer user. In AWS, when one launches a Ubuntu instance, one can access it via ssh ubuntu@<address>, a CentOS VM is ssh centos@<address>. At the end of this write up, I’ll have ssh alpine@<address>. This VMM will have access to the outside world so I can install additional software and even ping the BBC!
Read more

Vault on Firecracker with CNI plugins and Nomad

It’s good to know how to set up Firecracker VM by hand but that’s definitely suboptimal long term. So today I am looking at setting up Firecracker with CNI plugins. Firecracker needs four CNI plugins to operate: ptp, firewall, host-local and tc-redirect-tap. First three come from the CNI plugins1 repository, the last one comes from AWS Labs tc-redirect-tap2 repository. Golang CNI plugins and tc-redirect-tap require golang to build. I’m using 1.
Read more

Taking Firecracker for a spin

Firecracker1 is recently making rounds on the internet as this relatively new, awesome technology for running lightweight VMs. As something coming from AWS and powering AWS Lambda, my original perception was that it’s not easy to set up and use. However, this write from Julia Evans2 proved me wrong. So, as I have recently picked up a used Dell R720 with decent amount of RAM and CPUs, it was time to take these two for a spin together.
Read more