Unikernel Devel

Problem setting up F27 and halvm-xen


Just in case anyone’s run into this and knows the fix -

Has anyone followed the dev env setup instructions and hit this error on reboot?

error, file (grub2_path)/module2.mod not found

Not the right words, I tried a few things from bugzilla.redhat.com id=1486002, and lost the message.

I followed the Development Environment Setup (HACKING.md). I installed Fedora 27 workstation on Virtualbox 5.2.8, which is on MacOS 10.13.

It fell over on reboot, with the ‘module2.mod not found’ error.

Blatant guesswork and tweaking /etc/grub.d/20_linux_xen got me nowhere.

Commands executed were:

repo, system update

sudo dnf install http://repos.halvm.org/fedora-27/x86_64/halvm-yum-repo-27-3.fc27.noarch.rpm
sudo dnf update

remove conflict, file X from install of halvm-xen-libs-Y conflicts with file from package xen-libs-Z

sudo dnf remove xen-libs xen-licenses


sudo dnf install gcc gcc-c++ automake libtool patch ncurses-devel halvm-xen halvm-xen-devel zlib-devel openssl-devel git
sudo dnf install libgmp # for Fedora 22
sudo dnf install gmp-devel # for Fedora 23 and newer
sudo ln -s /usr/lib64/libtinfo.so.6 /usr/lib64/libtinfo.so.5 # for Fedora 24 and newer. It is a HACK


sudo dnf install ghc cabal-install alex happy hscolour
sudo dnf install ghc-random-devel ghc-mtl-devel ghc-zlib-devel ghc-tar-devel ghc-ed25519-devel ghc-base64-bytestring-devel ghc-cryptohash-sha256-devel ghc-base16-bytestring-devel ghc-stm-devel ghc-text-devel ghc-network-devel


vi /etc/default/grub

GRUB_CMDLINE_XEN=“console_to_ring loglvl=all guest_loglvl=all”
GRUB_DEFAULT=‘Fedora, with Xen hypervisor’

sudo grub2-mkconfig -o /boot/grub2/grub.cfg


Hello! This one’s a new one on me. I’ve never seen that exact message before. That being said, these sorts of weird boot problems tend to happen when there’s drift between the version of Xen that I produce and the version of Xen that Fedora releases. (Basically, there’s some interaction between Fedora’s Linux kernel and the Xen installation, and when they change something in the Linux kernel, a stale version of Xen becomes a problem.)

I’ve pushed up a version of halvm-xen that should be based on the most recent version of Fedora’s Xen. See if that helps. I’m also setting up a Fedora VM in another window, to make sure that it works on a clean install, but that’ll take me a bit.


Well, huh. Even with the update, I see that message. What’s interesting, though, is that if you just let the system continue to boot, it works fine. I can log in normally, and sudo xl list seems to work fine.


Got as far as make, which is churning away.

Good instructions. Two tiny issues.
vim isn’t installed by default on F27. I ran
sudo dnf install vim
typo here - Should be sysconfig
sudo vim /etc/sysconf/network-scripts/ifcfg-enXXXX

More commands

# xen test 
sudo reboot
sudo xl list
sudo xl dmesg

# network interfaces
sudo systemctl status network # check before enabling oldskool network
sudo systemctl enable network
sudo systemctl status network # after, no difference
ip a # before adding IF
sudo dnf install vim
sudo vim /etc/sysconfig/network-scripts/ifcfg-xenbr0
sudo vim /etc/sysconfig/network-scripts/ifcfg-enp0s3 # typo in doc
ping -c3 # check before changing IFs
sudo systemctl restart network # pregnant pause
ping -c3

# Build HaLVM!
git clone https://github.com/GaloisInc/HaLVM
cd HaLVM
git submodule update --init --recursive # check out a truckload of files
./configure --enable-gmp
make                     # the VirtualBox VM is torturing a CPU core for an hour
sudo make install
halvm-ghc-pkg recache


Got closer. Fell at the last hurdle - halvm-ghc-pkg recache

[nick@localhost HaLVM]$ halvm-ghc-pkg recache
bash: halvm-ghc-pkg: command not found...
[nick@localhost HaLVM]$

There is a copy in /home/nick/HaLVM/src/scripts/halvm-ghc-pkg but nothing in /usr/local/bin/
I can’t see anything in https://github.com/GaloisInc/HaLVM/issues (looks a bit like a work tracker for HaLVM3).
I see 88 warnings, 5 errors, and one SElinux ‘computer says no’ denial.

Would you like the transcript of the install process? I can’t attach the transcript - no text files allowed. It’s 8000 lines of fun, so I can’t paste it either.

Thanks, Nick

more details

3 errors

checking whether bootstrap compiler is affected by bug 9439... 
<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM 3.7
ghc: could not execute: /usr/bin/opt-3.7
failed to compile

XenStore.hsc:418:16: error: ‘XS_RESTRICT’ undeclared (first use in this function); did you mean ‘XS_RESUME’?

XenStore.hsc:418:16: error: ‘XS_RESTRICT’ undeclared (first use in this function); did you mean ‘XS_RESUME’?

1 SElinux deny
I don’t know what ostnamed is.

[nick@localhost ~]$ sudo ausearch -m avc -ts today -c ostnamed
time->Thu Apr 19 00:08:44 2018
type=AVC msg=audit(1524092924.648:336): avc:  denied  { remount } for  pid=2684 comm="(ostnamed)" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:xenstored_var_lib_t:s0 tclass=filesystem permissive=0
[nick@localhost ~]$ 

[nick@localhost ~]$ sudo aureport -f -i

File Report
# date time file syscall success exe auid event
1. 18/04/18 23:49:18 /run/xenstored/socket ? no ? unknown(-2) 113
[nick@localhost ~]$


I can’t debug this, so it’s not reasonable to ask anyone else to.

Is there a known good combination? Or a workaround?