1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
|
vmm
===
A QEMU/KVM virtual machine manager in one POSIX shell script. Each VM is
a directory of plain files with a KEY=VALUE config edited by hand.
cp vmm /usr/local/bin/vmm
vmm the verbs, their arguments, the environment
Needs qemu-img and qemu-system-x86_64 or qemu-system-aarch64, from QEMU
6.0 or newer, Linux /proc, the POSIX tools, and flock, readlink, setsid,
timeout, grep -m1 and du -h, which are not POSIX. Alpine BusyBox and
Debian base tools provide them. Native acceleration needs /dev/kvm;
HWACCEL=no uses emulation instead. `vmm viewer` needs remote-viewer, or
whatever client $VIEWER names.
Usage
-----
vmm create web 20G writes the config, opens $EDITOR
vmm start web
vmm console web Ctrl-] detaches
vmm stop web
Install an OS by pointing CDROM at an ISO and booting it first:
CDROM="/srv/iso/alpine-virt-x86_64.iso"
BOOT_ORDER=dc
The CD is a SCSI device, so an installer with no virtio drivers will not
see it. Drop both lines once the disk is bootable.
A GRAPHICS=vnc or spice guest puts its display on a unix socket in the VM
directory, and `vmm viewer web` hands that socket to remote-viewer. The
path goes into a URI unencoded, so a VMMDIR holding a space, #, ? or %
may not survive the client's parser. The client runs wherever vmm runs,
so from another machine forward the socket and point a viewer at the
forward instead:
ssh -N -L 5900:/home/you/.vm/web/vnc.sock vmhost &
remote-viewer vnc://127.0.0.1:5900
Every VM owns `$VMMDIR/<name>/disk.qcow2`; its path is not configurable.
To run many VMs from one immutable golden image, replace each new disk
with an overlay. The overlay takes its size from the backing file:
rm ~/.vm/web/disk.qcow2
qemu-img create -f qcow2 -b /srv/img/alpine.qcow2 -F qcow2 \
~/.vm/web/disk.qcow2
Configuration
-------------
One KEY=VALUE per line. A `#` in the first non-blank column starts a
comment; a line carrying a value cannot also carry one. Values may be
quoted to protect leading or trailing blanks. Unknown keys, duplicates,
unbalanced quotes and non-printable bytes are errors naming the line.
Nothing here is sourced or eval'd.
ARCH x86_64 | aarch64 default host
HWACCEL yes | no default yes
CPUS positive integer default 2
MEM positive integer, MiB default 2048
IMAGE_FORMAT qcow2 | raw default qcow2
FIRMWARE absolute path aarch64 needs it
GRAPHICS no | vnc | spice default no
NETWORK yes | no | hostonly default yes
SNAPSHOT yes | no default no
BOOT_ORDER 1-3 distinct of c(disk) d(cdrom) n(net) default c
BALLOON yes | no default yes
CDROM absolute path optional
HOSTFWD comma list of HOST:GUEST or ADDR:HOST:GUEST optional
- BOOT_ORDER is a restriction, not a preference: QEMU runs with
-boot strict=on, so a device left out of the list never boots. A
list naming no device the config creates would restrict nothing,
and is refused.
- SNAPSHOT=yes runs the guest on an overlay that is unlinked as soon
as QEMU holds it open, so guest writes have nowhere to survive.
- GRAPHICS adds a display and USB tablet, reached with `vmm viewer`.
vnc is built into QEMU, spice is a separate module. Clipboard
sharing and guest agents are out of scope. Stop the VM before
changing graphics.
- NETWORK=hostonly isolates the guest from the host and the internet.
QEMU's own DHCP and DNS still answer, and HOSTFWD still reaches in.
- HOSTFWD binds 127.0.0.1 unless the entry names an IPv4 address of
its own, so a two field entry is unreachable from another machine.
- BALLOON=yes lets the guest return freed pages to the host as it
frees them. There is no ballooning policy to set.
- Only the host's own ARCH can use KVM. Anything else is emulated, and
aarch64 has no built-in firmware to emulate without.
- IMAGE_FORMAT describes the fixed disk. `vmm create` makes qcow2; set
raw only after deliberately replacing that file with a raw image.
State
-----
One directory per VM, mode 0700, under $VMMDIR:
config KEY=VALUE, hand-edited configuration
uuid identity, feeds -uuid and the NIC MAC
disk.qcow2 created by `vmm create`
pid written and locked by QEMU itself, never by vmm
qmp.in qmp.out FIFOs carrying QMP to and from QEMU
stdout stderr QEMU's own two streams, truncated at start
console.log guest serial, truncated at start, attached or not
vnc.sock only when GRAPHICS=vnc
spice.sock only when GRAPHICS=spice
The pidfile is authoritative. Do not remove it while QEMU is running. A
VM reading as unknown has a pidfile naming a live process vmm cannot
identify, usually a pid something else took after a crash. Nothing that
would touch a running guest acts on that VM until the pidfile is removed
by hand.
status and list report the QEMU process, not the guest. A guest that
panics is paused rather than killed, so it still reads as running; ask
`vmm monitor <name> info status` for the guest's own view.
Nothing caps console.log. It is truncated when the VM starts and appended
to for as long as the VM runs, so a guest that streams to its serial port
fills the filesystem it lives on.
The MAC is derived from the uuid, so a VM keeps its DHCP lease across
restarts and a clone never collides with the VM it came from.
create and clone write config last. A failure may leave an incomplete
directory; inspect and remove it manually before retrying.
Security
--------
The VM directory is the only authentication. Whoever can write qmp.in
owns the monitor, and the monitor can attach a block device pointing at
any file QEMU can read. QEMU runs under its seccomp sandbox with
spawning denied, so the monitor cannot start a process on the host.
Nothing is ever bound to a TCP port for control.
Disk images are trusted input: a qcow2 names its backing file in its own
header and QEMU opens it. Run `qemu-img info --backing-chain` on an image
from anywhere else. A backing file must be immutable while its overlays
exist. Never back one VM with another VM's writable disk.
Exit codes
----------
0 success, including every idempotent no-op
1 usage, config, dependency, or operation failure
2 no such VM
3 VM is not running, or its monitor did not answer
4 VM is busy, locked, or has live state vmm cannot identify
5 VM was stopped only by force
`vmm stop` returning 0 means the guest shut itself down. 5 means it
ignored ACPI and was killed.
Debugging
---------
vmm dryrun web what would run, and whether the config parses
vmm logs web what QEMU said, what the guest printed
vmm monitor web info status ask the running guest
tail -f ~/.vm/web/console.log
dryrun launches nothing and needs no QEMU installed, so it doubles as a
config checker anywhere. Errors name the file and the line:
vmm: /home/you/.vm/web/config:7: unknown key: MEMROY
A VM that will not start prints the last of QEMU's stderr. vmm removes
runtime files only after proving QEMU is gone. A busy VM has another vmm
invocation holding its lock; the kernel releases it when that process exits.
An incomplete VM directory is deliberately left for manual inspection.
Testing
-------
./test every file in tests/
./test config argv those two only
TAP, no framework. ./test makes a throwaway VMMDIR, defines the
assertions, and sources each file in tests/ in turn. Tests that boot a
guest on KVM skip without /dev/kvm; the emulated ones need only the
matching qemu-system binary, and its firmware for aarch64.
|