Tools used by the Ubuntu QA Team

Merge ~jslarraz/ubuntu-qa-tools:uvt-snap into ubuntu-qa-tools:master

  1. Git
  2. lp:~jslarraz/ubuntu-qa-tools
  3. uvt-snap
  4. Merge into master
Proposed by Jorge Sancho Larraz
Status: Merged
Merged at revision: 6e82bbb0aa88ac53330a9823924e22a4a5c7836e
Proposed branch: ~jslarraz/ubuntu-qa-tools:uvt-snap
Merge into: ubuntu-qa-tools:master
Diff against target: 256 lines (+96/-26)
1 file modified
vm-tools/uvt (+96/-26)
Reviewer Review Type Date Requested Status
Marc Deslauriers Approve
Review via email: mp+462951@code.launchpad.net

Commit message

Add support for uvt snap

Description of the change

Given the limited set of changes needed to properly support uvt in a snap format I wonder if it will be possible to include those changes in the master branch to make it easier to maintain (in contrast of rebasing onto master for new commits)

To post a comment you must log in.
Revision history for this message
Jorge Sancho Larraz (jslarraz) :
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

I really don't want uvt to be available in the snap store. It's not meant for general use, it does a whole bunch of things in an insecure fashion and hardcodes a bunch of stuff in the virtual machines it creates.

I don't want to get bug reports about it. I don't want to get CVEs assigned to it. I don't want this to be used outside of the security team.

That being said, if you want to upload it as a private snap, that's fine, and I will accept some of the changes based on the env variables, but I don't want the snap packaging itself to be included here as I don't want others to upload it either.

Revision history for this message
Marc Deslauriers (mdeslaur) :
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

So, adding the uvt ssh command is fine. And you can add the ssh known host and 'StrictHostKeyChecking=accept-new' changes by default. The gpg --homedir option by default is fine too, if it works properly with the updated $HOME.

The rest can be snap-specific behind is_snap()

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Does gpg --lock-never work?

Revision history for this message
Marc Deslauriers (mdeslaur) :
Revision history for this message
Jorge Sancho Larraz (jslarraz) wrote :

Updated according to our previous discussion

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

ACK, LGTM, thanks!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/vm-tools/uvt b/vm-tools/uvt
2index 0702939..2f6eca3 100755
3--- a/vm-tools/uvt
4+++ b/vm-tools/uvt
5@@ -468,6 +468,58 @@ def cmd_cmd():
6 print("Error: VM '%s' command failed. Aborting." % machine, file=sys.stderr)
7 sys.exit(1)
8
9+def cmd_ssh():
10+ '''Run a command inside a virtual machine'''
11+
12+ usage = "usage: %prog ssh [options] <vm>"
13+
14+ epilog = "\n" + \
15+ "Eg:\n" + \
16+ "$ uvt ssh sec-jammy-amd64\n\n" + \
17+ "This will open an interactive session on the single VM named 'sec-jammy-amd64'\n"
18+
19+ optparse.OptionParser.format_epilog = lambda self, formatter: self.epilog
20+ parser = optparse.OptionParser(usage = usage, epilog = epilog)
21+
22+ parser.add_option("-s", "--start", dest="start", default=False, action='store_true',
23+ help="Start the VM (and shutdown if it wasn't running")
24+
25+ parser.add_option("-t", "--timeout", dest="timeout", default=90, metavar="TIMEOUT",
26+ help="wait TIMEOUT seconds for VM to come up if -s is used (default: %default)")
27+
28+ parser.add_option("-f", "--force-ssh", dest="force_ssh", default=False, action='store_true',
29+ help="force the SSH keys to be taken")
30+
31+ parser.add_option("-r", "--root", dest="root", default=False, action='store_true',
32+ help="login to the VM as root")
33+
34+ parser.add_option("-u", "--user", dest="user", default=None, metavar="USER",
35+ help="login to the VM as user")
36+
37+ parser.add_option("-q", "--quiet", dest="quiet", default=False, action='store_true',
38+ help="only report hostnames and output")
39+
40+ (opt, args) = parser.parse_args()
41+ machine = args[0]
42+
43+ if opt.user is not None and opt.root:
44+ print("Error: may specify only one of --root and --user.\n", file=sys.stderr)
45+ sys.exit(1)
46+
47+ print("----- %s -----" % machine)
48+ if check_vm_exists(machine) == False:
49+ print("Error: VM '%s' does not exist, skipping." % machine, file=sys.stderr)
50+ return
51+
52+ result = vm_run_command(machine, "bash", root=opt.root, start=opt.start,
53+ start_timeout=opt.timeout, force_keys=opt.force_ssh,
54+ quiet=opt.quiet, output=True, interactive=True,
55+ verbose=False, user=opt.user)
56+
57+ if result == False:
58+ print("Error: VM '%s' command failed. Aborting." % machine, file=sys.stderr)
59+ sys.exit(1)
60+
61 def cmd_repo():
62 '''Adds or removes a local repo to a VM'''
63
64@@ -1098,7 +1150,7 @@ def vm_update_packages(vm_name, autoremove=False, snapshot=True):
65 def vm_update_local_repo(vm_name, release, enable=True):
66 '''Enables or disables a local repo'''
67
68- if vm_ping(vm_name) == "":
69+ if not ssh_connect(vm_name):
70 return False
71
72 sources_file = "/etc/apt/sources.list.d/uvt-repo.list"
73@@ -1429,9 +1481,8 @@ def vm_run_command(vm_name, command, root=False, start=False,
74 print("Could not start VM: %s" % vm_name)
75 return False
76
77- dns_name = vm_ping(vm_name)
78- if dns_name == "":
79- print("Could not ping VM: %s" % vm_name)
80+ if not ssh_connect(vm_name):
81+ print("Could not connect VM: %s" % vm_name)
82 return False
83
84 # Wait a bit before doing the ssh
85@@ -1454,8 +1505,10 @@ def vm_run_command(vm_name, command, root=False, start=False,
86 ssh_command += ['-q']
87 ssh_command += ['-o', 'BatchMode=yes']
88 ssh_command += ['-i', uvt_conf['vm_ssh_key'].split(".pub")[0]]
89+ ssh_command += ['-o', 'StrictHostKeyChecking=accept-new']
90+ ssh_command += ['-o', 'UserKnownHostsFile=' + uvt_conf['vm_known_hosts']]
91
92- ssh_command += [dns_name, command]
93+ ssh_command += [vm_name, command]
94
95 if interactive:
96 rc, out = runcmd(ssh_command, stderr = None, stdout = None, stdin = None)
97@@ -1479,7 +1532,7 @@ def vm_run_command(vm_name, command, root=False, start=False,
98
99 def remove_ssh_keys(vm_name):
100 '''Removes SSH keys for a host'''
101- runcmd(["ssh-keygen", "-R", vm_name])
102+ rc, out = runcmd(["ssh-keygen", "-R", vm_name, "-f", uvt_conf['vm_known_hosts']])
103
104 def crypt_password(password):
105 '''Crypts a password using mkpasswd'''
106@@ -1500,7 +1553,6 @@ def vm_start(vm_name):
107 rc, out = runcmd(["virsh", "--connect", uvt_conf["vm_connect"],
108 "start", vm_name])
109
110-
111 def vm_destroy(vm_name):
112 '''Powers off a VM'''
113 rc, out = runcmd(["virsh", "--connect", uvt_conf["vm_connect"],
114@@ -1534,9 +1586,8 @@ def vm_stop(vm_name):
115 # a shutdown menu.
116
117 # If we can connect using ssh, issue a shutdown command
118- dns_host = vm_ping(vm_name)
119- if dns_host != "" and ssh_connect(dns_host):
120- vm_run_command(dns_host, "shutdown -h now", root=True,
121+ if ssh_connect(vm_name):
122+ vm_run_command(vm_name, "shutdown -h now", root=True,
123 force_keys=True, output=False, ignore_rc=True)
124 else:
125 rc, out = runcmd(["virsh", "--connect", uvt_conf["vm_connect"],
126@@ -1563,7 +1614,7 @@ def vm_start_wait(vm_name, timeout=1800, quiet=False, clone_name=None):
127
128 # Second step, make sure the ssh port is reachable
129 for second in range(timeout-first):
130- if vm_ping(vm_name) != "":
131+ if ssh_connect(vm_name):
132 if quiet == False:
133 sys.stdout.write("\n")
134 return True
135@@ -1576,13 +1627,6 @@ def vm_start_wait(vm_name, timeout=1800, quiet=False, clone_name=None):
136 sys.stdout.write("\n")
137 return False
138
139-def vm_ping(vm_name):
140- '''Attempts to ping a VM'''
141- rc, out = runcmd(["ping", "-c1", "-w1", vm_name])
142- if rc == 0 and ssh_connect(vm_name) == True:
143- return vm_name
144- return ""
145-
146 def ssh_connect(host, resolve=True):
147 '''Attempts a connection on port 22'''
148 import socket
149@@ -3127,7 +3171,10 @@ def get_gpg_public_key(keyid=None):
150 return None
151
152 print("Exporting GPG key '%s'" % keyid)
153- rc, out = runcmd(['gpg', '--export', '--armor', keyid])
154+ cmd = ['gpg', '--export', '--armor']
155+ if is_snap():
156+ cmd += ['--homedir', os.getenv('SNAP_REAL_HOME', os.path.expanduser("~")) + '/.gnupg']
157+ rc, out = runcmd(cmd + [keyid])
158 # make sure something actually got exported
159 if rc != 0 or not out.startswith('-----BEGIN PGP PUBLIC KEY BLOCK-----'):
160 print("Error: Failed to export public key '%s'." % keyid, file=sys.stderr)
161@@ -3225,6 +3272,11 @@ def create_uvt_config():
162 #vm_ssh_key="$HOME/.ssh/id_rsa.pub"
163
164 #
165+# vm_known_hosts: SSH known_hosts file used by uvt to authenticate the
166+# the virtual machine when connecting via SSH
167+#vm_known_hosts="$HOME/.ssh/known_hosts"
168+
169+#
170 # vm_aptproxy: If set, this sets up an apt proxy in the virtual machine.
171 # No default setting. Can be used to set up apt-cacher-ng, for
172 # example.
173@@ -3341,7 +3393,6 @@ def check_required_tools():
174 'kvm-ok' : 'cpu-checker',
175 'gzip' : 'gzip',
176 'cpio' : 'cpio',
177- 'kvm' : 'qemu-kvm',
178 'mkpasswd' : 'whois' }
179
180 missing_tools = []
181@@ -3487,15 +3538,15 @@ def load_uvt_config():
182 if not 'vm_xkboptions' in config:
183 config['vm_xkboptions'] = keyboard.get('XKBOPTIONS', "")
184
185- # Set a default image size to 8GB and memory to 1024MB
186+ # Set a default image size to 20GB and memory to 4096MB
187 if config.get('vm_image_size', "") == "":
188- config['vm_image_size'] = "8"
189+ config['vm_image_size'] = "20"
190 if config.get('vm_memory', "") == "":
191- config['vm_memory'] = "1024"
192+ config['vm_memory'] = "4096"
193
194- # Set default vcpus to 1
195+ # Set default vcpus to 2
196 if config.get('vm_vcpus', "") == "":
197- config['vm_vcpus'] = "1"
198+ config['vm_vcpus'] = "2"
199
200 # Set a default username and password
201 if config.get('vm_username', "") == "":
202@@ -3511,9 +3562,11 @@ def load_uvt_config():
203 if config['vm_timezone'] == "":
204 config['vm_timezone'] = "UTC"
205
206- # Set default ssh key file
207+ # Set default ssh key file and known_hosts
208 if config.get('vm_ssh_key', "") == "":
209 config['vm_ssh_key'] = os.path.expanduser("~/.ssh/id_rsa.pub")
210+ if config.get('vm_known_hosts', "") == "":
211+ config['vm_known_hosts'] = os.path.expanduser("~/.ssh/known_hosts")
212
213 # Other stuff
214 if not 'vm_aptproxy' in config:
215@@ -3615,6 +3668,7 @@ repo Adds or removes a local repo to a VM
216 update Runs a dist-upgrade inside a VM
217 clone Clones a VM into a new one
218 cmd Run a command inside a virtual machine
219+ssh Opens an interactive session with the VM
220 list List virtual machines
221 view Connect to a virtual machine with VNC
222 config Create an optional config file (~/%s)
223@@ -3647,10 +3701,25 @@ class BetterUbuntuDistroInfo(distro_info.UbuntuDistroInfo):
224
225 return release.split()[0] # handle '16.04 LTS' vs '17.10'
226
227+def is_snap():
228+ if os.getenv("SNAP") is not None:
229+ return True
230+ return False
231+
232+def setup_snap_environment():
233+ # Update home if needed
234+ if (os.getenv("UVT_SNAP_COMPAT_MODE") is not None) and (os.getenv("SNAP_REAL_HOME") is not None):
235+ os.environ["HOME"] = os.getenv("SNAP_REAL_HOME")
236+ elif os.getenv("SNAP_USER_COMMON") is not None:
237+ os.environ["HOME"] = os.getenv("SNAP_USER_COMMON")
238+
239 #
240 # Main program
241 #
242
243+if is_snap():
244+ setup_snap_environment()
245+
246 config_file = ".uvt.conf"
247
248 cmd = None
249@@ -3683,6 +3752,7 @@ commands = {
250 'update' : cmd_update,
251 'clone' : cmd_clone,
252 'cmd' : cmd_cmd,
253+ 'ssh' : cmd_ssh,
254 'list' : cmd_list,
255 'config' : cmd_config,
256 'dump' : cmd_dump,

Subscribers

People subscribed via source and target branches