Merge lp:~soren/nova/raw-disk-images into lp:~hudson-openstack/nova/trunk
- raw-disk-images
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Soren Hansen | ||||
Approved revision: | 439 | ||||
Merged at revision: | 471 | ||||
Proposed branch: | lp:~soren/nova/raw-disk-images | ||||
Merge into: | lp:~hudson-openstack/nova/trunk | ||||
Diff against target: |
832 lines (+264/-268) 15 files modified
.mailmap (+1/-0) MANIFEST.in (+1/-1) nova/api/ec2/cloud.py (+1/-1) nova/compute/api.py (+11/-6) nova/compute/disk.py (+19/-3) nova/flags.py (+3/-4) nova/tests/virt_unittest.py (+108/-24) nova/virt/libvirt.rescue.qemu.xml.template (+0/-37) nova/virt/libvirt.rescue.uml.xml.template (+0/-26) nova/virt/libvirt.rescue.xen.xml.template (+0/-34) nova/virt/libvirt.uml.xml.template (+0/-26) nova/virt/libvirt.xen.xml.template (+0/-30) nova/virt/libvirt.xml.template (+61/-18) nova/virt/libvirt_conn.py (+58/-58) tools/pip-requires (+1/-0) |
||||
To merge this branch: | bzr merge lp:~soren/nova/raw-disk-images | ||||
Related bugs: |
|
||||
Related blueprints: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jay Pipes (community) | Approve | ||
Vish Ishaya (community) | Approve | ||
Salvatore Orlando (community) | Needs Information | ||
Rick Clark (community) | Approve | ||
Review via email: mp+43712@code.launchpad.net |
Commit message
Add raw disk image support.
Description of the change
This branch adds raw disk image support. It was originally implemented by Justin Santa Barbara, then forward-ported by Rick Clark, and now I (believe I) finished it off.
Soren Hansen (soren) wrote : | # |
I need to make sure this works after the eventlet merge..
Soren Hansen (soren) wrote : | # |
Should be fine now.
Salvatore Orlando (salvatore-orlando) wrote : | # |
Hi Soren,
I checked out your code, and I am not able to launch raw images through the EC2 api.
When I use ' --kernel nokernel', I get the following error:
ERROR:root:
I believe this happens because the image service still tries to retrieve the kernel image.
from compute/api.py lines 86-87:
if kernel_id:
In the original branch from justinsb, kernel and ramdisk images were not retrieve when the null_kernel 'magic number' was used.
if kernel_id == str(FLAGS.
# Make sure we have access to kernel and ramdisk (if not raw)
if kernel_id:
if ramdisk_id:
Is there something I'm not seeing at the moment, or do we need to fix compute/api.py?
Rick Clark (dendrobates) wrote : | # |
Salvatore,
I have reproduced your problem. Soren is now out for Christmas holiday. I suggest that we merge this to clear up the queue and then fix the bug. It does not break existing behavior. Waiting for Soren to fix this branch will delay merging an the fix to api.py should be simple.
Thierry Carrez (ttx) wrote : | # |
+1, just make sure we have a bug filed to remember to fix that one.
Salvatore Orlando (salvatore-orlando) wrote : | # |
Hi Rick,
I fixed it in my brach which has been proposed for merge.
Vish Ishaya (vishvananda) wrote : | # |
Lets get this in and aadd salvatore's fix.
Jay Pipes (jaypipes) wrote : | # |
Looks great. One tiny pep8 fix to make, though:
jpipes@
nova/compute/
Soren Hansen (soren) wrote : | # |
Fixed.
OpenStack Infra (hudson-openstack) wrote : | # |
The attempt to merge lp:~soren/nova/raw-disk-images into lp:nova failed. Below is the output from the failed tests.
Traceback (most recent call last):
File "run_tests.py", line 71, in <module>
from nova.tests.
File "/tmp/tmpgcaUjT
from nova.virt import libvirt_conn
File "/tmp/tmpgcaUjT
from Cheetah.Template import Template
ImportError: No module named Cheetah.Template
Soren Hansen (soren) wrote : | # |
soren@hudson:~$ sudo apt-get install python-cheetah
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
python-cheetah
0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
Need to get 0B/199kB of archives.
After this operation, 1073kB of additional disk space will be used.
Selecting previously deselected package python-cheetah.
(Reading database ... 41319 files and directories currently installed.)
Unpacking python-cheetah (from .../python-
Processing triggers for man-db ...
Setting up python-cheetah (2.0.1-2ubuntu7) ...
Processing triggers for python-support ...
soren@hudson:~$
Retrying.
OpenStack Infra (hudson-openstack) wrote : | # |
The attempt to merge lp:~soren/nova/raw-disk-images into lp:nova failed. Below is the output from the failed tests.
nova.tests.
AccessTestCase
test_
test_
test_
test_
nova.tests.
ApiEc2TestCase
test_
test_
test_
test_
test_
test_
XmlConversion
test_
nova.tests.
AuthManagerDb
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
test_
- 439. By Soren Hansen
-
Add my @linux2go.dk address to .mailmap
Soren Hansen (soren) wrote : | # |
I suck.
Note to self: Running tests before committing to make sure everything is fine isn't enough. *sigh*
Preview Diff
1 | === modified file '.mailmap' |
2 | --- .mailmap 2010-11-23 21:34:53 +0000 |
3 | +++ .mailmap 2010-12-20 21:55:17 +0000 |
4 | @@ -19,6 +19,7 @@ |
5 | <mordred@inaugust.com> <mordred@hudson> |
6 | <paul@openstack.org> <pvoccio@castor.local> |
7 | <paul@openstack.org> <paul.voccio@rackspace.com> |
8 | +<soren.hansen@rackspace.com> <soren@linux2go.dk> |
9 | <todd@ansolabs.com> <todd@lapex> |
10 | <todd@ansolabs.com> <todd@rubidine.com> |
11 | <vishvananda@gmail.com> <vishvananda@yahoo.com> |
12 | |
13 | === modified file 'MANIFEST.in' |
14 | --- MANIFEST.in 2010-11-20 21:26:15 +0000 |
15 | +++ MANIFEST.in 2010-12-20 21:55:17 +0000 |
16 | @@ -13,7 +13,7 @@ |
17 | include nova/compute/fakevirtinstance.xml |
18 | include nova/compute/interfaces.template |
19 | include nova/virt/interfaces.template |
20 | -include nova/virt/libvirt.*.xml.template |
21 | +include nova/virt/libvirt*.xml.template |
22 | include nova/tests/CA/ |
23 | include nova/tests/CA/cacert.pem |
24 | include nova/tests/CA/private/ |
25 | |
26 | === modified file 'nova/api/ec2/cloud.py' |
27 | --- nova/api/ec2/cloud.py 2010-12-09 17:30:03 +0000 |
28 | +++ nova/api/ec2/cloud.py 2010-12-20 21:55:17 +0000 |
29 | @@ -751,7 +751,7 @@ |
30 | kwargs['image_id'], |
31 | min_count=int(kwargs.get('min_count', max_count)), |
32 | max_count=max_count, |
33 | - kernel_id=kwargs.get('kernel_id'), |
34 | + kernel_id=kwargs.get('kernel_id', None), |
35 | ramdisk_id=kwargs.get('ramdisk_id'), |
36 | display_name=kwargs.get('display_name'), |
37 | description=kwargs.get('display_description'), |
38 | |
39 | === modified file 'nova/compute/api.py' |
40 | --- nova/compute/api.py 2010-12-14 21:56:42 +0000 |
41 | +++ nova/compute/api.py 2010-12-20 21:55:17 +0000 |
42 | @@ -73,14 +73,19 @@ |
43 | is_vpn = image_id == FLAGS.vpn_image_id |
44 | if not is_vpn: |
45 | image = self.image_service.show(context, image_id) |
46 | + |
47 | + # If kernel_id/ramdisk_id isn't explicitly set in API call |
48 | + # we take the defaults from the image's metadata |
49 | if kernel_id is None: |
50 | - kernel_id = image.get('kernelId', FLAGS.default_kernel) |
51 | + kernel_id = image.get('kernelId', None) |
52 | if ramdisk_id is None: |
53 | - ramdisk_id = image.get('ramdiskId', FLAGS.default_ramdisk) |
54 | + ramdisk_id = image.get('ramdiskId', None) |
55 | |
56 | # Make sure we have access to kernel and ramdisk |
57 | - self.image_service.show(context, kernel_id) |
58 | - self.image_service.show(context, ramdisk_id) |
59 | + if kernel_id: |
60 | + self.image_service.show(context, kernel_id) |
61 | + if ramdisk_id: |
62 | + self.image_service.show(context, ramdisk_id) |
63 | |
64 | if security_group is None: |
65 | security_group = ['default'] |
66 | @@ -103,8 +108,8 @@ |
67 | base_options = { |
68 | 'reservation_id': utils.generate_uid('r'), |
69 | 'image_id': image_id, |
70 | - 'kernel_id': kernel_id, |
71 | - 'ramdisk_id': ramdisk_id, |
72 | + 'kernel_id': kernel_id or '', |
73 | + 'ramdisk_id': ramdisk_id or '', |
74 | 'state_description': 'scheduling', |
75 | 'user_id': context.user_id, |
76 | 'project_id': context.project_id, |
77 | |
78 | === modified file 'nova/compute/disk.py' |
79 | --- nova/compute/disk.py 2010-12-08 20:59:22 +0000 |
80 | +++ nova/compute/disk.py 2010-12-20 21:55:17 +0000 |
81 | @@ -106,6 +106,13 @@ |
82 | % (outfile, local_type, local_first, local_last)) |
83 | |
84 | |
85 | +def extend(image, size, execute): |
86 | + file_size = os.path.getsize(image) |
87 | + if file_size >= size: |
88 | + return |
89 | + return execute('truncate -s size %s' % (image,)) |
90 | + |
91 | + |
92 | def inject_data(image, key=None, net=None, partition=None, execute=None): |
93 | """Injects a ssh key and optionally net data into a disk image. |
94 | |
95 | @@ -115,7 +122,7 @@ |
96 | If partition is not specified it mounts the image as a single partition. |
97 | |
98 | """ |
99 | - out, err = execute('sudo losetup -f --show %s' % image) |
100 | + out, err = execute('sudo losetup --find --show %s' % image) |
101 | if err: |
102 | raise exception.Error('Could not attach image to loopback: %s' % err) |
103 | device = out.strip() |
104 | @@ -129,6 +136,15 @@ |
105 | partition) |
106 | else: |
107 | mapped_device = device |
108 | + |
109 | + # We can only loopback mount raw images. If the device isn't there, |
110 | + # it's normally because it's a .vmdk or a .vdi etc |
111 | + if not os.path.exists(mapped_device): |
112 | + raise exception.Error('Mapped device was not found (we can' |
113 | + ' only inject raw disk images): %s' % |
114 | + mapped_device) |
115 | + |
116 | + # Configure ext2fs so that it doesn't auto-check every N boots |
117 | out, err = execute('sudo tune2fs -c 0 -i 0 %s' % mapped_device) |
118 | |
119 | tmpdir = tempfile.mkdtemp() |
120 | @@ -156,7 +172,7 @@ |
121 | execute('sudo kpartx -d %s' % device) |
122 | finally: |
123 | # remove loopback |
124 | - execute('sudo losetup -d %s' % device) |
125 | + execute('sudo losetup --detach %s' % device) |
126 | |
127 | |
128 | def _inject_key_into_fs(key, fs, execute=None): |
129 | @@ -165,7 +181,7 @@ |
130 | key is an ssh key string. |
131 | fs is the path to the base of the filesystem into which to inject the key. |
132 | """ |
133 | - sshdir = os.path.join(os.path.join(fs, 'root'), '.ssh') |
134 | + sshdir = os.path.join(fs, 'root', '.ssh') |
135 | execute('sudo mkdir -p %s' % sshdir) # existing dir doesn't matter |
136 | execute('sudo chown root %s' % sshdir) |
137 | execute('sudo chmod 700 %s' % sshdir) |
138 | |
139 | === modified file 'nova/flags.py' |
140 | --- nova/flags.py 2010-12-10 00:05:13 +0000 |
141 | +++ nova/flags.py 2010-12-20 21:55:17 +0000 |
142 | @@ -235,12 +235,11 @@ |
143 | |
144 | DEFINE_string('default_image', 'ami-11111', |
145 | 'default image to use, testing only') |
146 | -DEFINE_string('default_kernel', 'aki-11111', |
147 | - 'default kernel to use, testing only') |
148 | -DEFINE_string('default_ramdisk', 'ari-11111', |
149 | - 'default ramdisk to use, testing only') |
150 | DEFINE_string('default_instance_type', 'm1.small', |
151 | 'default instance type to use, testing only') |
152 | +DEFINE_string('null_kernel', 'nokernel', |
153 | + 'kernel image that indicates not to use a kernel,' |
154 | + ' but to use a raw disk image instead') |
155 | |
156 | DEFINE_string('vpn_image_id', 'ami-CLOUDPIPE', 'AMI for cloudpipe vpn server') |
157 | DEFINE_string('vpn_key_suffix', |
158 | |
159 | === modified file 'nova/tests/virt_unittest.py' |
160 | --- nova/tests/virt_unittest.py 2010-12-10 00:05:13 +0000 |
161 | +++ nova/tests/virt_unittest.py 2010-12-20 21:55:17 +0000 |
162 | @@ -40,19 +40,51 @@ |
163 | self.network = utils.import_object(FLAGS.network_manager) |
164 | FLAGS.instances_path = '' |
165 | |
166 | - def test_get_uri_and_template(self): |
167 | - ip = '10.11.12.13' |
168 | - |
169 | - instance = {'internal_id': 1, |
170 | - 'memory_kb': '1024000', |
171 | - 'basepath': '/some/path', |
172 | - 'bridge_name': 'br100', |
173 | - 'mac_address': '02:12:34:46:56:67', |
174 | - 'vcpus': 2, |
175 | - 'project_id': 'fake', |
176 | - 'bridge': 'br101', |
177 | - 'instance_type': 'm1.small'} |
178 | - |
179 | + test_ip = '10.11.12.13' |
180 | + test_instance = {'memory_kb': '1024000', |
181 | + 'basepath': '/some/path', |
182 | + 'bridge_name': 'br100', |
183 | + 'mac_address': '02:12:34:46:56:67', |
184 | + 'vcpus': 2, |
185 | + 'project_id': 'fake', |
186 | + 'bridge': 'br101', |
187 | + 'instance_type': 'm1.small'} |
188 | + |
189 | + def test_xml_and_uri_no_ramdisk_no_kernel(self): |
190 | + instance_data = dict(self.test_instance) |
191 | + self.do_test_xml_and_uri(instance_data, |
192 | + expect_kernel=False, expect_ramdisk=False) |
193 | + |
194 | + def test_xml_and_uri_no_ramdisk(self): |
195 | + instance_data = dict(self.test_instance) |
196 | + instance_data['kernel_id'] = 'aki-deadbeef' |
197 | + self.do_test_xml_and_uri(instance_data, |
198 | + expect_kernel=True, expect_ramdisk=False) |
199 | + |
200 | + def test_xml_and_uri_no_kernel(self): |
201 | + instance_data = dict(self.test_instance) |
202 | + instance_data['ramdisk_id'] = 'ari-deadbeef' |
203 | + self.do_test_xml_and_uri(instance_data, |
204 | + expect_kernel=False, expect_ramdisk=False) |
205 | + |
206 | + def test_xml_and_uri(self): |
207 | + instance_data = dict(self.test_instance) |
208 | + instance_data['ramdisk_id'] = 'ari-deadbeef' |
209 | + instance_data['kernel_id'] = 'aki-deadbeef' |
210 | + self.do_test_xml_and_uri(instance_data, |
211 | + expect_kernel=True, expect_ramdisk=True) |
212 | + |
213 | + def test_xml_and_uri_rescue(self): |
214 | + instance_data = dict(self.test_instance) |
215 | + instance_data['ramdisk_id'] = 'ari-deadbeef' |
216 | + instance_data['kernel_id'] = 'aki-deadbeef' |
217 | + self.do_test_xml_and_uri(instance_data, |
218 | + expect_kernel=True, expect_ramdisk=True, |
219 | + rescue=True) |
220 | + |
221 | + def do_test_xml_and_uri(self, instance, |
222 | + expect_ramdisk, expect_kernel, |
223 | + rescue=False): |
224 | user_context = context.RequestContext(project=self.project, |
225 | user=self.user) |
226 | instance_ref = db.instance_create(user_context, instance) |
227 | @@ -60,13 +92,14 @@ |
228 | self.network.set_network_host(context.get_admin_context(), |
229 | network_ref['id']) |
230 | |
231 | - fixed_ip = {'address': ip, |
232 | + fixed_ip = {'address': self.test_ip, |
233 | 'network_id': network_ref['id']} |
234 | |
235 | ctxt = context.get_admin_context() |
236 | fixed_ip_ref = db.fixed_ip_create(ctxt, fixed_ip) |
237 | - db.fixed_ip_update(ctxt, ip, {'allocated': True, |
238 | - 'instance_id': instance_ref['id']}) |
239 | + db.fixed_ip_update(ctxt, self.test_ip, |
240 | + {'allocated': True, |
241 | + 'instance_id': instance_ref['id']}) |
242 | |
243 | type_uri_map = {'qemu': ('qemu:///system', |
244 | [(lambda t: t.find('.').get('type'), 'qemu'), |
245 | @@ -78,23 +111,71 @@ |
246 | (lambda t: t.find('./devices/emulator'), None)]), |
247 | 'uml': ('uml:///system', |
248 | [(lambda t: t.find('.').get('type'), 'uml'), |
249 | - (lambda t: t.find('./os/type').text, 'uml')])} |
250 | + (lambda t: t.find('./os/type').text, 'uml')]), |
251 | + 'xen': ('xen:///', |
252 | + [(lambda t: t.find('.').get('type'), 'xen'), |
253 | + (lambda t: t.find('./os/type').text, 'linux')]), |
254 | + } |
255 | + |
256 | + for hypervisor_type in ['qemu', 'kvm', 'xen']: |
257 | + check_list = type_uri_map[hypervisor_type][1] |
258 | + |
259 | + if rescue: |
260 | + check = (lambda t: t.find('./os/kernel').text.split('/')[1], |
261 | + 'rescue-kernel') |
262 | + check_list.append(check) |
263 | + check = (lambda t: t.find('./os/initrd').text.split('/')[1], |
264 | + 'rescue-ramdisk') |
265 | + check_list.append(check) |
266 | + else: |
267 | + if expect_kernel: |
268 | + check = (lambda t: t.find('./os/kernel').text.split('/' |
269 | + )[1], 'kernel') |
270 | + else: |
271 | + check = (lambda t: t.find('./os/kernel'), None) |
272 | + check_list.append(check) |
273 | + |
274 | + if expect_ramdisk: |
275 | + check = (lambda t: t.find('./os/initrd').text.split('/' |
276 | + )[1], 'ramdisk') |
277 | + else: |
278 | + check = (lambda t: t.find('./os/initrd'), None) |
279 | + check_list.append(check) |
280 | |
281 | common_checks = [ |
282 | (lambda t: t.find('.').tag, 'domain'), |
283 | - (lambda t: t.find('./devices/interface/filterref/parameter').\ |
284 | - get('name'), 'IP'), |
285 | - (lambda t: t.find('./devices/interface/filterref/parameter').\ |
286 | - get('value'), '10.11.12.13')] |
287 | + (lambda t: t.find('./devices/interface/filterref/parameter' |
288 | + ).get('name'), 'IP'), |
289 | + (lambda t: t.find('./devices/interface/filterref/parameter' |
290 | + ).get('value'), '10.11.12.13'), |
291 | + (lambda t: t.findall('./devices/interface/filterref/parameter' |
292 | + )[1].get('name'), 'DHCPSERVER'), |
293 | + (lambda t: t.findall('./devices/interface/filterref/parameter' |
294 | + )[1].get('value'), '10.0.0.1'), |
295 | + (lambda t: t.find('./devices/serial/source').get('path' |
296 | + ).split('/')[1], 'console.log'), |
297 | + (lambda t: t.find('./memory').text, '2097152')] |
298 | + |
299 | + if rescue: |
300 | + common_checks += [(lambda t: t.findall('./devices/disk/source' |
301 | + )[0].get('file').split('/')[1], |
302 | + 'rescue-disk'), |
303 | + (lambda t: t.findall('./devices/disk/source' |
304 | + )[1].get('file').split('/')[1], |
305 | + 'disk')] |
306 | + else: |
307 | + common_checks += [(lambda t: t.findall('./devices/disk/source' |
308 | + )[0].get('file').split('/')[1], |
309 | + 'disk')] |
310 | |
311 | for (libvirt_type, (expected_uri, checks)) in type_uri_map.iteritems(): |
312 | FLAGS.libvirt_type = libvirt_type |
313 | conn = libvirt_conn.LibvirtConnection(True) |
314 | |
315 | - uri, _template, _rescue = conn.get_uri_and_templates() |
316 | + uri = conn.get_uri() |
317 | self.assertEquals(uri, expected_uri) |
318 | |
319 | - xml = conn.to_xml(instance_ref) |
320 | + xml = conn.to_xml(instance_ref, rescue) |
321 | tree = xml_to_tree(xml) |
322 | for i, (check, expected_result) in enumerate(checks): |
323 | self.assertEqual(check(tree), |
324 | @@ -106,6 +187,9 @@ |
325 | expected_result, |
326 | '%s failed common check %d' % (xml, i)) |
327 | |
328 | + # This test is supposed to make sure we don't override a specifically |
329 | + # set uri |
330 | + # |
331 | # Deliberately not just assigning this string to FLAGS.libvirt_uri and |
332 | # checking against that later on. This way we make sure the |
333 | # implementation doesn't fiddle around with the FLAGS. |
334 | @@ -114,7 +198,7 @@ |
335 | for (libvirt_type, (expected_uri, checks)) in type_uri_map.iteritems(): |
336 | FLAGS.libvirt_type = libvirt_type |
337 | conn = libvirt_conn.LibvirtConnection(True) |
338 | - uri, _template, _rescue = conn.get_uri_and_templates() |
339 | + uri = conn.get_uri() |
340 | self.assertEquals(uri, testuri) |
341 | |
342 | def tearDown(self): |
343 | |
344 | === removed file 'nova/virt/libvirt.rescue.qemu.xml.template' |
345 | --- nova/virt/libvirt.rescue.qemu.xml.template 2010-10-24 21:57:37 +0000 |
346 | +++ nova/virt/libvirt.rescue.qemu.xml.template 1970-01-01 00:00:00 +0000 |
347 | @@ -1,37 +0,0 @@ |
348 | -<domain type='%(type)s'> |
349 | - <name>%(name)s</name> |
350 | - <os> |
351 | - <type>hvm</type> |
352 | - <kernel>%(basepath)s/rescue-kernel</kernel> |
353 | - <initrd>%(basepath)s/rescue-ramdisk</initrd> |
354 | - <cmdline>root=/dev/vda1 console=ttyS0</cmdline> |
355 | - </os> |
356 | - <features> |
357 | - <acpi/> |
358 | - </features> |
359 | - <memory>%(memory_kb)s</memory> |
360 | - <vcpu>%(vcpus)s</vcpu> |
361 | - <devices> |
362 | - <disk type='file'> |
363 | - <source file='%(basepath)s/rescue-disk'/> |
364 | - <target dev='vda' bus='virtio'/> |
365 | - </disk> |
366 | - <disk type='file'> |
367 | - <source file='%(basepath)s/disk'/> |
368 | - <target dev='vdb' bus='virtio'/> |
369 | - </disk> |
370 | - <interface type='bridge'> |
371 | - <source bridge='%(bridge_name)s'/> |
372 | - <mac address='%(mac_address)s'/> |
373 | - <!-- <model type='virtio'/> CANT RUN virtio network right now --> |
374 | - <filterref filter="nova-instance-%(name)s"> |
375 | - <parameter name="IP" value="%(ip_address)s" /> |
376 | - <parameter name="DHCPSERVER" value="%(dhcp_server)s" /> |
377 | - </filterref> |
378 | - </interface> |
379 | - <serial type="file"> |
380 | - <source path='%(basepath)s/console.log'/> |
381 | - <target port='1'/> |
382 | - </serial> |
383 | - </devices> |
384 | -</domain> |
385 | |
386 | === removed file 'nova/virt/libvirt.rescue.uml.xml.template' |
387 | --- nova/virt/libvirt.rescue.uml.xml.template 2010-09-20 08:50:08 +0000 |
388 | +++ nova/virt/libvirt.rescue.uml.xml.template 1970-01-01 00:00:00 +0000 |
389 | @@ -1,26 +0,0 @@ |
390 | -<domain type='%(type)s'> |
391 | - <name>%(name)s</name> |
392 | - <memory>%(memory_kb)s</memory> |
393 | - <os> |
394 | - <type>%(type)s</type> |
395 | - <kernel>/usr/bin/linux</kernel> |
396 | - <root>/dev/ubda1</root> |
397 | - </os> |
398 | - <devices> |
399 | - <disk type='file'> |
400 | - <source file='%(basepath)s/rescue-disk'/> |
401 | - <target dev='ubd0' bus='uml'/> |
402 | - </disk> |
403 | - <disk type='file'> |
404 | - <source file='%(basepath)s/disk'/> |
405 | - <target dev='ubd1' bus='uml'/> |
406 | - </disk> |
407 | - <interface type='bridge'> |
408 | - <source bridge='%(bridge_name)s'/> |
409 | - <mac address='%(mac_address)s'/> |
410 | - </interface> |
411 | - <console type="file"> |
412 | - <source path='%(basepath)s/console.log'/> |
413 | - </console> |
414 | - </devices> |
415 | -</domain> |
416 | |
417 | === removed file 'nova/virt/libvirt.rescue.xen.xml.template' |
418 | --- nova/virt/libvirt.rescue.xen.xml.template 2010-10-25 00:18:24 +0000 |
419 | +++ nova/virt/libvirt.rescue.xen.xml.template 1970-01-01 00:00:00 +0000 |
420 | @@ -1,34 +0,0 @@ |
421 | -<domain type='%(type)s'> |
422 | - <name>%(name)s</name> |
423 | - <os> |
424 | - <type>linux</type> |
425 | - <kernel>%(basepath)s/kernel</kernel> |
426 | - <initrd>%(basepath)s/ramdisk</initrd> |
427 | - <root>/dev/xvda1</root> |
428 | - <cmdline>ro</cmdline> |
429 | - </os> |
430 | - <features> |
431 | - <acpi/> |
432 | - </features> |
433 | - <memory>%(memory_kb)s</memory> |
434 | - <vcpu>%(vcpus)s</vcpu> |
435 | - <devices> |
436 | - <disk type='file'> |
437 | - <source file='%(basepath)s/rescue-disk'/> |
438 | - <target dev='sda' /> |
439 | - </disk> |
440 | - <disk type='file'> |
441 | - <source file='%(basepath)s/disk'/> |
442 | - <target dev='sdb' /> |
443 | - </disk> |
444 | - <interface type='bridge'> |
445 | - <source bridge='%(bridge_name)s'/> |
446 | - <mac address='%(mac_address)s'/> |
447 | - </interface> |
448 | - <console type="file"> |
449 | - <source path='%(basepath)s/console.log'/> |
450 | - <target port='1'/> |
451 | - </console> |
452 | - </devices> |
453 | -</domain> |
454 | - |
455 | |
456 | === removed file 'nova/virt/libvirt.uml.xml.template' |
457 | --- nova/virt/libvirt.uml.xml.template 2010-09-27 11:13:29 +0000 |
458 | +++ nova/virt/libvirt.uml.xml.template 1970-01-01 00:00:00 +0000 |
459 | @@ -1,26 +0,0 @@ |
460 | -<domain type='%(type)s'> |
461 | - <name>%(name)s</name> |
462 | - <memory>%(memory_kb)s</memory> |
463 | - <os> |
464 | - <type>%(type)s</type> |
465 | - <kernel>/usr/bin/linux</kernel> |
466 | - <root>/dev/ubda1</root> |
467 | - </os> |
468 | - <devices> |
469 | - <disk type='file'> |
470 | - <source file='%(basepath)s/disk'/> |
471 | - <target dev='ubd0' bus='uml'/> |
472 | - </disk> |
473 | - <interface type='bridge'> |
474 | - <source bridge='%(bridge_name)s'/> |
475 | - <mac address='%(mac_address)s'/> |
476 | - <filterref filter="nova-instance-%(name)s"> |
477 | - <parameter name="IP" value="%(ip_address)s" /> |
478 | - <parameter name="DHCPSERVER" value="%(dhcp_server)s" /> |
479 | - </filterref> |
480 | - </interface> |
481 | - <console type="file"> |
482 | - <source path='%(basepath)s/console.log'/> |
483 | - </console> |
484 | - </devices> |
485 | -</domain> |
486 | |
487 | === removed file 'nova/virt/libvirt.xen.xml.template' |
488 | --- nova/virt/libvirt.xen.xml.template 2010-10-25 00:18:24 +0000 |
489 | +++ nova/virt/libvirt.xen.xml.template 1970-01-01 00:00:00 +0000 |
490 | @@ -1,30 +0,0 @@ |
491 | -<domain type='%(type)s'> |
492 | - <name>%(name)s</name> |
493 | - <os> |
494 | - <type>linux</type> |
495 | - <kernel>%(basepath)s/kernel</kernel> |
496 | - <initrd>%(basepath)s/ramdisk</initrd> |
497 | - <root>/dev/xvda1</root> |
498 | - <cmdline>ro</cmdline> |
499 | - </os> |
500 | - <features> |
501 | - <acpi/> |
502 | - </features> |
503 | - <memory>%(memory_kb)s</memory> |
504 | - <vcpu>%(vcpus)s</vcpu> |
505 | - <devices> |
506 | - <disk type='file'> |
507 | - <source file='%(basepath)s/disk'/> |
508 | - <target dev='sda' /> |
509 | - </disk> |
510 | - <interface type='bridge'> |
511 | - <source bridge='%(bridge_name)s'/> |
512 | - <mac address='%(mac_address)s'/> |
513 | - </interface> |
514 | - <console type="file"> |
515 | - <source path='%(basepath)s/console.log'/> |
516 | - <target port='1'/> |
517 | - </console> |
518 | - </devices> |
519 | -</domain> |
520 | - |
521 | |
522 | === renamed file 'nova/virt/libvirt.qemu.xml.template' => 'nova/virt/libvirt.xml.template' |
523 | --- nova/virt/libvirt.qemu.xml.template 2010-09-27 11:13:29 +0000 |
524 | +++ nova/virt/libvirt.xml.template 2010-12-20 21:55:17 +0000 |
525 | @@ -1,32 +1,75 @@ |
526 | -<domain type='%(type)s'> |
527 | - <name>%(name)s</name> |
528 | +<domain type='${type}'> |
529 | + <name>${name}</name> |
530 | + <memory>${memory_kb}</memory> |
531 | <os> |
532 | - <type>hvm</type> |
533 | - <kernel>%(basepath)s/kernel</kernel> |
534 | - <initrd>%(basepath)s/ramdisk</initrd> |
535 | - <cmdline>root=/dev/vda1 console=ttyS0</cmdline> |
536 | +#if $type == 'uml' |
537 | + #set $disk_prefix = 'ubd' |
538 | + #set $disk_bus = 'uml' |
539 | + <type>uml</type> |
540 | + <kernel>/usr/bin/linux</kernel> |
541 | + <root>/dev/ubda1</root> |
542 | +#else |
543 | + #if $type == 'xen' |
544 | + #set $disk_prefix = 'sd' |
545 | + #set $disk_bus = 'scsi' |
546 | + <type>linux</type> |
547 | + <root>/dev/xvda1</root> |
548 | + #else |
549 | + #set $disk_prefix = 'vd' |
550 | + #set $disk_bus = 'virtio' |
551 | + <type>hvm</type> |
552 | + #end if |
553 | + #if $getVar('rescue', False) |
554 | + <kernel>${basepath}/rescue-kernel</kernel> |
555 | + <initrd>${basepath}/rescue-ramdisk</initrd> |
556 | + #else |
557 | + #if $getVar('kernel', None) |
558 | + <kernel>${kernel}</kernel> |
559 | + #if $type == 'xen' |
560 | + <cmdline>ro</cmdline> |
561 | + #else |
562 | + <cmdline>root=/dev/vda1 console=ttyS0</cmdline> |
563 | + #end if |
564 | + #if $getVar('ramdisk', None) |
565 | + <initrd>${ramdisk}</initrd> |
566 | + #end if |
567 | + #else |
568 | + <boot dev="hd" /> |
569 | + #end if |
570 | + #end if |
571 | +#end if |
572 | </os> |
573 | <features> |
574 | <acpi/> |
575 | </features> |
576 | - <memory>%(memory_kb)s</memory> |
577 | - <vcpu>%(vcpus)s</vcpu> |
578 | + <vcpu>${vcpus}</vcpu> |
579 | <devices> |
580 | - <disk type='file'> |
581 | - <source file='%(basepath)s/disk'/> |
582 | - <target dev='vda' bus='virtio'/> |
583 | - </disk> |
584 | +#if $getVar('rescue', False) |
585 | + <disk type='file'> |
586 | + <source file='${basepath}/rescue-disk'/> |
587 | + <target dev='${disk_prefix}a' bus='${disk_bus}'/> |
588 | + </disk> |
589 | + <disk type='file'> |
590 | + <source file='${basepath}/disk'/> |
591 | + <target dev='${disk_prefix}b' bus='${disk_bus}'/> |
592 | + </disk> |
593 | +#else |
594 | + <disk type='file'> |
595 | + <source file='${basepath}/disk'/> |
596 | + <target dev='${disk_prefix}a' bus='${disk_bus}'/> |
597 | + </disk> |
598 | +#end if |
599 | <interface type='bridge'> |
600 | - <source bridge='%(bridge_name)s'/> |
601 | - <mac address='%(mac_address)s'/> |
602 | + <source bridge='${bridge_name}'/> |
603 | + <mac address='${mac_address}'/> |
604 | <!-- <model type='virtio'/> CANT RUN virtio network right now --> |
605 | - <filterref filter="nova-instance-%(name)s"> |
606 | - <parameter name="IP" value="%(ip_address)s" /> |
607 | - <parameter name="DHCPSERVER" value="%(dhcp_server)s" /> |
608 | + <filterref filter="nova-instance-${name}"> |
609 | + <parameter name="IP" value="${ip_address}" /> |
610 | + <parameter name="DHCPSERVER" value="${dhcp_server}" /> |
611 | </filterref> |
612 | </interface> |
613 | <serial type="file"> |
614 | - <source path='%(basepath)s/console.log'/> |
615 | + <source path='${basepath}/console.log'/> |
616 | <target port='1'/> |
617 | </serial> |
618 | </devices> |
619 | |
620 | === modified file 'nova/virt/libvirt_conn.py' |
621 | --- nova/virt/libvirt_conn.py 2010-12-16 00:31:32 +0000 |
622 | +++ nova/virt/libvirt_conn.py 2010-12-20 21:55:17 +0000 |
623 | @@ -27,12 +27,7 @@ |
624 | :libvirt_type: Libvirt domain type. Can be kvm, qemu, uml, xen |
625 | (default: kvm). |
626 | :libvirt_uri: Override for the default libvirt URI (depends on libvirt_type). |
627 | -:libvirt_xml_template: Libvirt XML Template (QEmu/KVM). |
628 | -:libvirt_xen_xml_template: Libvirt XML Template (Xen). |
629 | -:libvirt_uml_xml_template: Libvirt XML Template (User Mode Linux). |
630 | -:libvirt_rescue_xml_template: XML template for rescue mode (KVM & QEMU). |
631 | -:libvirt_rescue_xen_xml_template: XML templage for rescue mode (XEN). |
632 | -:libvirt_rescue_uml_xml_template: XML template for rescue mode (UML). |
633 | +:libvirt_xml_template: Libvirt XML Template. |
634 | :rescue_image_id: Rescue ami image (default: ami-rescue). |
635 | :rescue_kernel_id: Rescue aki image (default: aki-rescue). |
636 | :rescue_ramdisk_id: Rescue ari image (default: ari-rescue). |
637 | @@ -62,36 +57,20 @@ |
638 | from nova.compute import power_state |
639 | from nova.virt import images |
640 | |
641 | +from Cheetah.Template import Template |
642 | + |
643 | libvirt = None |
644 | libxml2 = None |
645 | |
646 | |
647 | FLAGS = flags.FLAGS |
648 | -flags.DEFINE_string('libvirt_rescue_xml_template', |
649 | - utils.abspath('virt/libvirt.rescue.qemu.xml.template'), |
650 | - 'Libvirt RESCUE XML Template for QEmu/KVM') |
651 | -flags.DEFINE_string('libvirt_rescue_xen_xml_template', |
652 | - utils.abspath('virt/libvirt.rescue.xen.xml.template'), |
653 | - 'Libvirt RESCUE XML Template for xen') |
654 | -flags.DEFINE_string('libvirt_rescue_uml_xml_template', |
655 | - utils.abspath('virt/libvirt.rescue.uml.xml.template'), |
656 | - 'Libvirt RESCUE XML Template for user-mode-linux') |
657 | # TODO(vish): These flags should probably go into a shared location |
658 | flags.DEFINE_string('rescue_image_id', 'ami-rescue', 'Rescue ami image') |
659 | flags.DEFINE_string('rescue_kernel_id', 'aki-rescue', 'Rescue aki image') |
660 | flags.DEFINE_string('rescue_ramdisk_id', 'ari-rescue', 'Rescue ari image') |
661 | flags.DEFINE_string('libvirt_xml_template', |
662 | - utils.abspath('virt/libvirt.qemu.xml.template'), |
663 | - 'Libvirt XML Template for QEmu/KVM') |
664 | -flags.DEFINE_string('libvirt_xen_xml_template', |
665 | - utils.abspath('virt/libvirt.xen.xml.template'), |
666 | - 'Libvirt XML Template for Xen') |
667 | -flags.DEFINE_string('libvirt_uml_xml_template', |
668 | - utils.abspath('virt/libvirt.uml.xml.template'), |
669 | - 'Libvirt XML Template for user-mode-linux') |
670 | -flags.DEFINE_string('injected_network_template', |
671 | - utils.abspath('virt/interfaces.template'), |
672 | - 'Template file for injected network') |
673 | + utils.abspath('virt/libvirt.xml.template'), |
674 | + 'Libvirt XML Template') |
675 | flags.DEFINE_string('libvirt_type', |
676 | 'kvm', |
677 | 'Libvirt domain type (valid options are: ' |
678 | @@ -120,12 +99,9 @@ |
679 | class LibvirtConnection(object): |
680 | |
681 | def __init__(self, read_only): |
682 | - (self.libvirt_uri, |
683 | - template_file, |
684 | - rescue_file) = self.get_uri_and_templates() |
685 | + self.libvirt_uri = self.get_uri() |
686 | |
687 | - self.libvirt_xml = open(template_file).read() |
688 | - self.rescue_xml = open(rescue_file).read() |
689 | + self.libvirt_xml = open(FLAGS.libvirt_xml_template).read() |
690 | self._wrapped_conn = None |
691 | self.read_only = read_only |
692 | |
693 | @@ -148,20 +124,14 @@ |
694 | return False |
695 | raise |
696 | |
697 | - def get_uri_and_templates(self): |
698 | + def get_uri(self): |
699 | if FLAGS.libvirt_type == 'uml': |
700 | uri = FLAGS.libvirt_uri or 'uml:///system' |
701 | - template_file = FLAGS.libvirt_uml_xml_template |
702 | - rescue_file = FLAGS.libvirt_rescue_uml_xml_template |
703 | elif FLAGS.libvirt_type == 'xen': |
704 | uri = FLAGS.libvirt_uri or 'xen:///' |
705 | - template_file = FLAGS.libvirt_xen_xml_template |
706 | - rescue_file = FLAGS.libvirt_rescue_xen_xml_template |
707 | else: |
708 | uri = FLAGS.libvirt_uri or 'qemu:///system' |
709 | - template_file = FLAGS.libvirt_xml_template |
710 | - rescue_file = FLAGS.libvirt_rescue_xml_template |
711 | - return uri, template_file, rescue_file |
712 | + return uri |
713 | |
714 | def _connect(self, uri, read_only): |
715 | auth = [[libvirt.VIR_CRED_AUTHNAME, libvirt.VIR_CRED_NOECHOPROMPT], |
716 | @@ -442,18 +412,28 @@ |
717 | if not os.path.exists(basepath('disk')): |
718 | images.fetch(inst.image_id, basepath('disk-raw'), user, |
719 | project) |
720 | - if not os.path.exists(basepath('kernel')): |
721 | - images.fetch(inst.kernel_id, basepath('kernel'), user, |
722 | - project) |
723 | - if not os.path.exists(basepath('ramdisk')): |
724 | - images.fetch(inst.ramdisk_id, basepath('ramdisk'), user, |
725 | - project) |
726 | + |
727 | + if inst['kernel_id']: |
728 | + if not os.path.exists(basepath('kernel')): |
729 | + images.fetch(inst['kernel_id'], basepath('kernel'), |
730 | + user, project) |
731 | + if inst['ramdisk_id']: |
732 | + if not os.path.exists(basepath('ramdisk')): |
733 | + images.fetch(inst['ramdisk_id'], basepath('ramdisk'), |
734 | + user, project) |
735 | |
736 | def execute(cmd, process_input=None, check_exit_code=True): |
737 | return utils.execute(cmd=cmd, |
738 | process_input=process_input, |
739 | check_exit_code=check_exit_code) |
740 | |
741 | + # For now, we assume that if we're not using a kernel, we're using a |
742 | + # partitioned disk image where the target partition is the first |
743 | + # partition |
744 | + target_partition = None |
745 | + if not inst['kernel_id']: |
746 | + target_partition = "1" |
747 | + |
748 | key = str(inst['key_data']) |
749 | net = None |
750 | network_ref = db.network_get_by_instance(context.get_admin_context(), |
751 | @@ -473,12 +453,20 @@ |
752 | inst['name'], inst.image_id) |
753 | if net: |
754 | logging.info('instance %s: injecting net into image %s', |
755 | - inst['name'], inst.image_id) |
756 | - disk.inject_data(basepath('disk-raw'), key, net, |
757 | - execute=execute) |
758 | + inst['name'], inst.image_id) |
759 | + try: |
760 | + disk.inject_data(basepath('disk-raw'), key, net, |
761 | + partition=target_partition, |
762 | + execute=execute) |
763 | + except Exception as e: |
764 | + # This could be a windows image, or a vmdk format disk |
765 | + logging.warn('instance %s: ignoring error injecting data' |
766 | + ' into image %s (%s)', |
767 | + inst['name'], inst.image_id, e) |
768 | |
769 | - if os.path.exists(basepath('disk')): |
770 | - utils.execute('rm -f %s' % basepath('disk')) |
771 | + if inst['kernel_id']: |
772 | + if os.path.exists(basepath('disk')): |
773 | + utils.execute('rm -f %s' % basepath('disk')) |
774 | |
775 | local_bytes = (instance_types.INSTANCE_TYPES[inst.instance_type] |
776 | ['local_gb'] |
777 | @@ -487,8 +475,13 @@ |
778 | resize = True |
779 | if inst['instance_type'] == 'm1.tiny' or prefix == 'rescue-': |
780 | resize = False |
781 | - disk.partition(basepath('disk-raw'), basepath('disk'), |
782 | - local_bytes, resize, execute=execute) |
783 | + |
784 | + if inst['kernel_id']: |
785 | + disk.partition(basepath('disk-raw'), basepath('disk'), |
786 | + local_bytes, resize, execute=execute) |
787 | + else: |
788 | + os.rename(basepath('disk-raw'), basepath('disk')) |
789 | + disk.extend(basepath('disk'), local_bytes, execute=execute) |
790 | |
791 | if FLAGS.libvirt_type == 'uml': |
792 | utils.execute('sudo chown root %s' % basepath('disk')) |
793 | @@ -514,14 +507,21 @@ |
794 | 'bridge_name': network['bridge'], |
795 | 'mac_address': instance['mac_address'], |
796 | 'ip_address': ip_address, |
797 | - 'dhcp_server': dhcp_server} |
798 | - if rescue: |
799 | - libvirt_xml = self.rescue_xml % xml_info |
800 | - else: |
801 | - libvirt_xml = self.libvirt_xml % xml_info |
802 | + 'dhcp_server': dhcp_server, |
803 | + 'rescue': rescue} |
804 | + if not rescue: |
805 | + if instance['kernel_id']: |
806 | + xml_info['kernel'] = xml_info['basepath'] + "/kernel" |
807 | + |
808 | + if instance['ramdisk_id']: |
809 | + xml_info['ramdisk'] = xml_info['basepath'] + "/ramdisk" |
810 | + |
811 | + xml_info['disk'] = xml_info['basepath'] + "/disk" |
812 | + |
813 | + xml = str(Template(self.libvirt_xml, searchList=[xml_info])) |
814 | logging.debug('instance %s: finished toXML method', instance['name']) |
815 | |
816 | - return libvirt_xml |
817 | + return xml |
818 | |
819 | def get_info(self, instance_name): |
820 | try: |
821 | |
822 | === modified file 'tools/pip-requires' |
823 | --- tools/pip-requires 2010-12-13 17:02:27 +0000 |
824 | +++ tools/pip-requires 2010-12-20 21:55:17 +0000 |
825 | @@ -2,6 +2,7 @@ |
826 | pep8==0.5.0 |
827 | pylint==0.19 |
828 | IPy==0.70 |
829 | +Cheetah==2.4.2.1 |
830 | M2Crypto==0.20.2 |
831 | amqplib==0.6.1 |
832 | anyjson==0.2.4 |
Like the new templates.