File verification uses lots of memory
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu system image |
Fix Released
|
High
|
Barry Warsaw | ||
ubuntu-download-manager |
Fix Released
|
High
|
Manuel de la Peña | ||
ubuntu-download-manager (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
The file verification after downloading and checking the GPG signature of a file needs a lot of memory.
The code looks like the whole file is read into memory to calculate the sha256 hash, using more than 250MB of RAM in my case (while only ~100M are free on this ported device). This causes a lot of swapping and a few minutes long freeze. In one out of three attempts, system-settings closed during the freeze (oom kill?) and I had to restart the verification.
In two places in state.py, hashlib is used like this :
# Verify the checksums.
for dst, checksum in checksums:
This could be changed to something like this (completely untested!):
# Verify the checksums.
for dst, checksum in checksums:
Related branches
- Diego Sarmentero (community): Approve
- Roberto Alsina (community): Approve
- PS Jenkins bot: Approve (continuous-integration)
-
Diff: 49 lines (+12/-5)3 files modifiedubuntu-download-manager-priv/downloads/file_download.cpp (+5/-5)
ubuntu-download-manager-priv/system/file_manager.cpp (+6/-0)
ubuntu-download-manager-priv/system/file_manager.h (+1/-0)
- Barry Warsaw (community): Approve
-
Diff: 157 lines (+58/-10)5 files modifiedREADME (+15/-0)
systemimage/gpg.py (+8/-5)
systemimage/helpers.py (+15/-0)
systemimage/state.py (+3/-3)
systemimage/tests/test_helpers.py (+17/-2)
- Registry Administrators: Pending requested
-
Diff: 7348 lines (+4720/-717)50 files modifiedNEWS.rst (+36/-0)
PKG-INFO (+1/-1)
README.rst (+71/-0)
cli-manpage.rst (+33/-2)
dbus-manpage.rst (+255/-2)
debian/changelog (+52/-0)
debian/control (+2/-0)
debian/rules (+11/-25)
debian/tests/control (+6/-1)
debian/tests/dryrun (+11/-0)
debian/tests/smoketest (+4/-2)
debian/upstream/signing-key.asc (+2459/-0)
debian/watch (+1/-1)
ini-manpage.rst (+2/-2)
setup.cfg (+2/-2)
system_image.egg-info/PKG-INFO (+1/-1)
system_image.egg-info/SOURCES.txt (+2/-0)
system_image.egg-info/entry_points.txt (+1/-1)
system_image.egg-info/requires.txt (+1/-1)
systemimage/api.py (+6/-5)
systemimage/bag.py (+13/-0)
systemimage/bindings.py (+15/-10)
systemimage/config.py (+4/-3)
systemimage/dbus.py (+25/-11)
systemimage/download.py (+87/-21)
systemimage/gpg.py (+5/-5)
systemimage/helpers.py (+33/-6)
systemimage/logging.py (+18/-11)
systemimage/main.py (+104/-2)
systemimage/reboot.py (+16/-1)
systemimage/scores.py (+1/-1)
systemimage/settings.py (+11/-0)
systemimage/state.py (+16/-16)
systemimage/testing/controller.py (+10/-8)
systemimage/testing/dbus.py (+31/-62)
systemimage/testing/helpers.py (+30/-6)
systemimage/tests/data/index_25.json (+22/-0)
systemimage/tests/test_api.py (+36/-21)
systemimage/tests/test_bag.py (+19/-0)
systemimage/tests/test_channel.py (+2/-6)
systemimage/tests/test_config.py (+19/-28)
systemimage/tests/test_dbus.py (+373/-324)
systemimage/tests/test_download.py (+234/-29)
systemimage/tests/test_helpers.py (+45/-5)
systemimage/tests/test_main.py (+497/-50)
systemimage/tests/test_settings.py (+29/-2)
systemimage/tests/test_state.py (+61/-40)
systemimage/tests/test_winner.py (+3/-1)
systemimage/version.txt (+1/-1)
tox.ini (+3/-1)
- Registry Administrators: Pending requested
-
Diff: 4058 lines (+3524/-347) (has conflicts)35 files modified.bzrignore (+0/-10)
PKG-INFO (+10/-0)
README.rst (+71/-0)
debian/archive-master.tar.xz.asc (+17/-0)
debian/changelog (+468/-0)
debian/compat (+1/-0)
debian/control (+62/-0)
debian/copyright (+35/-0)
debian/rules (+64/-0)
debian/source/format (+1/-0)
debian/source/include-binaries (+1/-0)
debian/system-image-common.dirs (+2/-0)
debian/system-image-common.manpages (+3/-0)
debian/system-image-common.postinst (+23/-0)
debian/system-image-common.postrm (+18/-0)
debian/tests/client.ini.in (+34/-0)
debian/tests/control (+7/-0)
debian/tests/dryrun (+11/-0)
debian/tests/prep.py (+22/-0)
debian/tests/smoketest (+13/-0)
debian/tests/unittests (+11/-0)
debian/upstream/signing-key.asc (+2459/-0)
debian/watch (+2/-0)
setup.cfg (+9/-3)
system_image.egg-info/PKG-INFO (+10/-0)
system_image.egg-info/SOURCES.txt (+141/-0)
system_image.egg-info/dependency_links.txt (+1/-0)
system_image.egg-info/entry_points.txt (+4/-0)
system_image.egg-info/requires.txt (+1/-0)
system_image.egg-info/top_level.txt (+1/-0)
systemimage/docs/Makefile (+0/-153)
systemimage/tests/data/index_25.json (+22/-0)
tools/heregpg (+0/-7)
tools/mkvenv.py (+0/-121)
tools/sd.py (+0/-53)
Changed in ubuntu-download-manager: | |
status: | New → In Progress |
importance: | Undecided → High |
assignee: | nobody → Manuel de la Peña (mandel) |
Changed in ubuntu-download-manager: | |
milestone: | none → 0.4 |
Changed in ubuntu-download-manager: | |
status: | Fix Committed → Fix Released |
tags: | added: client |
Changed in ubuntu-system-image: | |
status: | New → Triaged |
importance: | Undecided → High |
assignee: | nobody → Barry Warsaw (barry) |
milestone: | none → 2.3 |
Changed in ubuntu-system-image: | |
status: | Triaged → In Progress |
Changed in ubuntu-system-image: | |
status: | In Progress → Fix Committed |
Changed in ubuntu-system-image: | |
status: | Fix Committed → Fix Released |
Adding this to udm because it does a similar thing.