QA Regression Testing

Merge ~federicoquattrin/qa-regression-testing:add_cryptojs_tests into qa-regression-testing:master

  1. Git
  2. lp:~federicoquattrin/qa-regression-testing
  3. add_cryptojs_tests
  4. Merge into master
Proposed by Federico Quattrin
Status: Merged
Merged at revision: 768d65284c17bf966d28e3a5052007410cbc30bf
Proposed branch: ~federicoquattrin/qa-regression-testing:add_cryptojs_tests
Merge into: qa-regression-testing:master
Diff against target: 304 lines (+264/-1)
4 files modified
.launchpad.yaml (+16/-0)
scripts/cryptojs/cryptojs_index.html (+56/-0)
scripts/test-cryptojs.py (+187/-0)
scripts/testlib.py (+5/-1)
Reviewer Review Type Date Requested Status
Steve Beattie Approve
Emilia Torino Needs Fixing
Review via email: mp+465083@code.launchpad.net

Commit message

added cryptojs tests

Description of the change

added cryptojs tests

To post a comment you must log in.
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

I think you have a copy of the test script in data/ also by mistake?

Revision history for this message
Federico Quattrin (federicoquattrin) wrote (last edit ):

> I think you have a copy of the test script in data/ also by mistake?

I added it there because I'm not sure how the structure should be followed.

We can remove it from the data directory if that's OK since my script doesn't use it from there.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Yes, please remove it, the data directory is for sample files that are used by multiple tests, like images and such. scripts/cryptojs/cryptojs_index.html is definitely the right place for that.

Revision history for this message
Emilia Torino (emitorino) wrote :

Thanks for the script addition! LGTM, I only added some minor suggestions.

review: Needs Fixing
Revision history for this message
Emilia Torino (emitorino) wrote :

Thanks for including the changes! LGTM! Thanks!

The only other thing to consider, is to add this tests as part of the lpci config (.launchpad.yaml).

Please see a recent related MP https://code.launchpad.net/~sbeattie/qa-regression-testing/+git/qa-regression-testing/+merge/465105

Revision history for this message
Steve Beattie (sbeattie) wrote :

Hey Federico, thanks for doing this, it's really appreciated.

Some feedback inline, reworking the tests to make them independent is probably too much to ask here, but I'd like to see the others addressed before landing.

Thanks!

review: Needs Fixing
Revision history for this message
Steve Beattie (sbeattie) wrote :

Oh, one more comment inline:

Revision history for this message
Steve Beattie (sbeattie) wrote :

 review approve

Thanks for fixing up the requested fixes. Merging.

--
Steve Beattie
<email address hidden>

review: Approve
Revision history for this message
Steve Beattie (sbeattie) wrote :

Oh, I just noticed the cryptojs wasn't showing up in the list of tests being run by lpic -- I missed that the the cryptojs job didn't get addded to the set of jobs to be run in the pipeline; see https://git.launchpad.net/qa-regression-testing/tree/.launchpad.yaml#n1

I'll add a fixup commit to add it.

Thanks!

Revision history for this message
Steve Beattie (sbeattie) wrote :

Pushed https://git.launchpad.net/qa-regression-testing/commit/?id=93fa4c2c6aef3f4883b1553b5900f4e3f0220ce8

Revision history for this message
Emilia Torino (emitorino) wrote :

Thanks for https://git.launchpad.net/qa-regression-testing/commit/?id=93fa4c2c6aef3f4883b1553b5900f4e3f0220ce8 Steve!

Fede had this as a pending change to add to this MP. I am trying to recall if there was something else also pending to add, Fede can you please check this?

Thanks!

Revision history for this message
Steve Beattie (sbeattie) wrote :

On Mon, May 06, 2024 at 03:08:02PM -0000, Emilia Torino wrote:
> Thanks for
> https://git.launchpad.net/qa-regression-testing/commit/?id=93fa4c2c6aef3f4883b1553b5900f4e3f0220ce8
> Steve!
>
> Fede had this as a pending change to add to this MP. I am trying to
> recall if there was something else also pending to add, Fede can you
> please check this?

Also, it seems that the headless run of firefox is not able to find the
profile directory when running in lpci; it's possible the home directory
is in an alternate location there.

See
https://launchpadlibrarian.net/728203270/buildlog_ci_qa-regression-testing_2c498d3c528918a3fb513d1c1d6108b2c21fd9e4_BUILDING.txt.gz
for an example of a failing lpci job.

--
Steve Beattie
<email address hidden>

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/.launchpad.yaml b/.launchpad.yaml
2index a32b91d..d4378ef 100644
3--- a/.launchpad.yaml
4+++ b/.launchpad.yaml
5@@ -226,3 +226,19 @@ jobs:
6 DEBIAN_FRONTEND=noninteractive apt upgrade --assume-yes
7 run: |
8 ./lpcraft-runner amanda
9+ cryptojs:
10+ matrix:
11+ - series: jammy
12+ architectures: amd64
13+ - series: focal
14+ architectures: amd64
15+ - series: bionic
16+ architectures: amd64
17+ - series: xenial
18+ architectures: amd64
19+ packages:
20+ - sudo
21+ run-before: |
22+ DEBIAN_FRONTEND=noninteractive apt upgrade --assume-yes
23+ run: |
24+ ./lpcraft-runner cryptojs
25\ No newline at end of file
26diff --git a/scripts/cryptojs/cryptojs_index.html b/scripts/cryptojs/cryptojs_index.html
27new file mode 100644
28index 0000000..c645048
29--- /dev/null
30+++ b/scripts/cryptojs/cryptojs_index.html
31@@ -0,0 +1,56 @@
32+<!DOCTYPE html>
33+<html lang="en">
34+<head>
35+ <meta charset="UTF-8">
36+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
37+ <title>Output Display</title>
38+</head>
39+<body>
40+ <div id="output"></div>
41+ <script src="/usr/share/javascript/cryptojs/components/core.js"></script>
42+ <script src="/usr/share/javascript/cryptojs/components/sha1.js"></script>
43+ <script src="/usr/share/javascript/cryptojs/components/sha256.js"></script>
44+ <script src="/usr/share/javascript/cryptojs/components/hmac.js"></script>
45+ <script src="/usr/share/javascript/cryptojs/components/pbkdf2.js"></script>
46+ <script>
47+ var salt = "ATHENA.MIT.EDUraeburn";
48+
49+ // Test 1
50+ // Replicates unit test from upstream: testKeySize256
51+ // Y.Assert.areEqual('262fb72ea65b44ab5ceba7f8c8bfa7815ff9939204eb7357a59a75877d745777', C.PBKDF2('password', 'ATHENA.MIT.EDUraeburn', { keySize: 256/32, iterations: 2 }).toString());
52+ // We only use two iterations instead of 250,000 to save time.
53+ var testKeySize256 = CryptoJS.PBKDF2('password', salt, { keySize: 256/32, iterations: 2 }).toString();
54+
55+ // Test 2
56+ // Replicates unit test from upstream: testKeySize256Iterations5
57+ // Y.Assert.areEqual('74e98b2e9eeddaab3113c1efc6d82b073c4860195b3e0737fa21a4778f376321', C.PBKDF2('password', C.enc.Hex.parse('1234567878563412'), { keySize: 256/32, iterations: 5 }).toString());
58+ // We only use five iterations instead of 250,000 to save time.
59+ var testKeySize256Iterations5 = CryptoJS.PBKDF2('password', CryptoJS.enc.Hex.parse('1234567878563412'), { keySize: 256/32, iterations: 5 }).toString();
60+
61+ // Test 3
62+ // Replicates unit test from upstream: testKeySize256Iterations1200PassPhraseEqualsBlockSize
63+ // Y.Assert.areEqual('c1dfb29a4d2f2fb67c6f78d074d663671e6fd4da1e598572b1fecf256cb7cf61', C.PBKDF2('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'pass phrase equals block size', { keySize: 256/32, iterations: 1200 }).toString());
64+ // We only use 1200 iterations instead of 250,000 to save time.
65+ var testKeySize256Iterations1200PassPhraseEqualsBlockSize = CryptoJS.PBKDF2('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'pass phrase equals block size', { keySize: 256/32, iterations: 1200 }).toString();
66+
67+ var output = {
68+ testKeySize256: testKeySize256,
69+ testKeySize256Iterations5: testKeySize256Iterations5,
70+ testKeySize256Iterations1200PassPhraseEqualsBlockSize: testKeySize256Iterations1200PassPhraseEqualsBlockSize
71+ };
72+
73+ var jsonOutput = JSON.stringify(output);
74+
75+ document.getElementById('output').innerText = jsonOutput;
76+
77+ var blob = new Blob([jsonOutput], {type: 'application/json'});
78+ var anchor = document.createElement('a');
79+ anchor.download = 'cryptoJS_test_output.json';
80+ anchor.href = window.URL.createObjectURL(blob);
81+ anchor.style.display = 'none';
82+ document.body.appendChild(anchor);
83+ anchor.click();
84+ document.body.removeChild(anchor);
85+ </script>
86+</body>
87+</html>
88diff --git a/scripts/test-cryptojs.py b/scripts/test-cryptojs.py
89new file mode 100755
90index 0000000..900d3d6
91--- /dev/null
92+++ b/scripts/test-cryptojs.py
93@@ -0,0 +1,187 @@
94+#!/usr/bin/python3
95+#
96+# test-PKG.py quality assurance test script for PKG
97+# Copyright (C) 2012 Canonical Ltd.
98+# Author: Federico Quattrin <federico.quattrin@canonical.com>
99+#
100+# This program is free software: you can redistribute it and/or modify
101+# it under the terms of the GNU General Public License version 3,
102+# as published by the Free Software Foundation.
103+#
104+# This program is distributed in the hope that it will be useful,
105+# but WITHOUT ANY WARRANTY; without even the implied warranty of
106+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
107+# GNU General Public License for more details.
108+#
109+# You should have received a copy of the GNU General Public License
110+# along with this program. If not, see <http://www.gnu.org/licenses/>.
111+#
112+# QRT-Packages: libjs-cryptojs
113+# QRT-Depends: cryptojs/cryptojs_index.html
114+
115+'''
116+ In general, this test should be run in a virtual machine (VM) or possibly
117+ a chroot and not on a production machine. While efforts are made to make
118+ these tests non-destructive, there is no guarantee this script will not
119+ alter the machine. You have been warned.
120+
121+ How to run in a clean VM:
122+ $ ./make-test-tarball test-<script>.py # creates tarball in /tmp/
123+ $ scp /tmp/qrt-test-<script>.tar.gz root@vm.host:/tmp
124+ on VM:
125+ # cd /tmp ; tar zxvf ./qrt-test-<script>.tar.gz
126+ # cd /tmp/qrt-test-<script> ; ./install-packages ./test-<script>.py
127+ # ./test-<script>.py -v
128+
129+ To run in all VMs named sec*:
130+ $ vm-qrt -p sec test-<script.py>
131+
132+ ### TODO: update for ./install-packages step ###
133+ How to run in a clean schroot named 'lucid':
134+ $ schroot -c lucid -u root -- sh -c 'apt-get -y install lsb-release <QRT-Packages> && ./test-PKG.py -v'
135+'''
136+
137+from __future__ import print_function
138+
139+import os
140+import subprocess
141+import sys
142+import tempfile
143+import shutil
144+import unittest
145+import testlib
146+import json
147+
148+
149+
150+class CryptoJSTest(testlib.TestlibCase):
151+ '''Test my thing.'''
152+
153+ def setUp(self):
154+ self.home_dir = os.environ.get("HOME")
155+ self._temp_html_file = tempfile.NamedTemporaryFile(prefix="cryptojs_index_", suffix=".html", dir=self.home_dir)
156+ self._output_file = "{}/Downloads/cryptoJS_test_output.json".format(self.home_dir)
157+ self.need_cleanup = False
158+ self.original_value_handlers= None
159+ self.temp_cryptojs_dir_name = None
160+
161+ # Depending on the Distribution and user configuration, the browser can be set up to prompt a dialog
162+ # asking wheter to open or download the file and where to save the file it.
163+ # For this test we need to download the file without user interation and therefore we need
164+ # to configure the browser to download JSON files by default.
165+ self.handlers_json = self._find_handlers_json()
166+ self._modify_handlers_json()
167+
168+
169+ def tearDown(self):
170+ '''Clean up after each test_* function'''
171+ if self._temp_html_file:
172+ print("[!] Cleaning up dropped HTML file {}".format(self._temp_html_file.name))
173+ self._temp_html_file.close()
174+ if os.path.isfile(self._output_file):
175+ print("[!] Cleaning up downloaded JSON file {}".format(self._output_file))
176+ os.remove(self._output_file)
177+ if self.temp_cryptojs_dir_name:
178+ print("[!] Cleaning up stagged cryptojs directory {}".format(self.temp_cryptojs_dir_name.name))
179+ self.temp_cryptojs_dir_name.cleanup()
180+ if not self.need_cleanup:
181+ return
182+ with open(self.handlers_json, "r") as f:
183+ h_json = json.load(f)
184+ if not self.original_value_handlers:
185+ print("[!] Removing key 'application/json' from handlers.json")
186+ h_json["mimeTypes"].pop("application/json")
187+ else:
188+ print("[!] Modifying key 'application/json' in handlers.json")
189+ h_json["mimeTypes"]["application/json"] = self.original_value_handlers
190+ with open(self.handlers_json, "w") as f:
191+ f.write(json.dumps(h_json))
192+
193+ def test_CVE_2023_46233(self):
194+ '''Test thing'''
195+ with open("{}/cryptojs/cryptojs_index.html".format(os.path.dirname(os.path.realpath(__file__))), "r") as index_file:
196+ content = index_file.read()
197+ if self.lsb_release.get("Release") >= 22.04:
198+ # since Jammy firefox comes as snap. Therefore it can't access the /tmp directory. We need to stage the library in other directory.
199+ self.temp_cryptojs_dir_name = tempfile.TemporaryDirectory(prefix="cryptojs_", dir=self.home_dir)
200+ content = content.replace("/usr/share/javascript/cryptojs", self.temp_cryptojs_dir_name.name)
201+ shutil.copytree("/usr/share/javascript/cryptojs/", self.temp_cryptojs_dir_name.name, dirs_exist_ok=True)
202+ # Moving HTML file away from /tmp.
203+ with open(self._temp_html_file.name, "w") as temp_html_file:
204+ temp_html_file.write(content)
205+
206+
207+ self._execute_firefox(self._temp_html_file.name)
208+
209+ try:
210+ with open(self._output_file, "r") as output_file:
211+ output_text = json.load(output_file)
212+ except OSError:
213+ print("It was not possible to download the JSON file.")
214+ output_text = {}
215+
216+ # Replicates upstream Unit Test: testKeySize256
217+ # Y.Assert.areEqual('262fb72ea65b44ab5ceba7f8c8bfa7815ff9939204eb7357a59a75877d745777', C.PBKDF2('password', 'ATHENA.MIT.EDUraeburn', { keySize: 256/32, iterations: 2 }).toString());
218+ self.assertEqual(output_text.get("testKeySize256"), "262fb72ea65b44ab5ceba7f8c8bfa7815ff9939204eb7357a59a75877d745777")
219+
220+
221+ # Replicates upstream Unit Test: testKeySize256Iterations5
222+ # Y.Assert.areEqual('74e98b2e9eeddaab3113c1efc6d82b073c4860195b3e0737fa21a4778f376321', C.PBKDF2('password', C.enc.Hex.parse('1234567878563412'), { keySize: 256/32, iterations: 5 }).toString());
223+ self.assertEqual(output_text.get("testKeySize256Iterations5"), "74e98b2e9eeddaab3113c1efc6d82b073c4860195b3e0737fa21a4778f376321")
224+
225+ # Replicates upstream Unit Test: testKeySize256Iterations1200PassPhraseEqualsBlockSize
226+ # Y.Assert.areEqual('c1dfb29a4d2f2fb67c6f78d074d663671e6fd4da1e598572b1fecf256cb7cf61', C.PBKDF2('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'pass phrase equals block size', { keySize: 256/32, iterations: 1200 }).toString());
227+ self.assertEqual(output_text.get("testKeySize256Iterations1200PassPhraseEqualsBlockSize"), "c1dfb29a4d2f2fb67c6f78d074d663671e6fd4da1e598572b1fecf256cb7cf61")
228+
229+ def _execute_firefox(self, url):
230+ try:
231+ print("[*] Executing Firefox with arguments: ", url)
232+ testlib.cmd(["firefox", "-headless", url], timeout=10)
233+ except subprocess.TimeoutExpired:
234+ pass
235+
236+ def _modify_handlers_json(self):
237+ # we need to modify this to make the browser automatically download the file.
238+ with open(self.handlers_json, "r") as f:
239+ h_json = json.load(f)
240+ if not h_json.get("mimeTypes").get("application/json"):
241+ print("[*] Adding the key 'application/json' to handlers.json")
242+ h_json["mimeTypes"]["application/json"] = {
243+ "action": 0,
244+ "extensions": ["json"]
245+ }
246+ self.original_value_handlers = None
247+ self.need_cleanup = True
248+ elif h_json.get("mimeTypes").get("application/json").get("action") == 0 and \
249+ "json" in h_json.get("mimeTypes").get("application/json").get("extensions"):
250+ print("[*] handlers.json is set as required")
251+ self.need_cleanup = False
252+ return
253+ else:
254+ print("[*] Modifying the key 'application/json' in handlers.json")
255+ self.original_value_handlers = h_json.get("mimeTypes").get("application/json").copy()
256+ self.need_cleanup = True
257+ h_json["mimeTypes"]["application/json"] = {
258+ "action": 0,
259+ "extensions": ["json"]
260+ }
261+ with open(self.handlers_json, "w") as f:
262+ f.write(json.dumps(h_json))
263+
264+ def _find_handlers_json(self):
265+ # Find handlers.json to make json download automatically without asking
266+ for p in ["{}/.mozilla/firefox".format(self.home_dir), "{}/snap/firefox/common/.mozilla/firefox".format(self.home_dir)]:
267+ print("[*] Searching for handlers.json in {}".format(p))
268+ output = subprocess.run(["find", p, "-name", "handlers.json"], stdout=subprocess.PIPE)
269+ if output.stdout.decode().split():
270+ print("[!] Found {}".format(output.stdout.decode().split()[0]))
271+ return output.stdout.decode().split()[0]
272+ print("[!] handlers.json has not been found. This might be because you don't have a firefox profile.\n"
273+ "Executing Firefox to create a profile and retrying...")
274+
275+ self._execute_firefox("https://ubuntu.com/")
276+ return self._find_handlers_json()
277+
278+
279+if __name__ == '__main__':
280+ unittest.main()
281diff --git a/scripts/testlib.py b/scripts/testlib.py
282index a91ff5b..c99eee2 100644
283--- a/scripts/testlib.py
284+++ b/scripts/testlib.py
285@@ -471,7 +471,7 @@ def ubuntu_release():
286 def cmd(command, input=None, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, stdin=None, timeout=None, env=None, shell=False, text=True):
287 '''Try to execute given command (array) and return its stdout, or return
288 a textual error if it failed.'''
289-
290+ out = outerr = None
291 try:
292 sp = subprocess.Popen(command, stdin=stdin, stdout=stdout, stderr=stderr, close_fds=True, preexec_fn=subprocess_setup, env=env, universal_newlines=text, shell=shell)
293 if sys.version_info[0] >= 3 and sys.version_info[1] > 3:
294@@ -480,6 +480,10 @@ def cmd(command, input=None, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, s
295 out, outerr = sp.communicate(input)
296 except OSError as e:
297 return [127, str(e)]
298+ except subprocess.TimeoutExpired as e:
299+ sp.kill()
300+ sp.communicate(input)
301+
302
303 # Handle redirection of stdout
304 if out is None:

Subscribers

People subscribed via source and target branches