apport bugs from official raspi or riscv images are not identified

This bug was fixed in the package apport - 2.20.11-0ubuntu61

---------------
apport (2.20.11-0ubuntu61) hirsute; urgency=medium

  * apport/hookutils.py: when creating KernLog with MAC information use
    root_command_output with dmesg. Missed when fixing LP #1896095.
  * data/general-hooks/ubuntu.py: tag bugs from Raspberry Pi images and RISCV
    images appropriately. (LP: #1920837)

 -- Brian Murray <email address hidden> Wed, 24 Mar 2021 08:07:43 -0700

I just looked at the change and I have a question regarding it. I don't know the apport code-base so I'd like to confirm something: right now I see you're keying on 'ImageMediaBuild' being provided (is it only on preinstalled media?) and then adding the 'raspi-image' tag when the arch is arm64 or armhf. Is this raspberry-pi specific? What I mean is, what if we start building some preinstalled non-pi arm images? Will those then get tagged as raspi-image as well then?

I know that currently we don't build any arm preinstalled images like these, but this feels to me like something we might forget changing in the future and then start seeing bunch of unrelated bugs tagged as such. Is there no way we could check for raspi subarch? Or is this the best way?

The most reliable means I know of at the moment for detecting Raspberry Pi hardware at runtime is to query the device-tree.

For example, flash-kernel determines the type of board it is running on, by matching the content of /proc/device-tree/model (e.g. "Raspberry Pi 400 Rev 1.0") against a pre-determined list of strings in its database. Personally I don't much like matching fixed strings -- new board revisions frequently come out and then f-k needs patching to add them, but last cycle I did add the ability to match the "Raspberry Pi *" glob as an attempt to support new boards even if they come out mid-cycle. Something similar could be accomplished with some Python like so:

try:
    with open('/proc/device-tree/model', 'r', encoding='ascii') as f:
        is_a_pi = f.read().startswith('Raspberry Pi ')
except FileNotFoundError:
    is_a_pi = False

That seems to work reliably (even on compute modules where the model is "Raspberry Pi Compute Module 4 Rev 1.0"), but I've no idea if the /proc/device-tree/model file *always* exists or whether other boards publish useful information in there.

Another possibility is to look at /proc/device-tree/compatible which I *think* should always exist assuming a device-tree is in use on the system. This is a binary file containing a sequence of NUL-terminated strings, each of which is of the form "vendor,model" and is intended for kernel driver matching. For example, on the Pi 400 it contains:

raspberrypi,400\0brcm,bcm2711\0

(where \0 is the NUL character). And on a 3B+ contains:

raspberrypi,3-model-b-plus\0brcm,bcm2837\0

My understanding is there's no strict requirement that these strings appear in any particular order so simple prefix matching is potentially unreliable. However, it's trivial enough to parse and extract the "raspberrypi" vendor from this with some Python like:

try:
    with open('/proc/device-tree/compatible', 'rb') as f:
        is_a_pi = any(vendor == 'raspberrypi'
                      for s in f.read().split(b'\0') if s
                      for vendor, model in (s.decode('ascii').split(',', 1),))
except FileNotFoundError:
    is_a_pi = False

Something similar *should* be reasonably useful with just about any board that uses a device tree (but obviously PCs don't have device-trees so we still need the file-not-found check).

A couple of thoughts:

1)

In addition to future supported platforms, what about weird third party hacked up things? Are we going to end up with reports from users who have grabbed a custom image for some esoteric platform from someone who is hacking something together for that platform by basing their work on one of our official Raspberry Pi images? To be clear, they're welcome to do that; we just don't want images mislabelled as Raspberry Pi causing confusion in that case. Of course people can do that regardless of what apport does, but labelling them as Raspberry Pi would exacerbate the issue.

2)

+ if 'ImageMediaBuild' in report:
+ if report['Architecture'] in ('arm64', 'armhf'):
+ add_tag(report, 'raspi-image')

Why not simply add tags arm64-image and armhf-image? Then you'll get the same effect you want but without a potentially misleading tag. This may be kicking the can down the road, but maybe that's exactly what we want to do here?

This should presumably be settled in the development release before the SRU proceeds, so I'm deferring SRU review for now.

> labelling them as Raspberry Pi would exacerbate the issue

Sorry, I should have acknowledged that Dave's proposal would avoid this issue.

I've gone ahead and went with the /proc/device-tree/compatible - thanks for the patch Dave!

Hello Brian, or anyone else affected,

Accepted apport into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.11-0ubuntu50.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Brian, or anyone else affected,

Accepted apport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.11-0ubuntu27.17 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted apport (2.20.11-0ubuntu27.17) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.11-0ubuntu27.17 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#apport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

The verification passed for focal:

ubuntu@ubuntu:~$ apt-cache policy apport
apport:
  Installed: 2.20.11-0ubuntu27.16
  Candidate: 2.20.11-0ubuntu27.17
  Version table:
     2.20.11-0ubuntu27.17 500
        500 http://ports.ubuntu.com/ubuntu-ports focal-proposed/main armhf Packages
 *** 2.20.11-0ubuntu27.16 500
        500 http://ports.ubuntu.com/ubuntu-ports focal-updates/main armhf Packages
        500 http://ports.ubuntu.com/ubuntu-ports focal-security/main armhf Packages
        100 /var/lib/dpkg/status
     2.20.11-0ubuntu27 500
        500 http://ports.ubuntu.com/ubuntu-ports focal/main armhf Packages

`ubuntu-bug apport` shows:
== Tags =================================
 focal uec-images

ubuntu@ubuntu:~$ sudo apt install apport
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  python3-apport
Suggested packages:
  apport-gtk | apport-kde
The following packages will be upgraded:
  apport python3-apport
2 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
Need to get 214 kB of archives.
After this operation, 1024 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ports.ubuntu.com/ubuntu-ports focal-proposed/main armhf python3-apport all 2.20.11-0ubuntu27.17 [85.2 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-proposed/main armhf apport all 2.20.11-0ubuntu27.17 [129 kB]
Fetched 214 kB in 1s (247 kB/s)
(Reading database ... 99636 files and directories currently installed.)
Preparing to unpack .../python3-apport_2.20.11-0ubuntu27.17_all.deb ...
Unpacking python3-apport (2.20.11-0ubuntu27.17) over (2.20.11-0ubuntu27.16) ...
Preparing to unpack .../apport_2.20.11-0ubuntu27.17_all.deb ...
Unpacking apport (2.20.11-0ubuntu27.17) over (2.20.11-0ubuntu27.16) ...
Setting up python3-apport (2.20.11-0ubuntu27.17) ...
Setting up apport (2.20.11-0ubuntu27.17) ...
apport-autoreport.service is a disabled or a static unit, not starting it.
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.6) ...

`ubuntu-bug apport` shows
== Tags =================================
 armhf-image raspi-image focal uec-images package-from-proposed

The verification passed for groovy:

ubuntu@ubuntu:~$ apt-cache policy apport
apport:
  Installed: 2.20.11-0ubuntu50.5
  Candidate: 2.20.11-0ubuntu50.6
  Version table:
     2.20.11-0ubuntu50.6 500
        500 http://ports.ubuntu.com/ubuntu-ports groovy-proposed/main arm64 Packages
 *** 2.20.11-0ubuntu50.5 500
        500 http://ports.ubuntu.com/ubuntu-ports groovy-updates/main arm64 Packages
        500 http://ports.ubuntu.com/ubuntu-ports groovy-security/main arm64 Packages
        100 /var/lib/dpkg/status
     2.20.11-0ubuntu50 500
        500 http://ports.ubuntu.com/ubuntu-ports groovy/main arm64 Packages

`ubuntu-bug apport` shows
== Tags =================================
 groovy uec-images

ubuntu@ubuntu:~$ sudo apt install apport
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  python3-apport
Suggested packages:
  apport-gtk | apport-kde
The following packages will be upgraded:
  apport python3-apport
2 upgraded, 0 newly installed, 0 to remove and 20 not upgraded.
Need to get 214 kB of archives.
After this operation, 1024 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ports.ubuntu.com/ubuntu-ports groovy-proposed/main arm64 python3-apport all 2.20.11-0ubuntu50.6 [85.2 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports groovy-proposed/main arm64 apport all 2.20.11-0ubuntu50.6 [129 kB]
Fetched 214 kB in 1s (258 kB/s)
(Reading database ... 101926 files and directories currently installed.)
Preparing to unpack .../python3-apport_2.20.11-0ubuntu50.6_all.deb ...
Unpacking python3-apport (2.20.11-0ubuntu50.6) over (2.20.11-0ubuntu50.5) ...
Preparing to unpack .../apport_2.20.11-0ubuntu50.6_all.deb ...
Unpacking apport (2.20.11-0ubuntu50.6) over (2.20.11-0ubuntu50.5) ...
Setting up python3-apport (2.20.11-0ubuntu50.6) ...
Setting up apport (2.20.11-0ubuntu50.6) ...
apport-autoreport.service is a disabled or a static unit, not starting it.
Processing triggers for man-db (2.9.3-2) ...
Processing triggers for systemd (246.6-1ubuntu1.3) ...

`ubuntu-bug apport` shows
== Tags =================================
 arm64-image raspi-image groovy uec-images package-from-proposed

All autopkgtests for the newly accepted apport (2.20.11-0ubuntu50.6) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-release-upgrader/1:20.10.15 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#apport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

The verification of the Stable Release Update for apport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package apport - 2.20.11-0ubuntu50.6

---------------
apport (2.20.11-0ubuntu50.6) groovy; urgency=medium

  * data/general-hooks/ubuntu.py: tag bugs from Raspberry Pi images and RISCV
    images appropriately. (LP: #1920837)
  * apport/hookutils.py: spawn pkttyagent so that log files can be gathered as
    root in a non-graphical environment (LP: #1821415). Thanks to Iain Lane
    for the patch.
  * apport/hookutils.py: root access is needed to read the
    casper-md5check.json file so switch to using that. (LP: #1922937)

 -- Brian Murray <email address hidden> Mon, 26 Apr 2021 12:45:36 -0700

This bug was fixed in the package apport - 2.20.11-0ubuntu27.17

---------------
apport (2.20.11-0ubuntu27.17) focal; urgency=medium

  * data/general-hooks/ubuntu.py: tag bugs from Raspberry Pi images and RISCV
    images appropriately. (LP: #1920837)
  * apport/hookutils.py: spawn pkttyagent so that log files can be gathered as
    root in a non-graphical environment (LP: #1821415). Thanks to Iain Lane
    for the patch.
  * apport/hookutils.py: root access is needed to read the
    casper-md5check.json file so switch to using that. (LP: #1922937)

 -- Brian Murray <email address hidden> Mon, 26 Apr 2021 13:28:49 -0700