MAAS

[2.2] MAAS region controller unable to update NTP

Bug #1650481 reported by Peter teStrake
34
This bug affects 6 people
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
Critical
Gavin Panella
MAAS 2.2.0
2.1
Won't Fix
Critical
Gavin Panella
MAAS 2.1.3

Bug Description

Hi,
I get this error in /ver/log/maas/regiond.log

2016-12-16 08:21:54 maasserver.regiondservices.ntp: [critical] Failed to update NTP configuration.
 Traceback (most recent call last):
   File "/usr/lib/python3.5/threading.py", line 862, in run
     self._target(*self._args, **self._kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 824, in worker
     return target()
   File "/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 46, in work
     task()
   File "/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 190, in doWork
     task()
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
     result = inContext.theWork()
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
     inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 857, in callInContext
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 225, in wrapper
     result = func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/ntp/config.py", line 49, in configure
     mode=0o644)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/fs.py", line 281, in sudo_write_file
     raise ExternalProcessError(proc.returncode, command, stderr)
 provisioningserver.utils.shell.ExternalProcessError: Command `maas-rack atomic-write --filename /etc/ntp/maas.conf --mode 0644` returned non-zero exit status 1:
 None

It appears that the region controller is trying to update ntp, but it is looking for the maas-rack command line which is not installed ( region and rack controllers are on separate machines ).

On the region controller, the following packages are installed.

root@region:/etc/sudoers.d# dpkg -l | grep maas
ii maas-cli 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS client and command-line interface
ii maas-common 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS server common files
ii maas-dns 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS DNS server
ii maas-proxy 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS Caching Proxy
ii maas-region-api 2.1.2+bzr5555-0ubuntu1~16.04.1 all Region controller API service for MAAS
ii maas-region-controller 2.1.2+bzr5555-0ubuntu1~16.04.1 all Region Controller for MAAS
ii python3-django-maas 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.1.2+bzr5555-0ubuntu1~16.04.1 all MAAS server provisioning libraries (Python 3)
root@region:/etc/sudoers.d#

The MAAS version is MAAS Version 2.1.2+bzr5555-0ubuntu1 (16.04.1)

I can re-produce this problem on a fresh ubuntu install with

apt-add-repository ppa:maas/stable
apt-get install maas-region-conroller
cd var/log/maas
tail -f regiond.log

Thanks
Pete

Related branches

lp:~allenap/maas/atomic-delete-and-write-scripts
Blake Rouse (community): Approve
Diff: 311 lines (+240/-1)
4 files modified
scripts/maas-delete-file (+47/-0)
scripts/maas-write-file (+64/-0)
setup.py (+3/-1)
src/provisioningserver/utils/tests/test_fs.py (+126/-0)
lp:~allenap/maas/packaging.atomic-delete-and-write-scripts
Blake Rouse (community): Approve
Diff: 30 lines (+5/-1)
3 files modified
debian/extras/99-maas-sudoers (+2/-0)
debian/maas-rack-controller.install (+2/-0)
debian/maas-rack-controller.lintian-overrides (+1/-1)
lp:~allenap/maas/use-atomic-delete-and-write-scripts
Blake Rouse (community): Approve
Diff: 283 lines (+115/-21)
4 files modified
scripts/maas-delete-file (+13/-0)
scripts/maas-write-file (+13/-0)
src/provisioningserver/utils/fs.py (+37/-13)
src/provisioningserver/utils/tests/test_fs.py (+52/-8)
lp:~allenap/maas/remove-old-atomic-delete-and-write
Blake Rouse (community): Approve
Diff: 297 lines (+4/-233)
3 files modified
src/provisioningserver/__main__.py (+1/-7)
src/provisioningserver/utils/script.py (+0/-65)
src/provisioningserver/utils/tests/test_script.py (+3/-161)
lp:~allenap/maas/packaging.restrict-maas-rack-sudo-rule
Blake Rouse (community): Approve
Diff: 12 lines (+1/-1)
1 file modified
debian/extras/99-maas-sudoers (+1/-1)
lp:~allenap/maas/packaging.move-maas-delete-and-write-file-to-common
Andres Rodriguez (community): Approve
Diff: 57 lines (+8/-5)
6 files modified
debian/extras/99-maas-common-sudoers (+4/-0)
debian/extras/99-maas-sudoers (+0/-2)
debian/maas-common.install (+2/-0)
debian/maas-common.lintian-overrides (+1/-0)
debian/maas-rack-controller.install (+0/-2)
debian/maas-rack-controller.lintian-overrides (+1/-1)
Gavin Panella (allenap)
Changed in maas:
status: New → Triaged
importance: Undecided → High
Revision history for this message
Gavin Panella (allenap) wrote :

While we sort out a proper fix you could work around this with:

  sudo apt install maas-rack-controller
  sudo systemctl stop maas-rackd.service
  sudo systemctl disable maas-rackd.service

Andres Rodriguez (andreserl)
Changed in maas:
importance: High → Critical
Revision history for this message
Peter teStrake (peter.t) wrote :

Thanks Gavin - the workaround works.

Revision history for this message
Jim Tilander (p-jim-8) wrote :

I've got a password prompt warning / error just ahead a similar error message. Is that unrelated?

```
sudo: a password is required
2017-02-13 07:42:40 maasserver.regiondservices.ntp: [critical] Failed to update NTP configuration.
 Traceback (most recent call last):
   File "/usr/lib/python3.5/threading.py", line 862, in run
     self._target(*self._args, **self._kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 824, in worker
     return target()
   File "/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 46, in work
     task()
   File "/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 190, in doWork
     task()
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
     result = inContext.theWork()
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
     inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 857, in callInContext
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 225, in wrapper
     result = func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/ntp/config.py", line 49, in configure
     mode=0o644)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/fs.py", line 281, in sudo_write_file
     raise ExternalProcessError(proc.returncode, command, stderr)
 provisioningserver.utils.shell.ExternalProcessError: Command `maas-rack atomic-write --filename /etc/ntp/maas.conf --mode 0644` returned non-zero exit status 1:
 None
```

Revision history for this message
Peter teStrake (peter.t) wrote : Re: [Bug 1650481] Re: MAAS region controller unable to update NTP
Download full text (7.2 KiB)

Yeh - the MAAS user needs a bunch of SUDO access.

I have these
root@region:/etc/sudoers.d# pwd
/etc/sudoers.d
root@region:/etc/sudoers.d# ls -1 | grep maas
10_maas_rack
10_maas_restart_bind9
10_maas_restart_maas-dhcpd
10_maas_restart_maas-dhcpd6
10_maas_restart_maas_proxy
10_maas_restart_ntp
10_maas_restart_tgt
10_maas_start_bind9
10_maas_start_maas-dhcpd
10_maas_start_maas-dhcpd6
10_maas_start_maas_proxy
10_maas_start_ntp
10_maas_start_tgt
10_maas_status_bind9
10_maas_status_maas-dhcpd
10_maas_status_maas-dhcpd6
10_maas_status_maas_proxy
10_maas_status_ntp
10_maas_status_tgt
10_maas_stop_bind9
10_maas_stop_maas-dhcpd
10_maas_stop_maas-dhcpd6
10_maas_stop_maas_proxy
10_maas_stop_ntp
10_maas_stop_tgt
root@region:/etc/sudoers.d#

and each one looks a bit like this
root@region:/etc/sudoers.d# cat 10_maas_restart_bind9
maas ALL= NOPASSWD: /bin/systemctl restart bind9

Regards
Pete

> On 13 Feb 2017, at 15:46, Jim Tilander <email address hidden> wrote:
>
> I've got a password prompt warning / error just ahead a similar error
> message. Is that unrelated?
>
> ```
> sudo: a password is required
> 2017-02-13 07:42:40 maasserver.regiondservices.ntp: [critical] Failed to update NTP configuration.
> Traceback (most recent call last):
> File "/usr/lib/python3.5/threading.py", line 862, in run
> self._target(*self._args, **self._kwargs)
> File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 824, in worker
> return target()
> File "/usr/lib/python3/dist-packages/twisted/_threads/_threadworker.py", line 46, in work
> task()
> File "/usr/lib/python3/dist-packages/twisted/_threads/_team.py", line 190, in doWork
> task()
> --- <exception caught here> ---
> File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
> result = inContext.theWork()
> File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
> inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
> File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
> return self.currentContext().callWithContext(ctx, func, *args, **kw)
> File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
> return func(*args,**kw)
> File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 857, in callInContext
> return func(*args, **kwargs)
> File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 225, in wrapper
> result = func(*args, **kwargs)
> File "/usr/lib/python3/dist-packages/provisioningserver/ntp/config.py", line 49, in configure
> mode=0o644)
> File "/usr/lib/python3/dist-packages/provisioningserver/utils/fs.py", line 281, in sudo_write_file
> raise ExternalProcessError(proc.returncode, command, stderr)
> provisioningserver.utils.shell.ExternalProcessError: Command `maas-rack atomic-write --filename /etc/ntp/maas.conf --mode 0644` returned non-zero exit status 1:
> None
> ```
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad....

Read more...

Gavin Panella (allenap)
summary: - MAAS region controller unable to update NTP
+ [2.2] MAAS region controller unable to update NTP
Andres Rodriguez (andreserl)
Changed in maas:
status: Fix Committed → Fix Released
Revision history for this message
Gavin Panella (allenap) wrote :

This fix will not be backported to 2.1. The workaround is simple (install maas-rack-controller) and it's fixed in 2.2 which will be released soon.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.