Ubuntu
network-manager package

No connection after returning from area without coverage

Bug #1565717 reported by Alfonso Sanchez-Beato
80
This bug affects 19 people
Affects Status Importance Assigned to Milestone
Canonical System Image
Fix Released
Critical
John McAleely
Canonical System Image 12 "OTA12"
network-manager (Ubuntu)
Fix Committed
Critical
Tony Espy

Bug Description

After moving in an out of my own "faraday cage" an ubuntu phone, NM got into an state where *both* WiFi and cellular data were disconnected and NM was not able to recover.

****************
* This still happens with network-manager 1.1.93-0ubuntu1~vivid3, in

phablet@ubuntu-phablet:~$ system-image-cli -i
current build number: 336
device name: krillin
channel: ubuntu-touch/rc-proposed/bq-aquaris.en
last update: 2016-05-17 09:19:51
version version: 336
version ubuntu: 20160517
version device: 20160329-a9bacdb
version custom: 20160505-975-38-9
****************

Originally reported for:

NM version:
network-manager 0.9.10.0-4ubuntu15.1.11

root@ubuntu-phablet:/home/phablet# system-image-cli -i
current build number: 298
device name: krillin
channel: ubuntu-touch/rc-proposed/bq-aquaris.en
last update: 2016-03-30 16:58:21
version version: 298
version ubuntu: 20160330
version device: 20160323-1467d3c
version custom: 20160324--36-54-vivid

phablet@ubuntu-phablet:~$ nmcli d
DEVICE TYPE STATE CONNECTION
ril_0 gsm connecting (prepare) /214050030479893/context1
wlan0 wifi disconnected --
ril_1 gsm unavailable --
ifb0 ifb unmanaged --
ifb1 ifb unmanaged --
lo loopback unmanaged --
ip6tnl0 unknown unmanaged --
sit0 unknown unmanaged --
tunl0 unknown unmanaged --

phablet@ubuntu-phablet:~$ nmcli c
NAME UUID TYPE DEVICE
WLAN_1609 fe9b17ad-88fa-43b7-bcab-8fbc1985ce42 802-11-wireless --
Wireless 98ff5b6f-b0d9-471a-b867-8a7be51e12c2 802-11-wireless --
Ubuntu 549c352f-8a2b-4e48-b8c8-eb7ab0e53089 802-11-wireless --
/234304107917083/context1 1f192cdc-ccea-2dc3-fa13-619ed3053828 gsm --
/214321010036211/context1 0992421f-2369-0765-ceaa-ba2a3091e111 gsm --
/214050030479893/context1 d30e4f19-9d00-ab1f-ab43-6d9c1b628556 gsm ril_0
/214019301737411/context1 88b28978-e2fc-6336-8665-b0ae78d321ae gsm --

phablet@ubuntu-phablet:~$ /usr/share/ofono/scripts/list-contexts
[ /ril_1 ]
[ /ril_0 ]
    [ /ril_0/context1 ]
        Settings = { Gateway=10.50.83.25 DomainNameServers=80.58.61.250,80.58.61.254, Interface=ccmni0 Address=10.50.83.25 Method=static Netmask=255.255.255.0 }
        Name = Pepephone
        Type = internet
        AccessPointName = gprs.pepephone.com
        Active = 1
        Protocol = ip
        AuthenticationMethod = chap
        Password =
        Preferred = 0
        Username =
        IPv6.Settings = { }

    [ /ril_0/context2 ]
        Name = MMS Pepephone
        Type = mms
        Active = 0
        Settings = { }
        MessageCenter = http://www.pepephone.com
        MessageProxy = 10.138.255.43:8080
        Preferred = 0
        Password =
        AccessPointName = gprs.pepephone.com
        Protocol = ip
        AuthenticationMethod = chap
        Username =
        IPv6.Settings = { }

phablet@ubuntu-phablet:~$ ifconfig
ccmni0 Link encap:Ethernet HWaddr 46:23:7c:fb:73:cb
          inet addr:10.50.83.25 Mask:255.0.0.0
          UP RUNNING NOARP MTU:1500 Metric:1
          RX packets:1995 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2299 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:603432 (603.4 KB) TX bytes:193513 (193.5 KB)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:11189 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11189 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1733281 (1.7 MB) TX bytes:1733281 (1.7 MB)

wlan0 Link encap:Ethernet HWaddr b8:64:91:47:21:b6
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:3184 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1626 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:308130 (308.1 KB) TX bytes:172906 (172.9 KB)

phablet@ubuntu-phablet:~$ /usr/share/ofono/scripts/list-modems
[ /ril_1 ]
    Serial = 354142069998062
    Features = rat sim
    Revision = MOLY.WR8.W1315.MD.WG.MP.V37.P5, 2014/05/15 11:49
    Manufacturer = Fake Manufacturer
    Powered = 1
    Lockdown = 0
    Interfaces = org.ofono.RadioSettings org.ofono.SimManager org.ofono.MtkSettings org.ofono.CallVolume org.ofono.VoiceCallManager org.ofono.NetworkTime
    Type = hardware
    Model = Fake Modem Model
    Online = 1
    Emergency = 0
    [ org.ofono.RadioSettings ]
        TechnologyPreference = gsm
        AvailableTechnologies = gsm
        FastDormancy = 0
    [ org.ofono.SimManager ]
        Present = 0
    [ org.ofono.MtkSettings ]
        Has3G = 0
    [ org.ofono.CallVolume ]
        MicrophoneVolume = 0
        SpeakerVolume = 0
        Muted = 0
    [ org.ofono.VoiceCallManager ]
        EmergencyNumbers = 08 000 999 110 112 911 118 119
    [ org.ofono.NetworkTime ]

[ /ril_0 ]
    Serial = 354140069999981
    Features = gprs ussd net sms rat sim
    Revision = MOLY.WR8.W1315.MD.WG.MP.V37.P5, 2014/05/15 11:49
    Manufacturer = Fake Manufacturer
    Powered = 1
    Lockdown = 0
    Interfaces = org.ofono.ConnectionManager org.ofono.Phonebook org.ofono.CallBarring org.ofono.CallForwarding org.ofono.CallSettings org.ofono.SupplementaryServices org.ofono.NetworkRegistration org.ofono.PushNotification org.ofono.MessageManager org.ofono.MessageWaiting org.ofono.RadioSettings org.ofono.SimManager org.ofono.MtkSettings org.ofono.CallVolume org.ofono.VoiceCallManager org.ofono.NetworkTime
    Type = hardware
    Model = Fake Modem Model
    Online = 1
    Emergency = 0
    [ org.ofono.ConnectionManager ]
        Bearer = gprs
        Attached = 1
        RoamingAllowed = 0
        Powered = 1
        Suspended = 0
    [ org.ofono.Phonebook ]
    [ org.ofono.CallBarring ]
        VoiceIncoming = disabled
        VoiceOutgoing = disabled
    [ org.ofono.CallForwarding ]
        VoiceNoReply =
        VoiceNoReplyTimeout = 20
        VoiceNotReachable = +346808934593085
        VoiceUnconditional =
        VoiceBusy = +346808934593085
        ForwardingFlagOnSim = 0
    [ org.ofono.CallSettings ]
        CalledLinePresentation = disabled
        CallingLineRestriction = disabled
        ConnectedLineRestriction = unknown
        VoiceCallWaiting = disabled
        CallingLinePresentation = enabled
        ConnectedLinePresentation = unknown
        CallingNamePresentation = unknown
        HideCallerId = default
    [ org.ofono.SupplementaryServices ]
        State = idle
    [ org.ofono.NetworkRegistration ]
        Technology = gsm
        Status = registered
        Mode = auto
        CellId = 4321
        MobileCountryCode = 214
        Strength = 41
        MobileNetworkCode = 07
        LocationAreaCode = 2824
        Name = pepephone
    [ org.ofono.PushNotification ]
    [ org.ofono.MessageManager ]
        ServiceCenterAddress = +34609090909
        UseDeliveryReports = 0
        Bearer = cs-preferred
        Alphabet = default
    [ org.ofono.MessageWaiting ]
        VoicemailWaiting = 0
        VoicemailMessageCount = 0
        VoicemailMailboxNumber =
    [ org.ofono.RadioSettings ]
        TechnologyPreference = umts
        AvailableTechnologies = gsm umts
        FastDormancy = 0
    [ org.ofono.SimManager ]
        FixedDialing = 0
        LockedPins =
        Retries = [puk = 10] [pin2 = 3] [pin = 3] [puk2 = 10]
        SubscriberNumbers =
        PreferredLanguages = es
        MobileNetworkCode = 05
        ServiceNumbers = [Buzon de Voz (Roaming)] = '+34717700177' [SMS (Configura Internet)] = '22050' [Atencion al cliente ADSL] = '1214' [Buzon de Voz] = '22177' [Atencion al cliente] = '1212'
        SubscriberIdentity = 214050030479893
        CardIdentifier = 8934075782002298987
        MobileCountryCode = 214
        Present = 1
        BarredDialing = 0
        PinRequired = none
    [ org.ofono.MtkSettings ]
        Has3G = 1
    [ org.ofono.CallVolume ]
        MicrophoneVolume = 0
        SpeakerVolume = 0
        Muted = 0
    [ org.ofono.VoiceCallManager ]
        EmergencyNumbers = 112 911
    [ org.ofono.NetworkTime ]

phablet@ubuntu-phablet:~$ ip route
phablet@ubuntu-phablet:~$

phablet@ubuntu-phablet:~$ nmcli d show
GENERAL.DEVICE: ril_0
GENERAL.TYPE: gsm
GENERAL.HWADDR: (unknown)
GENERAL.MTU: 0
GENERAL.STATE: 40 (connecting (prepare))
GENERAL.CONNECTION: /214050030479893/context1
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/47

GENERAL.DEVICE: wlan0
GENERAL.TYPE: wifi
GENERAL.HWADDR: B8:64:91:47:21:B6
GENERAL.MTU: 1500
GENERAL.STATE: 30 (disconnected)
GENERAL.CONNECTION: --
GENERAL.CON-PATH: --
...

See original description
Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :
Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

NOTE: On the cellular side, it seems like NM gets stuck into an intermediate state (connecting (prepare)). I was able to reproduce something very similar deterministically by doing:

stop network-manager
/usr/share/ofono/scripts/activate-context /ril_0 1
start network-manager

It seems that NM cannot handle that unexpected connection when starting. When using the faraday cage, what happened is that NM was notified of a previous successful connection while in an intermediate state.

Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

Also, doing

kill -9 `pgrep NetworkManager`

alse leaves in a bad state the modem device:

root@ubuntu-phablet:/home/phablet# nmcli d
DEVICE TYPE STATE CONNECTION
wlan0 wifi connected WLAN_1609
ril_0 gsm connecting (prepare) /214050030479893/context1
...

Which is a valid bug, as this is what would happen in case NM crashes.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in network-manager (Ubuntu):
status: New → Confirmed
Revision history for this message
Pat McGowan (pat-mcgowan) wrote :

Saw this over the weekend while driving. Had a connection but the browser would not connect, no idea if I had previously lost the cell signal.
I then cycled flight mode, started browsing, lost signal, regained signal, browser would not connect.

To recreate this I blocked the cell signal momentarily, regained it, and the browser does not think its connected, so I assume this happens 100% of the time that we lose signal.

$ nmcli d
DEVICE TYPE STATE CONNECTION
ril_0 gsm disconnected --
wlan0 wifi disconnected --

but ril_0 is connected acc to the indicator

[ /ril_0 ]
    [ /ril_0/context1 ]
        IPv6.Settings = { }
        Settings = { }
        Password =
        AccessPointName = nxtgenphone
        Name = ATT Nextgenphone
        AuthenticationMethod = chap
        Active = 0
        Username =
        Type = internet
        MessageCenter = http://mmsc.mobile.att.net
        Protocol = ip
        Preferred = 0
        MessageProxy = proxy.mobile.att.net:80

    [ /ril_0/context2 ]
        IPv6.Settings = { }
        Settings = { }
        Password =
        AccessPointName = phone
        Name = ATT Phone
        AuthenticationMethod = chap
        Active = 0
        Username =
        Type = internet
        MessageCenter = http://mmsc.mobile.att.net
        Protocol = ip
        Preferred = 1
        MessageProxy = proxy.mobile.att.net:80

    [ /ril_0/context3 ]
        IPv6.Settings = { }
        Settings = { }
        Password =
        AccessPointName = wap.cingular
        Name = ATT WAP
        AuthenticationMethod = chap
        Active = 0
        Username =
        Type = internet
        MessageCenter = http://mmsc.cingular.com/
        Protocol = ip
        Preferred = 0
        MessageProxy = wireless.cingular.com

$ /usr/share/ofono/scripts/list-modems
[ /ril_1 ]
    Online = 1
    Features = gprs ussd net sms rat sim
    Powered = 1
    Serial = 354142060380047
    Interfaces = org.ofono.ConnectionManager org.ofono.Phonebook org.ofono.CallBarring org.ofono.CallForwarding org.ofono.CallSettings org.ofono.SupplementaryServices org.ofono.NetworkRegistration org.ofono.PushNotification org.ofono.MessageManager org.ofono.MessageWaiting org.ofono.RadioSettings org.ofono.SimManager org.ofono.MtkSettings org.ofono.CallVolume org.ofono.VoiceCallManager org.ofono.NetworkTime
    Type = hardware
    Model = Fake Modem Model
    Revision = MOLY.WR8.W1315.MD.WG.MP.V37.P5, 2014/05/15 11:49
    Lockdown = 0
    Emergency = 0
    Manufacturer = Fake Manufacturer
    [ org.ofono.ConnectionManager ]
        Bearer = none
        Powered = 0
        Attached = 0
        RoamingAllowed = 0
    [ org.ofono.Phonebook ]
    [ org.ofono.CallBarring ]

Changed in canonical-devices-system-image:
assignee: nobody → John McAleely (john.mcaleely)
importance: Undecided → Critical
milestone: none → 12
status: New → Confirmed
Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

@pat-mcgowan, not exactly the same bug that was reported, as you do not have active contexts. But still possibly a NM bug if you saw a cellular data icon when dropping down the indicator line (if you have the icon there but not in the top of the screen it means the modem is data attached but there is no active context).

Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

I have just confirmed this still happens with 1.1.93-0ubuntu1~vivid3

description: updated
Revision history for this message
Pat McGowan (pat-mcgowan) wrote :

@alfonso yes it was also shown as connected in the top of the screen

Tony Espy (awe)
Changed in network-manager (Ubuntu):
assignee: nobody → Tony Espy (awe)
importance: Undecided → Critical
Revision history for this message
Tony Espy (awe) wrote :

@Alfonso

The scenario in comment #2 isn't valid. That said, I am able to reproduce by killing NetworkManager, so I agree this is a valid bug that needs to be fixed.

@Pat

As Alfonso pointed out, your problem is different, as there's no active context, nor does NetworkManager appear stuck. Could you please file a new bug, and if possible nail down the exact steps to reproduce? As described, it sounds like an indicator bug.

Revision history for this message
Tony Espy (awe) wrote :

@Alfsonso

One more note, although comment #2 isn't a valid scenario, it does trigger the same behavior.

The root cause of the bug is that the current NM wwan plugin doesn't properly handle already activated contexts.

Changed in network-manager (Ubuntu):
status: Confirmed → In Progress
Revision history for this message
Pat McGowan (pat-mcgowan) wrote :

@tony the steps to reproduce it are in comment #5, I put my phone in a metal pot for 20 seconds.
The bug title is the exact description of the symptom I have, so seems odd to make a new bug because doing the same act caused a different state on the device.

Bug #1579098 is the same symptom and without an active context so we already have that

Revision history for this message
Tony Espy (awe) wrote :

@Pat

The reason I asked was that two of the three scenarios listed by Alfonso in the bug involved a crash or restart of NetworkManager, and the resulting state was that the mobile data connection gets stuck trying to activate an already active ( or activating ) connection.

I wasn't sure if he was able to reproduce this exact scenario with the faraday scenario. Looking at his syslog in comment #1, I see that it did end up in the state. I think it's just harder a smaller window to hit ( ie. the signal loss has to line up timing-wise with the activation attempt ).

I added comments to bug #1579098 re: the fact that I can reproduce it, and have a pending fix.

Revision history for this message
Tony Espy (awe) wrote :

OK, so there are two low-level problems:

1. InProgress error returned by SetProperty('Active',true) call to ofono for a gprs-context.

Alfonso's first syslog shows that this can happen, but I'm not sure exactly how to reproduce. NM would've had to start activation, then Attached bounced down and back up again, without ofono canceling the context activation. The next time NM tried to activate, in theory it'd be returned InProgress.

Right now, NMModemOfono::stage1_prepare_done() always emits a failed MODEM_PREPARE_RESULT for any error returned by ofono in response to the SetProperty call.

The fix is to check explicitly for error == InProgress, and just return without emitting a result ( this is the same thing that happens if no error is returned ).

2. The context in question is already active.

This can be reproduced by killing NetworkManager, when it restarts, it gets stuck in the Prepare stage.

This is because ofono doesn't send an error if SetProperty('Active', true) would have no effect on the property's value ( ie. if it's already true ). Furthermore, as the context 'Settings' property is already populated, context_property_changed() is never invoked for the 'Settings', so a MODEM_PREPARE_RESULT is never sent, true or false, so the device is stuck in PREPARE state.

The fix involves the following changes to NMModemOfono:

do_context_activate() should check the cached value of 'Active', and if true, should call a new function called process_context_settings(), which should be based on the current context_property_changed(). This will result in a PREPARE_RESULT being generated, and thus the device should no longer get stuck in PREPARE state.

In theory this should work, as NM should just deal with any EEXISTS errors when it tries to re-configure the IP address, routing table entries, and DNS nameservers. It this *doesn't* just work, then we may need some upstream assistance.

Pat McGowan (pat-mcgowan)
tags: added: hotfix
Pat McGowan (pat-mcgowan)
Changed in canonical-devices-system-image:
status: Confirmed → In Progress
Tony Espy (awe)
Changed in network-manager (Ubuntu):
status: In Progress → Fix Committed
Pat McGowan (pat-mcgowan)
Changed in canonical-devices-system-image:
status: In Progress → Fix Committed
Pat McGowan (pat-mcgowan)
Changed in canonical-devices-system-image:
status: Fix Committed → Fix Released
Aron Xu (happyaron)
tags: added: nm-touch
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.