Linino Locks up after ~12-24 hours Running php Script every minute

I don't think 100ms for a ping is reasonable. Here are mine:

64 bytes from 192.168.1.100: icmp_seq=1 ttl=64 time=7.57 ms
64 bytes from 192.168.1.100: icmp_seq=2 ttl=64 time=1.84 ms
64 bytes from 192.168.1.100: icmp_seq=3 ttl=64 time=1.47 ms
64 bytes from 192.168.1.100: icmp_seq=4 ttl=64 time=1.17 ms
64 bytes from 192.168.1.100: icmp_seq=5 ttl=64 time=1.62 ms

Just ran the opkg command on my yun. Works like a charm

root@Marvin2:~# opkg update
Downloading http://download.linino.org/dogstick/all-in-one/latest/packages//Packages.gz.
Updated list of available packages in /var/opkg-lists/barrier_breaker.
Downloading http://download.linino.org/dogstick/all-in-one/latest/packages//Packages.sig.
Signature check passed.

Are you sure your firewall is not blocking something? does wget work?

root@Marvin2:~# wget http://download.linino.org/dogstick/all-in-one/latest/packa
ges//Packages.sig
Connecting to download.linino.org (88.198.63.58:80)
Packages.sig         100% |*******************************|   543   0:00:00 ETA
root@Marvin2:~# ls
Packages.sig
root@Marvin2:~#

Sure, 100 ms ping times aren’t great, but I am overseas. I tried all this on a different internet connection (4G hotspot from mobile phone) and am once again able to install packages… for now. So it seems it is a router/modem issue, but I didn’t change anything and this happened quite randomly. So there is still something wrong, but at least I have isolated it from the Yun… unless it was an Ethernet issue and it just happens to be working because of using Wifi. Now on to implementing NRPE…

The new ping times :)!:

root@yun01:~# ping download.linino.org -c 4
PING download.linino.org (88.198.63.58): 56 data bytes
64 bytes from 88.198.63.58: seq=0 ttl=47 time=453.278 ms
64 bytes from 88.198.63.58: seq=1 ttl=47 time=443.713 ms
64 bytes from 88.198.63.58: seq=2 ttl=47 time=422.705 ms
64 bytes from 88.198.63.58: seq=3 ttl=47 time=425.386 ms

--- download.linino.org ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 422.705/436.270/453.278 ms
root@yun01:~# ping sun.com -c 4
PING sun.com (156.151.59.35): 56 data bytes
64 bytes from 156.151.59.35: seq=0 ttl=241 time=349.327 ms
64 bytes from 156.151.59.35: seq=1 ttl=241 time=416.751 ms
64 bytes from 156.151.59.35: seq=2 ttl=241 time=383.680 ms
64 bytes from 156.151.59.35: seq=3 ttl=241 time=490.609 ms

--- sun.com ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 349.327/410.091/490.609 ms

Ok, now that that crazy issue with internet / opkg is out of the way, I have some concerns about NRPE and my initial issue.

It appears that the Linino isn't locking up, but that it may just be losing network connectivity. As I said, I can still see it connected to the router when I lose SSH connectivity and it fails to sync to my remote MySQL database. Resetting Linino if it loses internet connectivity is a potential solution, but it would introduce further issues:

My project only logs data locally if it has the correct time. It does this by comparing the latest entry in the sqlite3 database with the current time, and makes sure the current time is later. There may be times where the 4G router internet goes down, but I don't want to reset the Linino because it would cause it to lose the current time and not be able to restore it until the internet comes back up. I need it to continue logging data locally in the case of an internet failure, and my PHP script automatically syncs the data to my MySQL server when internet connectivity is restored.

Any suggestions on how to determine the cause of losing network connectivity after ~12-24 hours or differentiate between Linino losing network connectivity and my 4G router losing internet? (Note that when this issue occurs my 4G router still has internet access... I can login to it remotely.)

On top of this, are there any suggestions on how to reboot the Linino when I can only see it as a connected device in the router? I have this in the field right now and it would be beneficial to avoid a site visit just to reboot Linino.

Thanks for all the help... this is a strange issue!

On top of this, are there any suggestions on how to reboot the Linino when I can only see it as a connected device in the router? I have this in the field right now and it would be beneficial to avoid a site visit just to reboot Linino.

If your router support:

  • Dynamic DNS (DDNS)
  • SSH

Dynamic DNS (DDNS) is a method of automatically updating a name server in the Domain Name System (DNS), often in real time, with the active DNS configuration of its configured hostnames, addresses or other information.

Secure Shell (SSH) is a cryptographic network protocol for secure data communication, remote command-line login, remote command execution, and other secure network services between two networked computers that connects, via a secure channel over an insecure network, a server and a client .

Best candidates is Openwrt router (same OS as Yun). Now you could login router then from there ssh to Yun.

Plan B: load Dynamic DNS (DDNS) at Yun.

opkg update
opkg install ddns-scripts

http://wiki.openwrt.org/doc/howto/ddns.client

Then setup port forwarding forward port 22 of Yun.

Plan C: same as Plan B, but put Yun at DMZ zone.

sonnyyu: Plan B: load Dynamic DNS (DDNS) at Yun.

opkg update
opkg install ddns-scripts

http://wiki.openwrt.org/doc/howto/ddns.client

Then setup port forwarding forward port 22 of Yun.

Plan C: same as Plan B, but put Yun at DMZ zone.

Thanks for the suggestions, but I don't think this is what I need. I am able to access the router and Yun (SSH) fine for the first ~12 hours of uptime. I don't think I need DDNS... I have a static IP on my 4G router and I haven't been unable to connect to it remotely over the past ~1 week.

sonnyyu: Now you could login router then from there ssh to Yun.

Can you elaborate on this last line? Unfortunately, it is impractical to install OpenWRT or similar on my 4G router (NETGEAR MBR1516) and it is impractical to add a second router to the setup. (FYI, my next implementation will use a 4G USB dongle directly plugged into the Yun.) I have a static WAN IP on the router from my ISP, I have the Yun set to a static IP on the router's subnet, and I have had port 22 forwarded to the Yun from the beginning. As I mentioned above, after a Linino reboot I can SSH to the Yun fine (until ~12-24 hours later) and I can also access the Luci config pages (I also have port 80 forwarded to the Yun).

The problem is that after ~12-24 hours the Yun is essentially unreachable, other than the fact that it shows up here: After a modem reboot, the device comes back as 'attached'.

back to original point:

  • use nagios to monitor or profile system metrics.
  • put detect internet connection script at cronjob that runs every ~10 minutes, reboot machine if connection drop.

sonnyyu: back to original point:

  • use nagios to monitor or profile system metrics.
  • put detect internet connection script at cronjob that runs every ~10 minutes, reboot machine if connection drop.

Yes, this will work, but will have some undesirable consequences -> downtime (minor), but also the device would stop logging even if all is well and the 4G internet actually goes down. Perhaps there is a way to check the connection between the Yun and the router instead? I'd still like to get to the bottom of this issue because I don't think it should be happening at all. I am buying a 3rd Yun to test because my 2nd one isn't exhibiting these issues.

I was able to regain connection to my Yun in the field by changing the static IP in the 4G router setttings. I changed it from 192.168.0.10 to .180 and after some time, it obtained the new IP. I don't think it immediately got the new IP address after rebooting the 4G router. With the new IP of .180 and the ports forwarded to match, I am once again able to SSH and HTTP to the Yun. The Yun shows ~3 days of uptime, so it never restarted, confirming the suspicion that it is just losing internet connectivity.

Any ideas? I am thinking it has to do with DHCP lease time, even though I have it setup as static in the router. The recurring elapsed time to failure of ~12 hours is just too much to be a coincidence. I have already found some issues with the MBR1516 firmware that caused me a lot of grief. There is a bug where a port forward set up as TCP/UDP reverts to UDP when saved. So then trying to connect via TCP doesn't work. I wonder if there is also a bug with their DHCP or if it was a problem to have the Yun set to static within the DHCP range (now changed range to .2 to .100 and set Yun to .180).

Something clearly isn't right!

Please post output of

cat /etc/config/network

sonnyyu: Please post output of

cat /etc/config/network

sonnyyu, thank you so much for all your time. Looking around the forum I can see that you are very helpful to people on the board. Now that I can connect to the Yun in the field and have another one to test with, I am implementing the DHCP fix on all of them... I think that is the issue. When we were talking before I was unable to fix the remote located one. I was a bit confused on DHCP... I know what DHCP is, but wasn't thinking straight. Even though the router sets up a 'static' IP for the Yun, the Yun is still behaving as DHCP from its perspective. I will report back to confirm this. It's glad a fix was found (the link you provided), but it's frustrating that OpenWRT/Yun was released like this with such an obvious problem on the LAN. Hopefully not too many other people consume so much time by dealing with this issue.

root@yun03:~# cat /etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option proto 'static'
        option ipaddr '192.168.240.1'
        option netmask '255.255.255.0'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'
        option metric '10'

UPDATE: Looks like everything is working now!

I have changed dhcp.script on all my Yuns, but I am still not getting a proper route:

root@yun03:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     *               255.255.255.0   U     10     0        0 eth1
192.168.240.0   *               255.255.255.0   U     0      0        0 wlan0

Is there anything else I have to do other than edit dhcp.script and give the Yun a reboot? When does dhcp.script run? At startup or when a connection is initiated?

setup_interface () {
    proto_init_update "*" 1
    proto_add_ipv4_address "$ip" "${subnet:-255.255.255.0}"
    # TODO: apply $broadcast

    for i in $router; do
        proto_add_ipv4_route 0.0.0.0 0 "$i"
            route add -net 0.0.0.0/0 gw "$i"
    done

UPDATE: It looks like the problem was from not directly following instructions... I have a habit of using Notepad++ for editing files, but in this case it added an undesirable "^M" to the end of the line above the one added to dhcp.config.

Some final questions: The "/" in the line added looks out of place. Should it be a " " instead, or does it not matter?

Is dhcp.script not wiped and reloaded when "resetting to factory settings"? Because the Yun that I updated and had a "^M" did not have its ethernet connection fixed after a full reset.

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'
        option metric '10'

You are on dhcp. You could change it to static IP address.

config interface 'wan'
        option ifname 'eth1'
        option 'proto' 'static'
        option 'ipaddr' '192.168.0.239'
        option 'netmask' '255.255.255.0'
    option 'gateway' '192.168.0.1'
    option 'dns' '8.8.8.8'

Never use windows app to edit Linux config file, you must be comfortable to use vi (some time vi is only one available), after Linux box up online then you could install nano (very closed to notepad).

whisprer:
Some final questions: The “/” in the line added looks out of place. Should it be a " " instead, or does it not matter?

The “0.0.0.0/0” is CIDR notation. In this case it would be the full range of addresses (0.0.0.0 to 255.255.255.255).

One more thought:

Turn off Wifi if you don’t use it.

AR9331 (the heart of Yun) has a hardware bug where spurious RF ( WIFI reset ) emissions affect the USB PLL. Of course, Qualcomm Atheros will not acknowledge it as a critical bug as they found a software workaround for it.

The issue appears to be solved. I now have 4 Yuns set up and not losing eth1 after router DHCP lease time expires. My solution was to use the DHCP fix linked to by Sonnyyu earlier in this thread. I prefer to not set up static from the Yun side because the ethernet devices I will connect to may vary in their IP ranges.

This issue should really be publicized clearly, perhaps in "Getting Started with Yun" or similar. I realize it is an issue with OpenWRT, but the reality is that most users expect to be able to run the Yun for extended periods of time (days, weeks, months, even years) with a reliable ethernet connection.

Sonnyyu, thanks again for the help. You turned a potentially unsolvable problem for me into one that only took a few days!

@whisprer Please add it to the FAQ http://playground.arduino.cc/Hardware/Yun#.UzMOu_GlDPE I'm quite willing to test and proof read.

Best regards Jantje

I have a similar problem with wifi (no eth) that hangup after a while, there is a way to solve it with wifi? (php website on yun, after a while, dosen't answer at all or sometimes after 10/15 seconds.)

whisprer: The issue appears to be solved. I now have 4 Yuns set up and not losing eth1 after router DHCP lease time expires. My solution was to use the DHCP fix linked to by Sonnyyu earlier in this thread. I prefer to not set up static from the Yun side because the ethernet devices I will connect to may vary in their IP ranges.

You need to tell us either network drop out or php website (uhttpd) time out. In network drop out case either Yun drop out from network completed or Yun drop out internet connection only? In php website ( uhttpd) time out case either maximum number of concurrent requests is over or script_timeout is over? Did you test with ethernet?

Hi, I’m here again (see answer only now, forgot ‘Notify me of replies.’ option), then…

sonnyyu:
You need to tell us either network drop out or php website (uhttpd) time out.
In network drop out case either Yun drop out from network completed or Yun drop out internet connection only?
In php website ( uhttpd) time out case either maximum number of concurrent requests is over or script_timeout is over?
Did you test with ethernet?

Network drop out. I can’t connect to YUN (website on SD nor to /cgi-bin/luci/webpanel/homepage)
I use the myArduinoName.local to connect.

Not fully tested on ETH (I’m doing this now)

Find also the “option max_requests 2” on /etc/config/uhttpd … can I change? how many?
(the interface could be used by more concurrent users… to be safe I think 4)

Changing to ETH I find also something strange:
→ wifi connect to my router (with static IP)
→ eht connect to my router (with obviously different static IP)
→ change the redirect (meta refresh on index html inside www pointing to sd)
Result:

  1. if I point to IP (wifi/eth) it redirects to /cgi-bin/luci/webpanel/homepage
  2. if I point to myNameArduino.local it redirects (correctly) to sd

for static ip I change only /etc/config/network in this way

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option proto 'static'        
        option ipaddr '192.168.1.203'                 
        option netmask '255.255.255.0'                        
        option gateway '192.168.1.1'                                
        option dns '192.168.1.1'  

config interface 'wan'
        option ifname 'eth1'
        option proto 'static'        
        option ipaddr '192.168.1.103'                 
        option netmask '255.255.255.0'                        
        option gateway '192.168.1.1'                                
        option dns '192.168.1.1'