Arduino Forum

Products => Arduino Yún => Topic started by: fibasile on Oct 16, 2013, 08:09 pm

Title: Node.js for the YUN
Post by: fibasile on Oct 16, 2013, 08:09 pm
Hi,

I managed to cross-compile the latest node.js v0.10.20 for the YUN.

In order to save you some time, I uploaded the binaries on github, you can download them at this link:
https://raw.github.com/fibasile/yun-packages/master/node-v0.10.20-mips.tgz

The package contains only the node binary and the required .so libraries. Copy them to your sd-card or usb stick and link node in /usr/bin and the .so in /usr/lib. 

If you like to know how to build it yourself please check the tutorial on my blog:

http://fibasile.github.io/compiling-nodejs-for-arduino-yun.html

If you need additional modules (as you probably will) you should build the binaries following the tutorial and also follow the instructions at the end, since you will need to build the modules on your host system and then transfer them to the YUN.

Hope you find it useful!
Title: Re: Node.js for the YUN
Post by: federicofissore on Oct 17, 2013, 09:09 am
Great! I'll look into this and try to port it to a linino makefile
Title: Re: Node.js for the YUN
Post by: fibasile on Oct 17, 2013, 10:10 am
That would be great.

Even if I'm not sure how to handle building node modules, since it must be done on the host as many  require gcc etc.

Unfortunately there's no way to have a "slim" package management for node, all modules encapsulate all their dependencies making it quite bloated.

For now, I think the best solution is to "package" a specific app on the host in a tgz and move it to the sd card.
Title: Re: Node.js for the YUN
Post by: federicofissore on May 16, 2014, 12:46 pm
Hi fibasile, just wanted you to know that we did it :)
https://github.com/arduino/openwrt-packages-yun/commit/34103658fedf926ea4bb8dc5fcc383603244a4f1
Thank you very much for your work :)
Title: Re: Node.js for the YUN
Post by: Squonk42 on May 17, 2014, 03:41 pm
Hi Frederico,

I have node running on the latest Yun image with extroot and got the console test working ok.

However, when I try to install the express framework, I get the following message:
root@Arduino:~# npm install -g express
npm http GET https://registry.npmjs.org/express
npm http 304 https://registry.npmjs.org/express
npm http GET https://registry.npmjs.org/parseurl
npm http GET https://registry.npmjs.org/methods
npm http GET https://registry.npmjs.org/send
npm http GET https://registry.npmjs.org/accepts
npm http GET https://registry.npmjs.org/cookie-signature
npm http GET https://registry.npmjs.org/type-is
npm http GET https://registry.npmjs.org/merge-descriptors
npm http GET https://registry.npmjs.org/utils-merge
npm http GET https://registry.npmjs.org/range-parser
npm http GET https://registry.npmjs.org/escape-html
npm http GET https://registry.npmjs.org/qs
npm http GET https://registry.npmjs.org/cookie
npm http GET https://registry.npmjs.org/serve-static
npm http GET https://registry.npmjs.org/path-to-regexp
npm http GET https://registry.npmjs.org/buffer-crc32
npm http GET https://registry.npmjs.org/debug
npm http GET https://registry.npmjs.org/fresh
npm http 304 https://registry.npmjs.org/methods
npm http 304 https://registry.npmjs.org/send
npm http 304 https://registry.npmjs.org/parseurl
npm http 304 https://registry.npmjs.org/accepts
npm http 304 https://registry.npmjs.org/merge-descriptors
npm http 304 https://registry.npmjs.org/type-is
npm http 304 https://registry.npmjs.org/cookie-signature
npm http 304 https://registry.npmjs.org/utils-merge
npm http 304 https://registry.npmjs.org/range-parser
npm http 304 https://registry.npmjs.org/escape-html
npm http 304 https://registry.npmjs.org/qs
npm http 304 https://registry.npmjs.org/cookie
npm http 304 https://registry.npmjs.org/serve-static
npm http 304 https://registry.npmjs.org/path-to-regexp
npm http 304 https://registry.npmjs.org/buffer-crc32
npm http 304 https://registry.npmjs.org/debug
npm http 304 https://registry.npmjs.org/fresh
npm http GET https://registry.npmjs.org/mime
npm http GET https://registry.npmjs.org/negotiator
npm http 304 https://registry.npmjs.org/mime
npm http 304 https://registry.npmjs.org/negotiator
npm http GET https://registry.npmjs.org/debug
npm http 304 https://registry.npmjs.org/debug
FATAL ERROR: Evacuation Allocation failed - process out of memory


It looks like node is crashing because it is out of memory, although I am only performing a simple install.

It makes me worry about real node usability on a 64 MB-only memory without  activated swap.

Does anyone has experience with it?
Title: Re: Node.js for the YUN
Post by: federicofissore on May 17, 2014, 10:25 pm
I've experienced that too. It seems like npm is using a lot of memory for doing a simple install. Fortunately, node and express work well once installed.
The easiest way is probably to install express on your computer and then copy the node_modules folder on your yun.
Otherwise you can set up a temporary swap file, just for setting it up
Title: Re: Node.js for the YUN
Post by: mantissa00 on May 19, 2014, 01:49 pm
after extending the flash memory with the sd card I have installed express without any problem.. does it matter?
Title: Re: Node.js for the YUN
Post by: Squonk42 on May 22, 2014, 11:30 pm
I have a Yun with latest image, a 16 GB SD Card formatted using the YunDiskSpaceExpander sketch and followed the instructions for installing node using opkg.

I tried both with or without a 512 MB swap on SDCard, installing express or johnny-five from a fresh boot results in the same "FATAL ERROR: Evacuation Allocation failed - process out of memory" message  =(

Here is my config:
root@Arduino:/# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                   13.6G    375.1M     12.6G   3% /
/dev/root                 7.5M      7.5M         0 100% /rom
tmpfs                    29.8M    100.0K     29.7M   0% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                13.6G    375.1M     12.6G   3% /overlay
overlayfs:/overlay       13.6G    375.1M     12.6G   3% /
/dev/sda1               698.6M     12.0K    698.6M   0% /mnt/sda1
root@Arduino:/# free
            total         used         free       shared      buffers
Mem:         61116        43064        18052            0        16560
-/+ buffers:              26504        34612
Swap:       523260          168       523092
root@Arduino:/# ps -wwww
 PID USER       VSZ STAT COMMAND
   1 root      1496 S    init
   2 root         0 SW   [kthreadd]
   3 root         0 SW   [ksoftirqd/0]
   4 root         0 SW   [kworker/0:0]
   5 root         0 SW   [kworker/u:0]
   6 root         0 SW<  [khelper]
   7 root         0 SW   [kworker/u:1]
  90 root         0 SW   [sync_supers]
  92 root         0 SW   [bdi-default]
  94 root         0 SW<  [kblockd]
 106 root         0 SW   [khubd]
 132 root         0 SW   [kswapd0]
 180 root         0 SW   [fsnotify_mark]
 211 root         0 SW<  [ath79-spi]
 222 root         0 SW   [mtdblock0]
 227 root         0 SW   [mtdblock1]
 232 root         0 SW   [mtdblock2]
 237 root         0 SW   [mtdblock3]
 242 root         0 SW   [mtdblock4]
 247 root         0 SW   [mtdblock5]
 252 root         0 SW   [mtdblock6]
 257 root         0 SW   [mtdblock7]
 310 root         0 SW   [kworker/0:1]
 312 root         0 SW   [kworker/u:2]
 339 root         0 SW   [scsi_eh_0]
 341 root         0 SW   [usb-storage]
 473 root         0 SW   [kworker/0:2]
 530 root         0 SW   [flush-mtd-unmap]
 577 root         0 SW   [flush-8:0]
 578 root         0 SW   [jbd2/sda2-8]
 579 root         0 SW<  [ext4-dio-unwrit]
 601 root      1524 S    {rcS} /bin/sh /etc/init.d/rcS S boot
 602 root      1492 S    /bin/ash --login
 604 root      1488 S    logger -s -p 6 -t sysinit
 649 root         0 SW<  [cfg80211]
 756 root      1500 S    /sbin/syslogd -l 8 -C16
 758 root      1484 S    /sbin/klogd
 760 root       872 S    /sbin/hotplug2 --override --persistent --set-rules-file /etc/hotplug2.rules --set-coldplug-cmd /sbin/udevtrigger --max-children 1
 770 root       876 S    /sbin/ubusd
 834 root      1480 S    /sbin/netifd
 897 root      1492 S    udhcpc -p /var/run/udhcpc-eth1.pid -s /lib/netifd/dhcp.script -f -t 0 -i eth1 -C
1360 root      1628 S    wpa_supplicant -B -P /var/run/wifi-wlan0.pid -D nl80211 -i wlan0 -c /var/run/wpa_supplicant-wlan0.conf
1456 root      1500 S    udhcpc -p /var/run/udhcpc-wlan0.pid -s /lib/netifd/dhcp.script -f -t 0 -i wlan0 -C
1516 root      1492 S    /sbin/watchdog -t 5 /dev/watchdog
1615 root      1152 S    /usr/sbin/dropbear -P /var/run/dropbear.1.pid -p 22
1622 root      1564 S    /usr/sbin/uhttpd -f -h /www -r Arduino -x /cgi-bin -t 60 -T 30 -A 1 -n 2 -p 0.0.0.0:80 -C /etc/uhttpd.crt -K /etc/uhttpd.key -s 0.0.0.0:443
1631 root      1696 S    /usr/sbin/dbus-daemon --system
1654 nobody     948 S    /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf
1660 nobody    2168 S    avahi-daemon: running [Arduino.local]
1663 root       796 S    /usr/sbin/thd --socket /tmp/triggerhappy.socket --triggers /etc/triggerhappy/triggers.d/ --daemon /dev/input/event0
1687 root      1492 S    /usr/sbin/ntpd -n -p 0.openwrt.pool.ntp.org -p 1.openwrt.pool.ntp.org -p 2.openwrt.pool.ntp.org -p 3.openwrt.pool.ntp.org
1694 root      1496 S N  {uSDaemon} /bin/sh /sbin/uSDaemon
2399 root      1480 S N  sleep 2
2400 root      1488 R    ps -wwww

@mantissa00: do you see something that could be different from your setup?
Title: Re: Node.js for the YUN
Post by: federicofissore on May 23, 2014, 08:45 am
There is still much room for improvement on the nodejs side
@Squonk42, try this one
Access the yun via ssh or putty (do NOT use the IDE Serial Monitor)
Enter
Code: [Select]

nano /usr/bin/node


Make file look like this
Code: [Select]

#!/bin/sh

#NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs --stack_size=1024 --max_old_space_size=20 --max_new_space_size=2048 --max_executable_size=5 --gc_global --gc_interval=100 $@
NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs $@


And retry
Title: Re: Node.js for the YUN
Post by: Squonk42 on May 23, 2014, 10:04 pm
Hi Frederico,

Thank you very much, it works when removing the node memory options !

However, why do you recommend NOT to use the IDE serial monitor? Is it because it doesn't work well with a full screen text editor like nano?
Title: Re: Node.js for the YUN
Post by: federicofissore on May 24, 2014, 07:34 am
Exactly: IDE serial monitor is a not a fully fledged Terminal. Linux terminals use control characters that you cannot send with IDE Serial Monitor. For example, you could not use the arrow keys to move to the desired text
Title: Re: Node.js for the YUN
Post by: Squonk42 on May 24, 2014, 12:17 pm
The express installation looks it is OK, it look like it is installed in "/node/usr/lib/node_modules/express", but I cant't find the "express" command line script, and it is not in the PATH either.
Title: Re: Node.js for the YUN
Post by: federicofissore on May 24, 2014, 01:00 pm
Recent version of express bundle the lib only. You need to install express-generator for the command line
See https://github.com/visionmedia/express#quick-start
Title: Re: Node.js for the YUN
Post by: Squonk42 on May 25, 2014, 11:48 pm
Thanks Frederico, I have everything working now!

I performed some tests with a full-stack solution consisting in:

It takes 15~20 s to generate a page when not in cache, 2~3 s when in cache  :smiley-roll-sweat:

IMHO, Node.js is so slow that it is pretty unusable on the Yun in its current state.
Title: Re: Node.js for the YUN
Post by: federicofissore on May 26, 2014, 09:13 am
Surely the Yun is not the best development environment almost for everything else than small scripts.
As for Node.js, I would stick with a "development on my pc, production on the yun" approach, just like you would do if you were to put some website on the public internet.
I would avoid everything that needs some additional computation before being used. Jade for example needs to compile its templates at the first request and once compiled they go to memory. Try using another template engine that allows you to compile templates "offline" (I used dust.js in the past). Also avoid running coffeescript on the server side: precompile it to javascript (don't get me wrong: I'm a big fan of coffeescript)
Title: Re: Node.js for the YUN
Post by: federicofissore on May 26, 2014, 09:17 am
One more thing. As you know nodejs is single threaded. In your example, that means that it's either giving you back the glaphicons or it's answering some "hello world" request. Uglify and pack all the assets in few files (ideally: one js and one css)
Title: Re: Node.js for the YUN
Post by: mantissa00 on May 26, 2014, 03:51 pm
@Squonk42
I have a fresh install on a new yun with a smaller sd card (2GB instead of 8GB), and I had that issue.

here the df logs:

------------------ BIG SD CARD - NO PROBLEM ------------------
Code: [Select]


root@Arduino:~# cat /usr/bin/node
#!/bin/sh

NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs --stack_size=1024 --max_old_space_size=20 --max_new_space_size=2048 --max_executable_size=5 --gc_global --gc_interval=100 $@


root@Arduino:~# df -h

Filesystem                Size      Used Available Use% Mounted on
rootfs                    6.5G    325.3M      5.9G   5% /
/dev/root                 7.5M      7.5M         0 100% /rom
tmpfs                    29.8M    164.0K     29.7M   1% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                 6.5G    325.3M      5.9G   5% /overlay
overlayfs:/overlay        6.5G    325.3M      5.9G   5% /
/dev/sda1               898.2M     12.0K    898.2M   0% /mnt/sda1


root@Arduino:~#


------------------ SMALL SD CARD - PROBLEM ------------------
Code: [Select]


root@Arduino2:~/repo/node# cat /usr/bin/node
#!/bin/sh

#NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs --stack_size=1024 --max_old_space_size=20 --max_new_space_size=2048 --max_executable_size=5 --gc_global --gc_interval=100 $@
NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs $@


root@Arduino2:~/repo/node# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                    1.0G     81.7M    909.6M   8% /
/dev/root                 7.5M      7.5M         0 100% /rom
tmpfs                    29.8M    152.0K     29.7M   0% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                 1.0G     81.7M    909.6M   8% /overlay
overlayfs:/overlay        1.0G     81.7M    909.6M   8% /
/dev/sda1               898.2M     12.0K    898.2M   0% /mnt/sda1


root@Arduino2:~/repo/node#


(is free logs also needed?)
Title: Re: Node.js for the YUN
Post by: mantissa00 on May 26, 2014, 04:08 pm
I'm using express (with body-parser sub-module) for rest request,
ejs as renderer,
sqlite3 as db and dblite ( https://github.com/WebReflection/dblite ) as node module for controlling it
then my setup is like your: bootstrap and jquery used in the front end for layout, glyph icons and ajax.
Title: Re: Node.js for the YUN
Post by: Squonk42 on May 26, 2014, 10:52 pm
@mantissa00: I have a 16 GB class 4 SD card with only 3% used, so this is not the problem.

The problem is more with CPU performance and RAM usage:
root@Arduino:~/node/foo# ./bin/www 
GET /contact 304 16238ms
GET /stylesheets/bootstrap.min.css 304 151ms
GET /javascripts/jquery-1.11.1.min.js 304 134ms
GET /stylesheets/style.css 304 135ms
GET /javascripts/bootstrap.min.js 304 131ms
POST /thanks 200 2873ms - 1.21kb
GET /stylesheets/bootstrap.min.css 304 63ms
GET /javascripts/jquery-1.11.1.min.js 304 60ms
GET /stylesheets/style.css 304 58ms
GET /javascripts/bootstrap.min.js 304 70ms
GET / 304 5007ms
GET /stylesheets/bootstrap.min.css 304 43ms
GET /javascripts/jquery-1.11.1.min.js 304 43ms
GET /stylesheets/style.css 304 45ms
GET /javascripts/bootstrap.min.js 304 49ms
GET /fonts/glyphicons-halflings-regular.woff 304 24ms
GET /about 304 4519ms
GET /stylesheets/bootstrap.min.css 304 43ms
GET /javascripts/jquery-1.11.1.min.js 304 45ms
GET /stylesheets/style.css 304 47ms
GET /javascripts/bootstrap.min.js 304 48ms
GET /contact 304 3426ms
GET /stylesheets/bootstrap.min.css 304 58ms
GET /javascripts/jquery-1.11.1.min.js 304 61ms
GET /stylesheets/style.css 304 62ms
GET /javascripts/bootstrap.min.js 304 65ms
GET / 304 3072ms
GET /stylesheets/bootstrap.min.css 304 30ms
GET /stylesheets/style.css 304 45ms
GET /javascripts/jquery-1.11.1.min.js 304 46ms
GET /javascripts/bootstrap.min.js 304 40ms
GET /fonts/glyphicons-halflings-regular.woff 304 19ms

And when using the memory limit options in "/usr/bin/node", I get an "process out of memory" very quickly...

So I guess that a full-stack Node.js is just too much for the Yun, and that all that can be done are only very simple scripts.

But then, what is the point in using Node.js, then? Besides the "all-JS" advantage, why not use Python Tornado or Nginx/Lua-based OpenResty in this case?

Please note that I am novice to Node.js and that I like its overall philosophy. But I am also trying to be pragmatic and find the best solution for developing web applications on the Yun. As such, I am asking eagerly for advises and experience from other users!
Title: Re: Node.js for the YUN
Post by: sonnyyu on May 30, 2014, 04:29 am

...
And when using the memory limit options in "/usr/bin/node", I get an "process out of memory" very quickly...

So I guess that a full-stack Node.js is just too much for the Yun, and that all that can be done are only very simple scripts.
...


I am not surprise, since the heart of Yún's CPU - AR9331 is already on bottom of the food chain.


FYI;-

The heart of Yún is CPU - AR9331, here is the market position of it.

AR9331 (MIPS 24k@400MHz, 1x1:1, 150Mbs/2.4GHz, PHY/100Mbs)

AR9341 (MIPS 74Kc@500Mhz, 2x2:2, 300Mbs/2.4GHz, PHY/100Mbs)

AR9344 (MIPS 74Kc@600MHz, 2x2:2, 300Mbs/2.4GHz, 450Mbs/5GHz, PHY/1Gbs)

QCA9558 (MIPS 74Kc@720MHz, 3x3, 450Mbs/2.4GHz, 1300Mbs/5GHz, PHY/1Gbs)

It is already on bottom of the food chain.