Node.js for the YUN

Hi,

I managed to cross-compile the latest node.js v0.10.20 for the YUN.

In order to save you some time, I uploaded the binaries on github, you can download them at this link:
https://raw.github.com/fibasile/yun-packages/master/node-v0.10.20-mips.tgz

The package contains only the node binary and the required .so libraries. Copy them to your sd-card or usb stick and link node in /usr/bin and the .so in /usr/lib.

If you like to know how to build it yourself please check the tutorial on my blog:

http://fibasile.github.io/compiling-nodejs-for-arduino-yun.html

If you need additional modules (as you probably will) you should build the binaries following the tutorial and also follow the instructions at the end, since you will need to build the modules on your host system and then transfer them to the YUN.

Hope you find it useful!

Great! I'll look into this and try to port it to a linino makefile

That would be great.

Even if I'm not sure how to handle building node modules, since it must be done on the host as many require gcc etc.

Unfortunately there's no way to have a "slim" package management for node, all modules encapsulate all their dependencies making it quite bloated.

For now, I think the best solution is to "package" a specific app on the host in a tgz and move it to the sd card.

Hi fibasile, just wanted you to know that we did it :slight_smile:

Thank you very much for your work :slight_smile:

Hi Frederico,

I have node running on the latest Yun image with extroot and got the console test working ok.

However, when I try to install the express framework, I get the following message:

root@Arduino:~# npm install -g express
npm http GET https://registry.npmjs.org/express
npm http 304 https://registry.npmjs.org/express
npm http GET https://registry.npmjs.org/parseurl
npm http GET https://registry.npmjs.org/methods
npm http GET https://registry.npmjs.org/send
npm http GET https://registry.npmjs.org/accepts
npm http GET https://registry.npmjs.org/cookie-signature
npm http GET https://registry.npmjs.org/type-is
npm http GET https://registry.npmjs.org/merge-descriptors
npm http GET https://registry.npmjs.org/utils-merge
npm http GET https://registry.npmjs.org/range-parser
npm http GET https://registry.npmjs.org/escape-html
npm http GET https://registry.npmjs.org/qs
npm http GET https://registry.npmjs.org/cookie
npm http GET https://registry.npmjs.org/serve-static
npm http GET https://registry.npmjs.org/path-to-regexp
npm http GET https://registry.npmjs.org/buffer-crc32
npm http GET https://registry.npmjs.org/debug
npm http GET https://registry.npmjs.org/fresh
npm http 304 https://registry.npmjs.org/methods
npm http 304 https://registry.npmjs.org/send
npm http 304 https://registry.npmjs.org/parseurl
npm http 304 https://registry.npmjs.org/accepts
npm http 304 https://registry.npmjs.org/merge-descriptors
npm http 304 https://registry.npmjs.org/type-is
npm http 304 https://registry.npmjs.org/cookie-signature
npm http 304 https://registry.npmjs.org/utils-merge
npm http 304 https://registry.npmjs.org/range-parser
npm http 304 https://registry.npmjs.org/escape-html
npm http 304 https://registry.npmjs.org/qs
npm http 304 https://registry.npmjs.org/cookie
npm http 304 https://registry.npmjs.org/serve-static
npm http 304 https://registry.npmjs.org/path-to-regexp
npm http 304 https://registry.npmjs.org/buffer-crc32
npm http 304 https://registry.npmjs.org/debug
npm http 304 https://registry.npmjs.org/fresh
npm http GET https://registry.npmjs.org/mime
npm http GET https://registry.npmjs.org/negotiator
npm http 304 https://registry.npmjs.org/mime
npm http 304 https://registry.npmjs.org/negotiator
npm http GET https://registry.npmjs.org/debug
npm http 304 https://registry.npmjs.org/debug
FATAL ERROR: Evacuation Allocation failed - process out of memory

It looks like node is crashing because it is out of memory, although I am only performing a simple install.

It makes me worry about real node usability on a 64 MB-only memory without activated swap.

Does anyone has experience with it?

I've experienced that too. It seems like npm is using a lot of memory for doing a simple install. Fortunately, node and express work well once installed.
The easiest way is probably to install express on your computer and then copy the node_modules folder on your yun.
Otherwise you can set up a temporary swap file, just for setting it up

after extending the flash memory with the sd card I have installed express without any problem.. does it matter?

I have a Yun with latest image, a 16 GB SD Card formatted using the YunDiskSpaceExpander sketch and followed the instructions for installing node using opkg.

I tried both with or without a 512 MB swap on SDCard, installing express or johnny-five from a fresh boot results in the same "FATAL ERROR: Evacuation Allocation failed - process out of memory" message =(

Here is my config:

root@Arduino:/# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                   13.6G    375.1M     12.6G   3% /
/dev/root                 7.5M      7.5M         0 100% /rom
tmpfs                    29.8M    100.0K     29.7M   0% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                13.6G    375.1M     12.6G   3% /overlay
overlayfs:/overlay       13.6G    375.1M     12.6G   3% /
/dev/sda1               698.6M     12.0K    698.6M   0% /mnt/sda1
root@Arduino:/# free
             total         used         free       shared      buffers
Mem:         61116        43064        18052            0        16560
-/+ buffers:              26504        34612
Swap:       523260          168       523092
root@Arduino:/# ps -wwww
  PID USER       VSZ STAT COMMAND
    1 root      1496 S    init
    2 root         0 SW   [kthreadd]
    3 root         0 SW   [ksoftirqd/0]
    4 root         0 SW   [kworker/0:0]
    5 root         0 SW   [kworker/u:0]
    6 root         0 SW<  [khelper]
    7 root         0 SW   [kworker/u:1]
   90 root         0 SW   [sync_supers]
   92 root         0 SW   [bdi-default]
   94 root         0 SW<  [kblockd]
  106 root         0 SW   [khubd]
  132 root         0 SW   [kswapd0]
  180 root         0 SW   [fsnotify_mark]
  211 root         0 SW<  [ath79-spi]
  222 root         0 SW   [mtdblock0]
  227 root         0 SW   [mtdblock1]
  232 root         0 SW   [mtdblock2]
  237 root         0 SW   [mtdblock3]
  242 root         0 SW   [mtdblock4]
  247 root         0 SW   [mtdblock5]
  252 root         0 SW   [mtdblock6]
  257 root         0 SW   [mtdblock7]
  310 root         0 SW   [kworker/0:1]
  312 root         0 SW   [kworker/u:2]
  339 root         0 SW   [scsi_eh_0]
  341 root         0 SW   [usb-storage]
  473 root         0 SW   [kworker/0:2]
  530 root         0 SW   [flush-mtd-unmap]
  577 root         0 SW   [flush-8:0]
  578 root         0 SW   [jbd2/sda2-8]
  579 root         0 SW<  [ext4-dio-unwrit]
  601 root      1524 S    {rcS} /bin/sh /etc/init.d/rcS S boot
  602 root      1492 S    /bin/ash --login
  604 root      1488 S    logger -s -p 6 -t sysinit
  649 root         0 SW<  [cfg80211]
  756 root      1500 S    /sbin/syslogd -l 8 -C16
  758 root      1484 S    /sbin/klogd
  760 root       872 S    /sbin/hotplug2 --override --persistent --set-rules-file /etc/hotplug2.rules --set-coldplug-cmd /sbin/udevtrigger --max-children 1
  770 root       876 S    /sbin/ubusd
  834 root      1480 S    /sbin/netifd
  897 root      1492 S    udhcpc -p /var/run/udhcpc-eth1.pid -s /lib/netifd/dhcp.script -f -t 0 -i eth1 -C
 1360 root      1628 S    wpa_supplicant -B -P /var/run/wifi-wlan0.pid -D nl80211 -i wlan0 -c /var/run/wpa_supplicant-wlan0.conf
 1456 root      1500 S    udhcpc -p /var/run/udhcpc-wlan0.pid -s /lib/netifd/dhcp.script -f -t 0 -i wlan0 -C
 1516 root      1492 S    /sbin/watchdog -t 5 /dev/watchdog
 1615 root      1152 S    /usr/sbin/dropbear -P /var/run/dropbear.1.pid -p 22
 1622 root      1564 S    /usr/sbin/uhttpd -f -h /www -r Arduino -x /cgi-bin -t 60 -T 30 -A 1 -n 2 -p 0.0.0.0:80 -C /etc/uhttpd.crt -K /etc/uhttpd.key -s 0.0.0.0:443
 1631 root      1696 S    /usr/sbin/dbus-daemon --system
 1654 nobody     948 S    /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf
 1660 nobody    2168 S    avahi-daemon: running [Arduino.local]
 1663 root       796 S    /usr/sbin/thd --socket /tmp/triggerhappy.socket --triggers /etc/triggerhappy/triggers.d/ --daemon /dev/input/event0
 1687 root      1492 S    /usr/sbin/ntpd -n -p 0.openwrt.pool.ntp.org -p 1.openwrt.pool.ntp.org -p 2.openwrt.pool.ntp.org -p 3.openwrt.pool.ntp.org
 1694 root      1496 S N  {uSDaemon} /bin/sh /sbin/uSDaemon
 2399 root      1480 S N  sleep 2
 2400 root      1488 R    ps -wwww

@mantissa00: do you see something that could be different from your setup?

There is still much room for improvement on the nodejs side
@Squonk42, try this one
Access the yun via ssh or putty (do NOT use the IDE Serial Monitor)
Enter

nano /usr/bin/node

Make file look like this

#!/bin/sh

#NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs --stack_size=1024 --max_old_space_size=20 --max_new_space_size=2048 --max_executable_size=5 --gc_global --gc_interval=100 $@
NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs $@

And retry

Hi Frederico,

Thank you very much, it works when removing the node memory options !

However, why do you recommend NOT to use the IDE serial monitor? Is it because it doesn't work well with a full screen text editor like nano?

Exactly: IDE serial monitor is a not a fully fledged Terminal. Linux terminals use control characters that you cannot send with IDE Serial Monitor. For example, you could not use the arrow keys to move to the desired text

The express installation looks it is OK, it look like it is installed in "/node/usr/lib/node_modules/express", but I cant't find the "express" command line script, and it is not in the PATH either.

Recent version of express bundle the lib only. You need to install express-generator for the command line
See GitHub - expressjs/express: Fast, unopinionated, minimalist web framework for node.

Thanks Frederico, I have everything working now!

I performed some tests with a full-stack solution consisting in:

  • NodeJS as web server with server-side Javascript language and npm package manager
  • Express as MVC web application framework
  • Jade as the template engine
  • JQuery as client-side library for DOM traversal and manipulation, animation and Ajax for asynchronous updates
  • CoffeeScript as Javascript syntactic sugar
  • Markdown for plain text formatting syntax
  • Twitter Bootstrap as front-end client-side web UI framework
  • Glyphicons font for rendering icons

It takes 15~20 s to generate a page when not in cache, 2~3 s when in cache :cold_sweat:

IMHO, Node.js is so slow that it is pretty unusable on the Yun in its current state.

Surely the Yun is not the best development environment almost for everything else than small scripts.
As for Node.js, I would stick with a "development on my pc, production on the yun" approach, just like you would do if you were to put some website on the public internet.
I would avoid everything that needs some additional computation before being used. Jade for example needs to compile its templates at the first request and once compiled they go to memory. Try using another template engine that allows you to compile templates "offline" (I used dust.js in the past). Also avoid running coffeescript on the server side: precompile it to javascript (don't get me wrong: I'm a big fan of coffeescript)

One more thing. As you know nodejs is single threaded. In your example, that means that it's either giving you back the glaphicons or it's answering some "hello world" request. Uglify and pack all the assets in few files (ideally: one js and one css)

@Squonk42
I have a fresh install on a new yun with a smaller sd card (2GB instead of 8GB), and I had that issue.

here the df logs:

------------------ BIG SD CARD - NO PROBLEM ------------------

root@Arduino:~# cat /usr/bin/node
#!/bin/sh

NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs --stack_size=1024 --max_old_space_size=20 --max_new_space_size=2048 --max_executable_size=5 --gc_global --gc_interval=100 $@


root@Arduino:~# df -h

Filesystem                Size      Used Available Use% Mounted on
rootfs                    6.5G    325.3M      5.9G   5% /
/dev/root                 7.5M      7.5M         0 100% /rom
tmpfs                    29.8M    164.0K     29.7M   1% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                 6.5G    325.3M      5.9G   5% /overlay
overlayfs:/overlay        6.5G    325.3M      5.9G   5% /
/dev/sda1               898.2M     12.0K    898.2M   0% /mnt/sda1


root@Arduino:~#

------------------ SMALL SD CARD - PROBLEM ------------------

root@Arduino2:~/repo/node# cat /usr/bin/node
#!/bin/sh

#NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs --stack_size=1024 --max_old_space_size=20 --max_new_space_size=2048 --max_executable_size=5 --gc_global --gc_interval=100 $@
NODE_PATH=/usr/lib/node_modules /usr/bin/nodejs $@


root@Arduino2:~/repo/node# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                    1.0G     81.7M    909.6M   8% /
/dev/root                 7.5M      7.5M         0 100% /rom
tmpfs                    29.8M    152.0K     29.7M   0% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                 1.0G     81.7M    909.6M   8% /overlay
overlayfs:/overlay        1.0G     81.7M    909.6M   8% /
/dev/sda1               898.2M     12.0K    898.2M   0% /mnt/sda1


root@Arduino2:~/repo/node#

(is free logs also needed?)

I'm using express (with body-parser sub-module) for rest request,
ejs as renderer,
sqlite3 as db and dblite ( GitHub - WebReflection/dblite: sqlite for node.js without gyp problems ) as node module for controlling it
then my setup is like your: bootstrap and jquery used in the front end for layout, glyph icons and ajax.

@mantissa00: I have a 16 GB class 4 SD card with only 3% used, so this is not the problem.

The problem is more with CPU performance and RAM usage:

root@Arduino:~/node/foo# ./bin/www 
GET /contact 304 16238ms
GET /stylesheets/bootstrap.min.css 304 151ms
GET /javascripts/jquery-1.11.1.min.js 304 134ms
GET /stylesheets/style.css 304 135ms
GET /javascripts/bootstrap.min.js 304 131ms
POST /thanks 200 2873ms - 1.21kb
GET /stylesheets/bootstrap.min.css 304 63ms
GET /javascripts/jquery-1.11.1.min.js 304 60ms
GET /stylesheets/style.css 304 58ms
GET /javascripts/bootstrap.min.js 304 70ms
GET / 304 5007ms
GET /stylesheets/bootstrap.min.css 304 43ms
GET /javascripts/jquery-1.11.1.min.js 304 43ms
GET /stylesheets/style.css 304 45ms
GET /javascripts/bootstrap.min.js 304 49ms
GET /fonts/glyphicons-halflings-regular.woff 304 24ms
GET /about 304 4519ms
GET /stylesheets/bootstrap.min.css 304 43ms
GET /javascripts/jquery-1.11.1.min.js 304 45ms
GET /stylesheets/style.css 304 47ms
GET /javascripts/bootstrap.min.js 304 48ms
GET /contact 304 3426ms
GET /stylesheets/bootstrap.min.css 304 58ms
GET /javascripts/jquery-1.11.1.min.js 304 61ms
GET /stylesheets/style.css 304 62ms
GET /javascripts/bootstrap.min.js 304 65ms
GET / 304 3072ms
GET /stylesheets/bootstrap.min.css 304 30ms
GET /stylesheets/style.css 304 45ms
GET /javascripts/jquery-1.11.1.min.js 304 46ms
GET /javascripts/bootstrap.min.js 304 40ms
GET /fonts/glyphicons-halflings-regular.woff 304 19ms

And when using the memory limit options in "/usr/bin/node", I get an "process out of memory" very quickly...

So I guess that a full-stack Node.js is just too much for the Yun, and that all that can be done are only very simple scripts.

But then, what is the point in using Node.js, then? Besides the "all-JS" advantage, why not use Python Tornado or Nginx/Lua-based OpenResty in this case?

Please note that I am novice to Node.js and that I like its overall philosophy. But I am also trying to be pragmatic and find the best solution for developing web applications on the Yun. As such, I am asking eagerly for advises and experience from other users!

Squonk42:
...
And when using the memory limit options in "/usr/bin/node", I get an "process out of memory" very quickly...

So I guess that a full-stack Node.js is just too much for the Yun, and that all that can be done are only very simple scripts.
...

I am not surprise, since the heart of Yún's CPU - AR9331 is already on bottom of the food chain.

sonnyyu:
FYI;-

The heart of Yún is CPU - AR9331, here is the market position of it.

AR9331 (MIPS 24k@400MHz, 1x1:1, 150Mbs/2.4GHz, PHY/100Mbs)

AR9341 (MIPS 74Kc@500Mhz, 2x2:2, 300Mbs/2.4GHz, PHY/100Mbs)

AR9344 (MIPS 74Kc@600MHz, 2x2:2, 300Mbs/2.4GHz, 450Mbs/5GHz, PHY/1Gbs)

QCA9558 (MIPS 74Kc@720MHz, 3x3, 450Mbs/2.4GHz, 1300Mbs/5GHz, PHY/1Gbs)

It is already on bottom of the food chain.