Arduino Forum

Products => Arduino Yún => Topic started by: dctsystems on May 20, 2014, 11:28 am

Title: Native GCC for Yun
Post by: dctsystems on May 20, 2014, 11:28 am
For a lot of the stuff I'm interested in doing, having an Arduino that is totally stand-alone would be a big benefit. The Yun should fit the bill, as the Linux side is a reasonably powerful MIPS processor. Unfortuantly there's no C compiler... The only way to compile C for it was to cross-compile. That would be OK-ish, but setting up a build environment is a real pain.

I've said a few times that it shouldn't be so hard to build a compiler, but last week Frederico called me on it,  if I wanted it, I should pitch in and do it.

So here it is : GCC for Yun!
http://www.sniff.org.uk/2014/05/building-gcc-for-yun-in-12-not-easy.html

The link explains how to build it, and what problems I hit. Basically it was all reasonably straight forwards, but with a lot of minor hassles and glitches.  In addition to instructions for "rolling your own", there's also a link where you can download the pre-built files.

I've only done basic testing on it, and I'm sure there will be glitches: however it compiles C programs natively on the Yun. Cross compiling is still the best option if you're doing heavy builds, but for light use this is far easier.

Ian
Title: Re: Native GCC for Yun
Post by: federicofissore on May 20, 2014, 01:42 pm
Thank you Ian
Please consider joining effort/support with noblepepper that volunteered in porting your instructions into an openwrt makefile, so that gcc becomes an installable package
https://github.com/arduino/openwrt-yun/issues/13
Title: Re: Native GCC for Yun
Post by: NewLine on May 20, 2014, 10:40 pm
Great stuff! Cant wait to be able to write C++ for Yun  8)
Title: Re: Native GCC for Yun
Post by: dctsystems on May 24, 2014, 12:50 pm
I've now used the native gcc to build a couple of packages... Seems to be working pretty nice.

I hooked up an old 4Gb SSD I had and used that  (remounted async, ext4) to give me some working space, and it really helps with i/o performance over the SD card - but maybe thats cause I've got a cheap SD card.

Building some packages I came across a couple of niggles which I had to work round:

The Yun tar is part of busy box, and recent tar archives (gnu tar?) break it. So we'll need to install a new tar, but that breaks because busy box grep isn't powerful enough... you could install gnu grep 2.9 but that uses a "new" tar archive....

Gnu grep 2.0 downloads/untars and compiles fine.

Gnu Tar 1.27.1 needs new grep. Even when you remove "/bin/grep" it still gets picked up as its part of busy box, to configure you need to use:

./configure GREP=/usr/local/bin/grep

Most configure scripts will probably need this.

Now you've got a real tar, you can go back and upgrade grep to the latest version if you want to, but there doesn't seem a lot of point...

Bzip2 is needed for a few builds. It compiles and runs in the standard environment without incident.

Title: Re: Native GCC for Yun
Post by: noblepepper on May 24, 2014, 11:04 pm
I started from Ian's blog and almost have it where I can make an ipkg. When I get the pull request ready for the main repostory, I'll put a link here, it will be for the VERY BRAVE.
Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 02, 2014, 08:50 pm
FYI;-

Native GCC for up stream OS-Openwrt  exists for ages, might be 4~5 years.  PACKAGE_gcc  and PACKAGE_build-essential  is only for x86 platform since hardware limit.

but situation greatly changed since newer router has much powerful hardware.

Linksys WRT1900AC Router (Openwrt based)
CPU: Marvell MV78230-B0 (ARMADA XP) (1.2 GHz, 2 cores)
FLASH: 128 MiB
RAM: 256 MiB
USB 3.0, USB 2.0 / eSATA
Title: Re: Native GCC for Yun
Post by: federicofissore on Jun 02, 2014, 11:24 pm
OMG, you're right
https://github.com/arduino/openwrt-packages-yun/blob/master/devel/gcc/Makefile
Does it mean we can just add TARGET_ar71xx and it will work?
Title: Re: Native GCC for Yun
Post by: federicofissore on Jun 02, 2014, 11:35 pm
Just tried and the package become selectable. I still wonder how I could have missed it
Anyway I also had to remove the @NO_STRIP filter. Anyone has a clue about why I should avoid stripping gcc binaries?
Title: Re: Native GCC for Yun
Post by: noblepepper on Jun 04, 2014, 11:01 am
Did this compile for you?

I tried the makefile as is and the download of the source tarball failed, it was looking for a linaro release which doesn't seem to be available at the URL's provided. I changed the release to a simple 4.6.2 and it doesn't like my autoconf version.

!!!!????
Title: Re: Native GCC for Yun
Post by: federicofissore on Jun 04, 2014, 02:53 pm
No, I still haven't tried. I just tried to see if the package popped up with "make menuconfig" once the x86 filter was disabled and it did
Title: Re: Native GCC for Yun
Post by: noblepepper on Jun 04, 2014, 06:05 pm
OK, not me fat fingering something. I have a working GCC, but have struggled trying to get openwrt to make a package. With this as an example I think I can get a ipkg.
Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 05, 2014, 03:01 pm

OMG, you're right
https://github.com/arduino/openwrt-packages-yun/blob/master/devel/gcc/Makefile
Does it mean we can just add TARGET_ar71xx and it will work?


I guess not, at least not for me each TARGET and  Openwrt release I have to manually modify Makefile.


Just tried and the package become selectable. I still wonder how I could have missed it
Anyway I also had to remove the @NO_STRIP filter. Anyone has a clue about why I should avoid stripping gcc binaries?


Use "STRIP:" at buildroot./Makefile.


Did this compile for you?

I tried the makefile as is and the download of the source tarball failed, it was looking for a linaro release which doesn't seem to be available at the URL's provided. I changed the release to a simple 4.6.2 and it doesn't like my autoconf version.

!!!!????


I flip coin,  "4.6.3" is luck number for current Yun OS.

I am a big fan for "Native GCC for Yun", but not for GCC running at Yun. It is for GCC running powerful MIPS CPU or High speed MIPS emulator.  compile Yun OS took me 4 hours then compile GCC  took 30 mins, opkg install GCC into it took  few mins, but compile some C code could take 10+ hours. No way it is too much. I rather account finger than use computer. ( I only did it at old Yun's OS)

Title: Re: Native GCC for Yun
Post by: dctsystems on Jun 06, 2014, 12:05 pm
compile some C code could take 10+ hours. No way it is too much. I rather account finger than use computer. ( I only did it at old Yun's OS)


I'm not sure what you're trying to compile on the Yun, but I've compiled a bunch of stuff and performance was OK. The only things that are going to take anywhere near 10 hours are something like gcc, or the OS distro. For compiling huge packages it obviously makes sense to cross compile but for example when I needed a "real" grep it took only a few minutes to compile and install gnugrep. Then again I'm old enough to remember when a 400Hz Mips WAS a high end machine.

For a typical user, cross compiling is a near non-starter: just setting a Debian VM is enough work! Building the openWRT distro on that is a pain (and it regularly fails!). As you noted you spent hours getting to the stage of STARTING to compile your project. Also a lot of packages aren't set up to cross compile nicely.

Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 06, 2014, 05:52 pm
Quote
I'm not sure what you're trying to compile on the Yun, but I've compiled a bunch of stuff and performance was OK. The only things that are going to take anywhere near 10 hours are something like gcc, or the OS distro.


Opencv,  It took almost same time as gcc at  cross compile,  at cross compile Opencv could be compiled on some hardware target but not all. Native GCC in high speed MIPS CPU  could help. The list could go on and on.

Quote
For compiling huge packages it obviously makes sense to cross compile but for example when I needed a "real" grep it took only a few minutes to compile and install gnugrep.


agree

Quote
Then again I'm old enough to remember when a 400MHz Mips WAS a high end machine.


My first computer ( in kits format and I soldered it together when I was kids)
Sinclair ZX81
CPU   Z80 at 3.25 MHz
Memory   1 kB (64 kB max. 56 kB usable)

(http://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/Sinclair-ZX81.png/375px-Sinclair-ZX81.png)

but now there are multiple cores MIPS based computer could run a couple of GHz with a lot of memory.

(http://www.technologyreview.com/sites/default/files/images/loongson.chipx299.jpg)

very environmental green, could run without big heat sink or fan.

http://en.wikipedia.org/wiki/Loongson (http://en.wikipedia.org/wiki/Loongson)

Quote
For a typical user, cross compiling is a near non-starter: just setting a Debian VM is enough work! Building the openWRT distro on that is a pain (and it regularly fails!).


I start with openvz since it performance near type 0 hypervisor,  but back to bear metal + ghost for 10% gain. Now plan to migrate to Xen for Amazon Web Services.


...
The Atheros AR9331 (Hornet) is a Wi-Fi System-On-Chip (WiSOC), typically used in AccessPoints and router platforms. It is based on MIPS @400MHz 32 bits CPU with no floating point support, has 64 MB DDR2 RAM and 16 MB Flash Memory.
...
One of sample Web Services: Amazon Web Services (cr1.8xlarge) has 32 vCPU (32 of 1.7 GHz Xeon processor 64 bits CPU) and 244 GB RAM plus 2 x 120 SSD driver.
...


VirtualBox is type 2 hypervisor will be suffer more than type 1 hypervisor.

http://forum.arduino.cc/index.php?topic=235360.msg1726450#msg1726450 (http://forum.arduino.cc/index.php?topic=235360.msg1726450#msg1726450)

Quote
As you noted you spent hours getting to the stage of STARTING to compile your project.


4~5 hours to compile complete OS on bear metal 8 cores/16GB RAM/120GB SSD, then 30 mins for gcc and 28 mins for opencv. both could failed and offen will be failed. hit enter key and pray.

Quote
Also a lot of packages aren't set up to cross compile nicely.


True, but after careful tune up, I manage keep it at single digit. I do get ~4000 packages from  trunk.
Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 06, 2014, 06:55 pm
Method of compile:

Plan A:  openwrt buildroot
http://wiki.openwrt.org/doc/howto/buildroot.exigence (http://wiki.openwrt.org/doc/howto/buildroot.exigence)

Plan B:  Embedded Linux Development Kit
http://www.denx.de/wiki/DULG/ELDK (http://www.denx.de/wiki/DULG/ELDK)

Plan C: Sourcery CodeBench for MIPS GNU/Linux, Lite Edition is free.
http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/lite/?cmpid=7108&lite=MIPS&target_os=GNU%2FLinux&target_arch=MIPS (http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/lite/?cmpid=7108&lite=MIPS&target_os=GNU%2FLinux&target_arch=MIPS)

Plan D: Native GCC  at High speed MIPS CPU.

Plan E: Native GCC  at  MIPS QEMU emulator

Plan F: Native GCC run at Yun









Title: Re: Native GCC for Yun
Post by: arbor on Jun 06, 2014, 07:51 pm
Cross-compiling for the Yun I use the OpenWRT toolchain.  It works rather well.  Just set up your environment variables, configure for the mips target and your working.

Native GCC on the Yun is really only for playing around or if you an amazing amount of patience.  I got it working just because I thought I could.

I was able to compile a native mips avr-gcc toolchain for the Yun.  I'm testing that out with Arduino Makefile.  It's working well.  It's not as slow as I was expecting and is very usable.  I have my whole avr-tools setup as a nfs mount to the Yun and am satisfied with it.  I was going to make it a package, but having it separate from the main root filesystem is much easier and to me cleaner.

As far as qemu virtual mips there are some decent choices.  You can use the malta snapshot from OpenWRT trunk (http://downloads.openwrt.org/snapshots/trunk/malta/ (http://downloads.openwrt.org/snapshots/trunk/malta/)).  It's specifically for running in qemu . 
There's also the yocto project (https://www.yoctoproject.org/ (https://www.yoctoproject.org/))that is rather easy to setup by reading the documentation.
Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 07, 2014, 04:15 pm

...
Native GCC on the Yun is really only for playing around or if you an amazing amount of patience.  I got it working just because I thought I could.

I was able to compile a native mips avr-gcc toolchain for the Yun.  I'm testing that out with Arduino Makefile.  It's working well.  It's not as slow as I was expecting and is very usable.  I have my whole avr-tools setup as a nfs mount to the Yun and am satisfied with it.  I was going to make it a package, but having it separate from the main root filesystem is much easier and to me cleaner.
...


:P
Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 07, 2014, 04:19 pm
Plan G: yocto project

https://www.yoctoproject.org/ (https://www.yoctoproject.org/)

Plan H:emdebian

http://www.emdebian.org/ (http://www.emdebian.org/)

Plan I: openembedded

http://www.openembedded.org/wiki/Main_Page (http://www.openembedded.org/wiki/Main_Page)

Plan J: uclibc Buildroot

http://buildroot.uclibc.org/ (http://buildroot.uclibc.org/)
Title: Re: Native GCC for Yun
Post by: sonnyyu on Jun 09, 2014, 06:23 pm

...
As far as qemu virtual mips there are some decent choices.  You can use the malta snapshot from OpenWRT trunk (http://downloads.openwrt.org/snapshots/trunk/malta/ (http://downloads.openwrt.org/snapshots/trunk/malta/)).  It's specifically for running in qemu . 
...


I found few catch about malta;-

http://downloads.openwrt.org/snapshots/trunk/malta/ (http://downloads.openwrt.org/snapshots/trunk/malta/) is for little-endian image, AR9331 is big-endian, I manage to compile big-endian image as well as GCC for it, but Yun's OS is attitude_adjustment and so no go here.

http://downloads.openwrt.org/attitude_adjustment/12.09/malta/generic/ (http://downloads.openwrt.org/attitude_adjustment/12.09/malta/generic/) is empty, means upstream OS fail to compile it.
I manage to compile big-endian image for attitude_adjustment and have not chance to compile GCC yet.

an other problem is qemu's mips support is limit at Max 256MiB  RAM. I would like to have 4GiB RAM.

Title: Re: Native GCC for Yun
Post by: noblepepper on Jun 14, 2014, 06:22 pm
I have finally gotten a Makefile that produces an install-able, working gcc package.  You can get the package here https://www.dropbox.com/s/gdgjpmz845o5xl2/gcc_4.6.2-1_ar71xx.ipk (https://www.dropbox.com/s/gdgjpmz845o5xl2/gcc_4.6.2-1_ar71xx.ipk)  It is big, over 60mB and there are probably some issues that still need to be taken care of so please try it and let me know if it works for you.

@federico - one of the last things I found was that if you let openwrt strip the files compiling on the Yun chokes at the linking stage because it can't find symbols it needs.

To try it follow these steps:
1. Put an sd card in the Yun and run the diskSpaceExpander sketch http://arduino.cc/en/Tutorial/ExpandingYunDiskSpace (http://arduino.cc/en/Tutorial/ExpandingYunDiskSpace) , I am using a 16 gig sdcard with 10000 Mb data partion leaving 6 gig for the system.

2. Install binutils
Code: [Select]
root@Arduino:~# opkg update
Downloading http://downloads.arduino.cc/openwrtyun/1/packages/Packages.gz.
Updated list of available packages in /var/opkg-lists/attitude_adjustment.
Downloading http://downloads.arduino.cc/openwrtyun/1/packages/Packages.sig.
opkg install Signature check passed.
root@Arduino:~# opkg install binutils
Installing binutils (2.22-5) to root...
Downloading http://downloads.arduino.cc/openwrtyun/1/packages/binutils_2.22-5_ar71xx.ipk.
Installing objdump (2.22-5) to root...
Downloading http://downloads.arduino.cc/openwrtyun/1/packages/objdump_2.22-5_ar71xx.ipk.
Configuring objdump.
Configuring binutils.
root@Arduino:~#


3. Copy the ipk to your data partition. On linux I use:
Code: [Select]
scp gcc_4.6.2-1_ar71xx.ipk arduino.local:/mnt/sda1


4. Install gcc, this takes a LONG time so be patient.
Code: [Select]
root@Arduino:~# opkg install /mnt/sda1/gcc_4.6.2-1_ar71xx.ipk
Installing gcc (4.6.2-1) to root...
Configuring gcc.
root@Arduino:~#


5. Give it a source file named hello.c to compile, this works:
Code: [Select]
#include <stdio.h>

void main(void)
{
   printf("%s","hello world\n");
}


6. Compile it
Code: [Select]
root@Arduino:~# gcc -fno-use-linker-plugin -msoft-float hello.c -o hello
hello.c: In function 'main':
hello.c:5:5: warning: incompatible implicit declaration of built-in function 'printf' [enabled by default]
root@Arduino:~#


7. run the executable:
Code: [Select]
root@Arduino:~# ./hello
hello world
root@Arduino:~#


8. Yes it is slow:
Code: [Select]
time  gcc -fno-use-linker-plugin -msoft-float hello.c -o hello
hello.c: In function 'main':
hello.c:5:5: warning: incompatible implicit declaration of built-in function 'printf' [enabled by default]
real 0m 0.53s
user 0m 0.34s
sys 0m 0.10s


I need to clean up the makefile and then I'll put in a pull request to the yun repository so it can be built by anyone who wants to.
Title: Re: Native GCC for Yun
Post by: federicofissore on Jun 14, 2014, 07:03 pm
Can't wait to see it, amazing work :)
Title: Re: Native GCC for Yun
Post by: federicofissore on Sep 12, 2014, 09:35 am
Reviving this old topic to make everyone aware that, thanks to the work done here (https://github.com/arduino/openwrt-packages-yun/pull/7), a package name "yun-gcc" is now available. Thanks noblepepper and everyone for their past and future contributions
Title: Re: Native GCC for Yun
Post by: dctsystems on Sep 16, 2014, 11:37 pm
Thanks to noblepepper for turning my hacked code into something that's now part of the main tree!

nice one guys!