I've now used the native gcc to build a couple of packages... Seems to be working pretty nice.
I hooked up an old 4Gb SSD I had and used that (remounted async, ext4) to give me some working space, and it really helps with i/o performance over the SD card - but maybe thats cause I've got a cheap SD card.
Building some packages I came across a couple of niggles which I had to work round:
The Yun tar is part of busy box, and recent tar archives (gnu tar?) break it. So we'll need to install a new tar, but that breaks because busy box grep isn't powerful enough... you could install gnu grep 2.9 but that uses a "new" tar archive....
Gnu grep 2.0 downloads/untars and compiles fine.
Gnu Tar 1.27.1 needs new grep. Even when you remove "/bin/grep" it still gets picked up as its part of busy box, to configure you need to use:
./configure GREP=/usr/local/bin/grep
Most configure scripts will probably need this.
Now you've got a real tar, you can go back and upgrade grep to the latest version if you want to, but there doesn't seem a lot of point...
Bzip2 is needed for a few builds. It compiles and runs in the standard environment without incident.