Pages: [1]   Go Down
Author Topic: Uploading hex over tcpip in arduino network  (Read 2164 times)
0 Members and 1 Guest are viewing this topic.
0
Offline Offline
Jr. Member
**
Karma: 0
Posts: 73
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I am visualizing a large network of several hundred arduinos (328p) connected over TCPIP and Xbee's.
I need to find a solution to program these remotely by addressing each node and sending it a modified
hex file. The network control language is Java.
From what I have seen so far it seems I may have to invoke Avrdude from inside Java and give it the
tcp port to transmit over; the Java program would ensure the stream is directed to the appropriate
XBee-connected Arduino. However, the Avrdude docs state that TCP port addressing is not supported
under Windows. That seems to break my strategy. Any other ideas, anybody, please?
I suppose since I'm on Java I could move to 'Unix' or Mac instead.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 211
Posts: 13477
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Arduinos are programmed serial ,  Zigbee is serial, think it is done allready => search the forum for wireless flashing

TCP/IP is quite different. <entering brainstorm modus>

1) make Arduino pairs where one A. gets the hexcode over TCP/IP and stores it in a shared EEPROM. Then it can use it to flash the other Arduino. If this is done the second one can do whatever it needs to (including upgrading the first)

2) make a interpreter, lets call it BASIC, and send a new BASIC program to the A. the app is stored in EEPROM (again) and when all is in the interpreter starts it by running the app. =>   http://lmgtfy.com/?q=tiny+basic+interpreter+in+C

my 2 cents,
Rob


Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

0
Offline Offline
God Member
*****
Karma: 1
Posts: 513
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Here is what I wrote in February 2009 to upload files to the arduino bootloader from Java:
Code:
/* java based uploader for arduino and any other %100 open source projects,
 might also work with stk500 by chance  */

//uses rxtx http://users.frii.com/jarvi/rxtx/download.html

//courtesy dcb AT opengauge.org

import gnu.io.CommPortIdentifier;
import gnu.io.SerialPort;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.InputStream;
import java.io.OutputStream;
import java.lang.reflect.Method;

public class ArduinoLoader {

public static final long rxtimeoutms = 5000;

public static final byte[] hello = { 0x30, 0x20 };

public static final byte[] okrsp = { 0x14, 0x10 };

public static int startaddr = 0x00;

static InputStream input2;

static OutputStream output2;

public static int waitForChar(InputStream i) throws Exception {
long t = System.currentTimeMillis();
while (i.available() == 0) {
try {
Thread.sleep(50);
} catch (Exception e) {
}
if (System.currentTimeMillis() - t > rxtimeoutms) {
throw new Exception("Timed out waiting for response");
}
}
return i.read();
}

public static void chkok(InputStream i) throws Exception {
if ((waitForChar(i) != okrsp[0]) || (waitForChar(i) != okrsp[1]))
throw new Exception("Invalid response");
}


static int[] image = new int[200000];

static int imagesize = 0;

// pass hex file, com port ID, baud, page size
public static void main(String[] args) throws Exception {
long g = System.currentTimeMillis();
// args = new String[] { "/Blink.hex", "COM4", "19200", "128" };
if (args.length != 4) {
System.out
.println("Arguments: full_path_to_hex_file com_port_id baud page_size");
System.exit(1);
}



upload(args[0], args[1], Integer.parseInt(args[2]), Integer
.parseInt(args[3]), System.out);
g=System.currentTimeMillis()-g;
System.out.println("Completed, " + imagesize + " bytes uploaded " + g);

}

static void println(String s, OutputStream o) throws Exception {
o.write((s + "\n").getBytes());
}

public static void upload(String filename, String comport, int baud,
int pageSize, OutputStream out) throws Exception {
println("Serial Proxy Starting, port:" + comport + ", baud:" + baud
+ ", pagesize:" + pageSize + ", file:" + filename, out);

// load the hex file into memory
parsehex(filename);

// out=new FileOutputStream("/arduinoloader.log");
out = System.out;

CommPortIdentifier portId2 = CommPortIdentifier
.getPortIdentifier(comport);

SerialPort port2 = (SerialPort) portId2.open("serial madness2", 4001);
input2 = port2.getInputStream();
output2 = port2.getOutputStream();
port2.setSerialPortParams(baud, SerialPort.DATABITS_8,
SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);

port2.setDTR(false);
output2.write(hello);

try {
Thread.sleep(100);
} catch (InterruptedException e) {
}

while (input2.available() > 0) {
input2.read();
}
;

for (int x = 0; x < 3; x++) {
try {
output2.write(hello);
chkok(input2);
} catch (Exception e) {
}
try {
Thread.sleep(500);
} catch (Exception e) {
}
}
output2.write(hello);
chkok(input2);

// write the hex file
int addr = 0;
while (addr < imagesize) {
output2.write('U');
output2.write((addr / 2) % 256);
output2.write((addr / 2) / 256);
output2.write(' ');
chkok(input2);

int ps = Math.min(pageSize, imagesize - addr);

output2.write('d');
output2.write(ps / 256);
output2.write(ps % 256);
output2.write('F');
for (int x = 0; x < ps; x++) {
output2.write(image[addr + x]);
}
output2.write(' ');
chkok(input2);
addr += ps;
}

// validate the image
addr = 0;
output2.write('U');
output2.write((addr / 2) % 256);
output2.write((addr / 2) / 256);
output2.write(' ');
chkok(input2);

output2.write('t');
output2.write(imagesize / 256);
output2.write(imagesize % 256);
output2.write('F');
output2.write(' ');

if ((waitForChar(input2) != okrsp[0]))
throw new Exception("Invalid response");

while (addr < imagesize) {
int c = waitForChar(input2);
if (c != image[addr])
throw new Exception("Validation error at offset " + addr
+ ".  Expected " + image[addr] + " received " + c);
addr++;
// System.out.print(hexval(c));
if (addr % 16 == 0)
System.out.println("");
}

if ((waitForChar(input2) != okrsp[1]))
throw new Exception("Invalid response");

output2.write('Q');
output2.write(' ');
chkok(input2);

port2.setDTR(true);
port2.close();
}

public static void waitfor(int w) throws Exception {
int c;
do {
while (input2.available() == 0)
;
c = input2.read();
System.out.println((char) c + " " + (int) c);
} while (c != w);
}

static String hexvals[] = { "0", "1", "2", "3", "4", "5", "6", "7", "8",
"9", "A", "B", "C", "D", "E", "F" };

static String hexval(int i) {
return hexvals[i / 16] + hexvals[i % 16];
}

static void parsehex(String fname) throws Exception {
BufferedReader in = new BufferedReader(new FileReader(fname));
String t = in.readLine();
int line = 0;
imagesize = 0;
while (t != null) {
line++;
if (!":".equals(t.substring(0, 1)))
throw new Exception("line#" + line
+ " Invalid format in hex file " + fname);
int len = Integer.parseInt(t.substring(1, 3), 16);
imagesize += len;
int addr = Integer.parseInt(t.substring(3, 7), 16);
int type = Integer.parseInt(t.substring(7, 9), 16);
String datav = t.substring(9, 9 + (len * 2));
int[] data = new int[datav.length() / 2];
for (int x = 0; x < data.length; x++) {
data[x] = Integer.parseInt(datav.substring(x * 2, x * 2 + 2),
16);
}
int cksum = Integer.parseInt(t.substring(9 + (len * 2),
11 + (len * 2)), 16);

// compute checksum of line just read
int cks = (256 - len) + (256 - (addr / 256)) + (256 - (addr % 256))
+ (256 - type);
for (int x = 0; x < data.length; x++) {
cks += (256 - data[x]);
}
cks %= 256;
if (cks != cksum)
throw new Exception("line#" + line
+ " Invalid checksum in hex file " + fname);

// copy to the image so we can work with the page size easier
for (int x = 0; x < data.length; x++) {
image[addr + x] = data[x];
}

t = in.readLine();
}

}

}
 

to compile it I just saved it to ArduinoLoader.java in my arduino directory (Ardunino-0016 here) and ran:
C:\arduino-0016>javac -cp lib\RXTXcomm.jar ArduinoLoader.java

I run it from the arduino directory where the rxtx dll is (windows), i.e.:
C:\arduino-0016>java -cp .;lib\RXTXcomm.jar ArduinoLoader /tmp/blink.hex COM4 19200 128

Make sure you have the baud rate and com port set correctly, and that it is pointed to a valid hex file.
C:\arduino-0016\hardware>grep upload boards.txt|grep speed
atmega328.upload.speed=57600
diecimila.upload.speed=19200
mega.upload.speed=57600
mini.upload.speed=19200
nano.upload.speed=19200
bt.upload.speed=19200
lilypad328.upload.speed=57600
lilypad.upload.speed=19200
pro328.upload.speed=57600
pro.upload.speed=19200
atmega168.upload.speed=19200
atmega8.upload.speed=19200

It may also be that they have added some required stk500-like commands to the newer bootloaders, I haven't made any effort to stay "current" in arduino these days, unapologetically.  But give it a try with the right speed for your board and a valid path to a hex file and com port first.  Swapping boards can cause the com port to change, fyi.

It *should* be trivial to incorporate into a java based webserver (i.e. tomcat) once you get it working from a command line, and control it over the web, just watch your PATH and CLASSPATH on startup.  Also consider securing access to the web facing uploader as there are a limited number of flash writes on the chip.

Alternately you can shell out to avrdude from your web application like the IDE does, only I found avrdude to be quite cryptic as it tries to be many things to many people.

If you know you are working with arduino bootloaders (stk500 compatible) and valid hex files, then the above seems a reasonable approach, and helps demystify the whole upload process.  You will want to compile the hexs for their specific boards however as some boards have different clock rates and who knows what else.


History, I wrote this so there would be a loader that put less demands on the bootloader, so the bootloader could be shrunk back down in size:
http://arduino.cc/forum/index.php/topic,46486.0.html

« Last Edit: March 14, 2011, 05:08:37 am by dcb » Logged

0
Offline Offline
God Member
*****
Karma: 1
Posts: 513
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Also worth noting that using a serversocket in java is an option as well, you don't need a whole tomcat installation to put a program together that responds to network requests, but you do need to know how the network actually works.
Logged

0
Offline Offline
Jr. Member
**
Karma: 0
Posts: 73
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You have groundbreaking code - it's exactly what's needed to program-load a network of Arduino's, where each may have different code depending on what its function is, and where only some may have to be re-loaded. I already modified Andrew Rapp's famous Xbee libraries to remove RxTx and replace it with an encrypted socket handler. I'd like to do the same here.

When I try it against 3 different 328p's I do get timeouts or "invalid response" though. It would be nice if somebody could verify that.
I am looking at the bootloader source code to find out what's missing. There are control tables in there that are chip-specific, and
it's going to be a long haul. A few hints from the powers that manage that code would be nice...
Logged

0
Offline Offline
Jr. Member
**
Karma: 0
Posts: 73
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

DCB: your code works! But it has to be used at 57600 baud on Arduino-0021 for Atmega328p. Don't know how to override the uploader's speed. It looks like that may be built-in via boards.txt at bootloader burn time. Or it's used in some arcane fashion by the avrdude.
Good job.
« Last Edit: March 16, 2011, 04:08:46 pm by maxmike » Logged

0
Offline Offline
God Member
*****
Karma: 1
Posts: 513
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

very glad you got it working smiley  the upload speed is a function of the bootloader installed on the chip FYI.

I enjoyed writing it as SPI (and the bootloader) was pretty much a mystery to me before this. 
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 2
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Posting here for people still looking for uploading hex over tcp.

You can do upload over tcp directly with avrdude (search for tcp in man page), here's what I'm doing on ubuntu

I have a terminal server bridging TCP port 4000 to serial port at 57600 connected to arduino.

Then
/opt/arduino/hardware/tools/avrdude -q -V -p atmega328p -C /opt/arduino/hardware/tools/avrdude.conf -c stk500v1 -P net:192.168.0.117:4000  -U flash:w:build-cli/arduino.hex:i

this works fine if your arduino is running the bootloader but you can't set DTR/RTS for reset, so now what?

Here's my next trick. I have a jmp in my arduino pde that directly jumps to the bootloader. I can call that jmp by sending my 'reset' command over the tcp/serial/arduino connection prior to running avrdude. Here's my jmp code from my pde.

void reset() {
    asm volatile("jmp 0x3E00"); /* dont know where I got this but it works on 328 */
}

in this pde, I have to send a 'R' char over the connection which calls the pde's reset function.

Here's the printout of an upload to terminal server running at 192.168.0.117. I'm using netcat, nc, to send the 'R' char over the connection.

holla@knopfler[521]: echo -ne "R" | nc 192.168.0.117 4000;/opt/arduino/hardware/tools/avrdude -q -V -p atmega328p -C /opt/arduino/hardware/tools/avrdude.conf -c stk500v1 -P net:192.168.0.117:4000  -U flash:w:build-cli/arduino.hex:i

avrdude: AVR device initialized and ready to accept instructions
avrdude: Device signature = 0x1e950f
avrdude: NOTE: FLASH memory has been specified, an erase cycle will be performed
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: reading input file "build-cli/arduino.hex"
avrdude: writing flash (8814 bytes):
avrdude: 8814 bytes of flash written

avrdude: safemode: Fuses OK

avrdude done.  Thank you.


Hope this helps.
Logged

Pages: [1]   Go Up
Jump to: