Getting xPort shield; Arduino '328 to work?

I’ve been wrestling for months trying to get a Ladyada Ethernet shield with an xPort module to perform a simple HTTP GET from any CGI-supported Web site, such as, say, Twitter.

I’m using the latest libraries on v14 of the IDE on my Windows XP. Using a '168, I can at least dynamically assign an IP address and ping my xPort, and even occasionally, though not reliably, retrieve two strings of text from the NIST GOV site. However, the xPort/'168 combination will not work with Twitter or other CGI Web sites.

For what it’s worth, the xPort on the '168 works well with the Lantronix utility and appears to be quite healthy. However, Lantronix cannot “find” the same shield when mounted on a '328.

Since Lantronix cannot find the xPort with the '328, it is no surprise that the xPort will not reset(), connect(), or read() anything.

Has anyone had any luck getting these two boards (either the '168 or the '328) to work as advertised with the xPort shield?

As I must have DHCP, I cannot use the other Ethernet options, so I’m told. Also, the solution must be I2C Wire interface friendly.

I’m using standard sketches from this site and Ladyada.com, so I have not reposted the code here, but will, of course, upon request.

I look forward to your thoughts and suggestions.

I recently used an XPort Direct with a Duemilanove/328 and the shield from Adafruit. I used it to receive and process a file from web servers using a GET request.

There are only two reasons (I think) the XPort would not work: it's not getting power, or its reset line is activated. Check your wiring.

Another thing I found improved reliability was using the RTS and CTS lines. Note you have to set flow control mode 02 on the XPort to get these running, even if they were already soldered correctly.

Thanks for your input, macegr.

Great to hear that someone has gotten xPort to work. There are several of us who would be intrigued to know how you did it. Would you have some sample code that works with CGI that you could post?

Meanwhile, to answer your question: Power - Yep. It's there. I get flashing, blinking lights on the xPort whenever the Ethernet cable is connected on either the '168 or the '328. So while the '328 and the xPort aren't communicating somehow, the '328 is indeed providing it power.

Flow Control - Yep. It's 02 according to Lantronix when the xPort is mounted on the '168. Ofcourse, xPort and Lantronix don't talk to each other when it's mounted on the '328. Here's the Channel 1 config: *** Channel 1 Baudrate 9600, I/F Mode 4C, Flow 02 Port 00080 Connect Mode : D4 Send '+++' in Modem Mode enabled Show IP addr after 'RING' enabled Auto increment source port disabled Remote IP Adr: --- none ---, Port 00000 Disconn Mode : 00 Flush Mode : 77

*** Expert TCP Keepalive : disabled ARP cache timeout: 600s Monitor Mode @ bootup : enabled MTU Size: 1400 Alternate MAC: disabled Ethernet connection type: auto-negotiate

Finally, wiring is per the pin assignments as used in the Arduino Sketch:

define XPORT_RXPIN 2

define XPORT_TXPIN 3

define XPORT_RESETPIN 4

define XPORT_DTRPIN 5

define XPORT_CTSPIN 6

define XPORT_RTSPIN 7

Thanks again for any insights you might have.

The only important insight I had was that the AF_XPort library had a bug that prevented it from working with some webservers. Here's the bug description and the fix: http://www.ladyada.net/forums/viewtopic.php?f=31&t=10093

In that thread, you can find the fixed AF_XPort library which is also modified to work with the NewSoftSerial library.

Everything else I did was pretty much plain vanilla as shown in the various examples linked from the Adafruit store.

Flashing, blinking lights may not actually be good. For some reason, on 0013 on Vista, something would fail drastically whenever the XPort library was included, and the Arduino would continuously reset. On XP it was fine, maybe just a corrupt download. Fortunately 0014 came out and the problem went away. But yes, continously resetting the Arduino and XPort definitely prevents the Lantronix software from working.

Nice catch, macegr. I’ve made the corrections to AF_Xport.cpp, deleted the .o file and recompiled with an example program.

However, even with the change, notice that the xPort (on a '168) will download only a portion of the CNN.COM home page. It appears to bail out of the read loop and hang when it encounters two CR/LFs in a row.

Here’s the sketch:
#include <AFSoftSerial.h>
#include <AF_XPort.h>

//#define IPADDR “207.58.139.246” //www.ladyada.net
#define IPADDR “157.166.226.25” //cnn.com
#define PORT 80

char linebuffer[256]; // large buffer for storing data

#define XPORT_RXPIN 2
#define XPORT_TXPIN 3
#define XPORT_RESETPIN 4
#define XPORT_DTRPIN 5
#define XPORT_CTSPIN 6
#define XPORT_RTSPIN 7

AF_XPort xport = AF_XPort(XPORT_RXPIN, XPORT_TXPIN, XPORT_RESETPIN, XPORT_DTRPIN, XPORT_RTSPIN, XPORT_CTSPIN);

void setup() {

Serial.begin(9600);
xport.begin(9600);
xport.reset();
delay(1000);
Serial.println(“Finished Setup…”);

}

void loop()
{
byte ret;

Serial.println(“Connecting…”);
xport.connect(IPADDR, PORT);
xport.flush(300);
Serial.println(“Getting…”);
xport.println(“GET /index.html”);

ret=xport.readline_timeout(linebuffer, 255, 3000); // get first line
while(ret!=0){
Serial.print(linebuffer);
ret=xport.readline_timeout(linebuffer,255,4000);
}

Serial.print("Readline returned: ");Serial.println(ret,HEX);

}

Should you have a moment, I’d be interested to know if you’re able to download the entire site, somehow getting xPort to get past the double CR/LF.

Well, think about it: you're testing the readline command for zero. The readline command is looking for text followed by an LF. If the line only contains an LF, looking at the AF_XPort.cpp it would return zero in that function. Maybe use .available instead of the return value of the readline?

Thanks for the tip, macegr, and for acquainting me with the internals of AF_XPort.cpp. I really should have taken a look at it sooner.

Though using your suggestion improves things a bit, something is still disconnecting my xPort after only 15 to 21 characters or so.

To illustrate it, I wrote a short PHP script to respond to this URI: http://www.netify.com/xportGetTest.php4?user=foo&password=bar

In a browser, the PHP script will return: User is: foo; Password is: bar

However, with xPort, only a portion is returned before disconnecting, like this:

User is: foo; PassD <---------- Notice the ‘D’ for disconnect

Here is the code segment I used with the Arduino '168:
void loop()
{
byte ret;
xport.connect(IPADDR, PORT);
xport.flush(300);
xport.println(“GET http://www.netify.com/xportGetTest.php4?user=foo&password=bar”);

ret=xport.readline_timeout(linebuffer, 255, 3000); // get first line
Serial.print("Linebuffer: ");Serial.println(linebuffer);

}

Your suggestions have been magic so far. I hope I haven’t use up all my tokens.

Thanks again.

I'm not using the flush() before sending a GET request, I typically put it after the conversation is over. Flush() is kind of a kludge anyway, it's an excuse not to configure the XPort's built in flush routine correctly.

But here's my theory: you have Flush mode 77 enabled. I've been using 00 so far. What the settings do in the Flush bitmap is clear inbound or outbound buffers when certain events have been detected. You have all of those events enabled; the buffers will be cleared instantly when the webserver disconnects.

Imagine this: - Open connection - Send request - Receive data at high speed into XPort RS232 buffer - Begin slowly sending data out serial port - Webserver disconnects as soon as it sends the whole response - All data is cleared from the serial port buffer - You see "D" and don't get the rest of your data.

In short, you are still receiving data from the XPort buffer when the webserver disconnects. Your Flush setting will delete whatever's left in the buffer before you have a chance to receive it.

I think if you change your Flush bitmap to only clear the serial buffers when a connection is initiated, you will now receive all data sent from the web server (unless there is too much for the XPort buffer). Changing Flush to 0x33 might work.

So, um, did this work?

Macegr, your magic retains its power, and your diagnosis was spot on. Your solution worked immediately, so I apologize for not positing results sooner - I was just so pleased to break this log jam, I just dove into the next step.

But to recap - as you predicted, the request from xPort to the server was working fine, but the queue was being flushed before the read was complete - and indeed - the cure was to set the flush mode to 0x33. Nice work!

Now the challenge is to get the information out of the xPort Arduino to another Arduino over the I2C (Wire) interface.

Would you happen to know if there are any conflicts between the xPort and Wire?

Here's a brief description: The two Arduinos communicate just fine over Wire without the xPort code inserted. However, as soon as I make a call to xPort, say, xPort.begin(), the receiver Arduino (with no shield) stops receiving data from the xPort Arduino. The request() interrupt appears to work, but the data received is.. munged somehow.

I'll post more details if you could indulge me one more time. Should I start a new thread?

As it is, I already owe you a cup of coffee or something!

Cool!

I have not tried the XPort with Wire, but I will. And unfortunately still have no idea why your 328 behaves differently. Gotta be the XPort reset line, I'm still thinking....

Excellent, macegr, thanks.

For your cogitation, here are two interesting datapoints that may or may not help: - The xPort works fine with a '168 plugged into the board, but pull the '168 chip out an pop in a known-to-be-good '328, and the xPort stops responding to Ping or to the Lantronix utility. - The xPort shield by itself (i.e., powered by external GND and +5) responds both to Ping and to the Lantronix utility just fine, as you would expect, when not mounted on ANY Arduino board.

Can you put a Serial debug statement in the setup() function so you can tell if the Arduino is continuously resetting like mine was?

Bingo. The '328 is resetting. Here’s the sketch:


#include <AFSoftSerial.h>
#include <AF_XPort.h>

#define XPORT_RXPIN 2
#define XPORT_TXPIN 3
#define XPORT_RESETPIN 4
#define XPORT_DTRPIN 5
#define XPORT_CTSPIN 6
#define XPORT_RTSPIN 7

AF_XPort xport = AF_XPort(XPORT_RXPIN, XPORT_TXPIN, XPORT_RESETPIN, XPORT_DTRPIN, XPORT_RTSPIN, XPORT_CTSPIN);

void setup()  {

  Serial.begin(9600);
  Serial.println("Starting xPort");  
  xport.begin(9600);
  xport.reset();

  Serial.println("Finished Setup");

}

void loop()
{
  byte ret;

  Serial.print(".");
  delay(100);
}

With the xport.reset() commented out, the output on the Serial monitor is as you would expect.

However, with the xport.reset() in place (as shown above), I get:


Starting xPort

ǼStarting xPort

ǼStarting xPort

Notice the garbage characters along with the infinite reset.

Incidentally, the flow is set to 02, for what it’s worth.

Thoughts?

Oho.

Okay, I noticed this when trying to run an XPort sketch on my Vista (er, Windows 7) laptop. Whenever I activated the XPort, it would send my Arduino into a reset loop (this was also a '328). I assumed my copy of the Arduino IDE had somehow gotten corrupted, and simply copied the arduino-0014 folder from another computer onto a USB flash drive. Whenever I ran the IDE from the flash drive, it worked fine.

I also noticed that the message about binary size gave a smaller number on the faulty sketches.

Perhaps my download of the IDE was indeed corrupted (it was downloaded on a somewhat unreliable wireless connection). You're running on XP so the OS choice should not be a factor, my other working Arduino installs were on XP.

How about we both try re-downloading the Arduino IDE...wait. I don't have the '328 Arduino anymore. OK, so try to re-download the IDE and libraries and see if that helps.

Ok, macegr. Progress.

Thanks to your tip, I went to download the latest versions of libraries when I found your remarks regarding the AF_XPort.h mod you made, and how you made further changes to make it compliant with NewSoftSerial.

So I downloaded the NewSoftSerial and AFXPort libs and recompiled the sketch that never has worked with a '328 and guess what?

No more impromptu '328 resets. So now my xPort shield on any of my Arduino boards ('168 or '328) will communicate with the Lantronix utility as well as successfully perform an HTTP Get. All thanks to you!

But, sadly, the Wire issue remains. That is, an otherwise working Wire script will work fine with all xPort() commands commented out. But when some xPort() commands are "commented in," the sketch hangs on a reference, such as, say, xPort.connect(). Interestingly, the sketch does not hang, for some reason, on an xPort.begin() or xPort.reset() command.

Darn it. I was really hoping that solving the '328 problem would also solve the Wire problem. Double darn.

I'll post a sketch if you want to try to reproduce the hang.

I'm wondering if this is part of the problem: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1237732456

Does the Wire + XPort work on the 168 but not the 328?

Hm. Intriguing. I'll spend some time looking at that AVR issue. Thanks for the reference.

As for your question: No. The WIRE conflict appears to be the same whether on a '168 or a '328. Either, while in slave mode, will "hang" during the requestEvent() handler if it makes certain xPort references.

Here's what I've observed: After the Master issues a requestEvent() to the designated Slave, the Master, whether a '168 or a '328 (and with no xPort), appears to detect that the slave is available, causing the Master to begin reading data, when, in fact, the Master appears to be reading nothing. For example, let's say the slave wants to send the characters '1234' in response to the read request. If I have a variable cFoo on the Master that contains the characters '5678' prior to the Wire.read(), and I store the characters read into successive locations within cFoo, it will contain the characters '5678' after the read as well. It can be a tricky error to detect.

Here's what seems to be happening. 1) The Master requests 4 bytes of data and finds the Slave immediately available. 2) The Master then uses Wire.receive() to read each of the four characters. However, the Master appears to ignore the character read, and iterates to the next request until the Slave is no longer available - that is, until four reads have been executed.

I believe the Master is not getting any data because the slave is hung up somehow. Yet something is telling the Master that data is available at the beginning of the read cycle, and something is turning off Wire available when the four read attempts are finished. Not really sure about that part yet.

To test the problem, one might try to use an xPort in slave mode to read the index.html of any Web site (say www.arduino.cc) and WIRE the contents over to a Master through a series of read requests.

I'd be interested to know if anyone can get that to work.

Today was a good day. Some progress. The WIRE and xPort libraries are working together beautifully now, though I'm still not sure exactly what was wrong to begin with.

Here's what I changed:

1) Minimal Interrupt Handing: I removed all other code from the receiverequest() handler on the xPort Arduino not directly related to the exchange of data over WIRE. My current hypothesis is that the extra code in this handler was tripping over itself by the time the next receive request arrived. Not convinced that's right, but it's the best I've go so far.

2) Power? - Originally, I was powering the master Arduino from the USB port, and the xPort Arduino from the master via wires connecting the Vin and GND terminals of the two boards. When I accidentally plugged both boards into the USB ports, inexplicably, the system started working. No more hangs in the WIRE routines, and no more garbage. This is strange, but it's repeatable and consistent. I'd love to hear any theories to explain this.

Thanks, macegr, for your attention to this. I'm not out of the woods yet on this project, but this is a huge step forward.