PeterH is correct. I said the same in reply#10. The SPI will handily outrun the serial port.
For example, you can't load a dump truck with bricks using a wheel barrow, then complain that the dump truck isn't delivering bricks fast enough.
For example, you can't load a dump truck with bricks using a wheel barrow, then complain that the dump truck isn't delivering bricks fast enough.
You can if you only put one brick in the truck at a time, as OP is (or was) doing.
My request is based on the lots of tutorial and literature out there where people write simple code as this to show, how client.print will work. It seems to me, I got one level higher in integration...
Thanks all for your replay. I will try some code with the buffer.
I see many examples that use client.println() calls in a non-optimized fashion, as I described earlier. If you are sending one character at a time that is input by skinware on a keyboard, both serial and ethernet can handily outrun that. If the serial port is fed by an automated program that can send constantly at the 115200 speed, I recommend optimizing all interface code as I described above. It all counts.
markuszelg:
Okay, now we are deeply in the processors design.I understood the protocol speed is not the limit in my use case (stream von NanoSerial->Serial3 on Mega->SPI MegaOut->Ethernet). Limiting is the processors handling of each buffer. And this is in same speed. With the fact of handling spi-transfer for each char, the code requests more processor time for serial out (over SPI) than the serial3.read on input site. Hence, the buffer goes full.
No, no, no. I don't understand why you keep bringing SPI into the problem. It is nothing to do with SPI. It is nothing to do with the CPU speed, or anything requiring knowledge of the processor's design. This is purely about your code reading and writing to the hardware serial ports.
You have code that reads from one hardware serial port, Serial3. For each character it reads, it writes multiple characters to a different hardware serial port, Serial.
This means that it will generate more output than input. Assuming (and we have to assume, since you haven't POSTED YOUR CODE) that the two hardware serial ports are configured similarly, the output buffer will soon fill up. The implementation of the Arduino hardware serial library is that when the output buffer for a serial port becomes full attempts to write further output block (stop) until there is space for the output to be buffered. This means that for each character received, the Arduino will stop and wait for three characters to be transmitted before it tries to read the next character. In other words, it's now only capable of processing the incoming serial stream at a third of the speed it's arriving at. As soon as the receive buffer fills up, the excess input will be discarded.
You have code that reads from one hardware serial port, Serial3. For each character it reads, it writes multiple characters to a different hardware serial port, Serial.
Only for debugging purposes. The real intent of the program is to read serial data from Serial3 and send it to a client. That uses SPI. So, the question of the speed of SPI vs. the speed of Serial is a quite legitimate one.
Of course, as has been seen, the issue of SPI speed is not the limiting factor. It is the overhead of packeting one character at a time that is/was the problem.
PaulS:
Only for debugging purposes. The real intent of the program is to read serial data from Serial3 and send it to a client. That uses SPI. So, the question of the speed of SPI vs. the speed of Serial is a quite legitimate one.Of course, as has been seen, the issue of SPI speed is not the limiting factor. It is the overhead of packeting one character at a time that is/was the problem.
Granted that it's only for debugging purposes, but it is having the effect of throttling the whole sketch to the speed of the output stream, which will result in the input stream overflowing.
I don't see any reason to suppose that the overhead of packeting one character at a time over SPI is causing any problem. The Ethernet interface, and the SPI bus used to access it, and the processor which is performing the work, are all capable of dealing with far greater throughput than the serial stream.
To answer your direct question, I have a summary of protocol speeds here:
See this also:
There I calculate that the maximum rate of clocking out SPI bytes is 888,888 bytes per second.
However as others are trying to tell you, this is irrelevant.
First, the Ethernet card probably doesn't operate at that speed. Even if it did, it would not be able to send Ethernet data that fast, probably.
Then we have the more major issue that your debugging displays are almost certainly the issue.
Okay, now we are deeply in the processors design.
No, we are not.
We are wasting time because you are posting tiny snippets of code, jumping to a conclusion about what they mean, and refusing to post all your code.
here we go: latest and greatest with 2 issues:
- Serial3 stream to client
- Reset to setup() after client.stop(); as gifen with serial.print of the initial IP address only in setup();
#include <SPI.h>
#include <Ethernet.h>
struct ETH_REQUEST_OBJECT {
unsigned int device;
unsigned int command;
char value[11];
};
// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {0x90,0xA2,0xDA,0x00,0xE9,0x86};
IPAddress ip (192,168,178, 7);
byte gateway[]= {192,168,178,1};
byte subnet[] = {255,255,255,0};
byte dSS[] = {192,168,178,2};
EthernetServer server(80);
EthernetClient client;
unsigned long start;
ETH_REQUEST_OBJECT eth;
const char SERIAL_START ='[';
const char SERIAL_END =']';
const char SERIAL_SEPARATOR =',';
const String HTTPHEADER= "HTTP/1.1 200 OK\nContent-type: text/xml\n\n<?xml version=\"1.0\" standalone=\"yes\"?>\n<FMS>";
const String HTTPFOOTER = "</FMS>";
void setup() {
Serial.begin(115200);
Ethernet.begin(mac, ip);
server.begin();
Serial.print("System ready:\nIP: ");
Serial.println(Ethernet.localIP());
ETHrestOBJECT();
}
void loop() {
delay(5);
// check GUI request
ETH();
// check DOOR event
DOOR();
// start automatisation
SYS();
}
// ========================
// L E V E L 1 M A I N
// ========================
void ETH() {
client = server.available();
if (client) {
ETHreadREQUEST();
if (eth.device==250) {
DOORwriteCOM();
DOORread2ETH();
}
ETHrestOBJECT();
client.stop();
}
}
void DOOR() {
//check for any event such as door access by ID [010000BA24A2A2]
while (Serial3.available()) {
c=Serial3.read();
if (c==SERIAL_START) continue;
if (c==SERIAL_END) continue;
if (c==SERIAL_SEPARATOR) continue;
Serial.print(c);
}
}
void SYS() {}
// ===============================
// L E V E L 2 E T H E R N E T
// ===============================
void ETHrestOBJECT() {
eth.device=eth.command=0;
eth.value[0]='\0';
}
void ETHreadREQUEST() {
byte i=0,tCount=0;
char tBuf[41];
char *pch;
while (client.connected()) {
if (client.available()) {
char c = client.read();
if(tCount < 41) {
tBuf[tCount] = c;
tCount++;
tBuf[tCount] = 0;
}
else {
while(client.available()) { client.read();}
pch=strstr(tBuf,"dev=");
if ( pch!=NULL) eth.device=atoi(pch+4);
else eth.device=0;
pch=strstr(pch,"com=");
if ( pch!=NULL) eth.command=atoi(pch+4);
else eth.command=0;
pch=strstr(pch,"val=");
if ( pch!=NULL) {
pch+=4; //scip 'val='
while ( pch[i]>='0' && pch[i]<='z' && i<10) { // end with ' '
// accepted values: ASCII mit ::;>=>?@ und [\]^_´
eth.value[i]=pch[i];
i++;
}
eth.value[i]=0;
}
else eth.value[0]=0;
break;
} //got buffer filled
} // available
} // connected
} // function
void ETHreportERROR(int code, String text) {
// ERROR VALUES
// 01.. This runtime
// 02.. ETH
// 03.. DOOR
// 04.. SYS
// 1... RFID System
// 2... RS485 Member
ETHrestOBJECT();
client.print("<ERROR><code>");
client.print(code);
client.print("</code><message>");
client.print(text);
client.print("</message><device>");
client.print(eth.device);
client.print("</device><command>");
client.print(eth.command);
client.print("</command><value>");
client.print(eth.value);
client.print("</value></ERROR>");
client.println(HTTPFOOTER);
client.println();
}
// ========================
// L E V E L 2 D O O R
// ========================
void DOORread2ETH() {
char c;
byte i;
char tBuf[4];
unsigned int counter=0;
client.print(HTTPHEADER);
while ( Serial3.read()!=SERIAL_START && counter<0xFF ) { delayMicroseconds(50); counter++;}
if (counter==0xFF) { ETHreportERROR(301,"RFID System down"); return; }
c=Serial3.read();
if (c=='E') {
for (i=0; i<4; i++) {
tBuf[i]=Serial3.read();
}
ETHreportERROR(atoi(tBuf)+1000, "RFID System Error");
return;
}
else {
client.print("<DOOR><item>");
client.print(c);
while (Serial3.available()) {
c=Serial3.read();
if (c==SERIAL_END) break;
if (c==SERIAL_SEPARATOR) {client.print("</item><item>");}
else {client.print(c); }
//delay(20); // wait for NANO sending data to buffer, 1byte is 80microseconds
//Serial.println(Serial3.available());
}
client.print("</item></DOOR>");
client.println(HTTPFOOTER);
}
return;
}
const String HTTPHEADER= "HTTP/1.1 200 OK\nContent-type: text/xml\n\n<?xml version=\"1.0\" standalone=\"yes\"?>\n<FMS>";
const String HTTPFOOTER = "</FMS>";
You just insist on ignoring the advice to get rid of Strings. Why?
What are your issues with those two items?
I didn't check all the code, but this will cause problems:
char tBuf[41];
// if the current tCount == 40
if(tCount < 41) {
// set the last member of the array to c
tBuf[tCount] = c;
// this increments tCount to 41
tCount++;
// this overflows the array
tBuf[tCount] = 0;
PaulS:
You just insist on ignoring the advice to get rid of Strings. Why?
Your answer belongs to your answer #2? So, I understood doing the array for the incoming data on Serial3 - what I didnt had implemented yesterday. What is not common with the XML-Header data? One may put the string into flash memory with F().
SurferTim:
What are your issues with those two items?
I dont get your question. This part works pretty stable. The function gets only the first 41 chars from the http header. But this section absolutly not belongs to my streaming-issue nor to the reboot phenomenon.
markuszelg:
This part works pretty stable. The function gets only the first 41 chars from the http header. But this section absolutly not belongs to my streaming-issue nor to the reboot phenomenon.
The function gets the first 41 characters, but overflows the array with the zero terminator (42nd character). You are probably putting that zero terminator in this variable.
char *pch;
What do you expect pch to do?
@SurferTim: Maybe I'm totaly wrong here. My loop will write at the end of the while tBuff[41]=0, what I understoot is the terminator like '\0' in char. Not?
The char *pch? For strstr C-function. Works stable, no? Is my point to the first occurence of "dev=" in client HTTP GET request. Works stable as stones.
tBuf has 41 elements in the array, which are tBuf[0] to tBuf[40].
tBuf[41] is outside that array. It has a habit of putting that overflow into the next variable you declared, which in your case would be a pointer. If you put 0 in that pointer, that will now point to memory location zero. Assigning anything to that memory location will cause problems. In the "good old days" of the IBM PC, that would throw the "divide by zero" error. In your case, it might restart the program.
SurferTim:
tBuf has 41 elements in the array, which are tBuf[0] to tBuf[40].
Very unprofessional on my site. I forgot having char array which are different declared than the int array 
I was thinking of having a new feature: restart. This would be an other topic: How do a program a restart? Jumping to 0? Hmm... maybe not the stable way.
Back to the streaming-issue. Any ideas?
markuszelg:
Back to the streaming-issue. Any ideas?
You seem to have found and fixed numerous bugs which might be related. Can you recap what the symptoms of the remaining problem are, and post the latest version of the code with all the other bugs corrected?
Right, I should fix the code first. Next step is update of buffering serial3 input stream for sending array string to client. I will update soon.
To back up PaulS:
Please note that, at present, the String library has bugs as discussed here and here.
In particular, the dynamic memory allocation used by the String class may fail and cause random crashes.
I recommend reworking your code to manage without String. Use C-style strings instead (strcpy, strcat, strcmp, etc.).
Alternatively, install the fix described here:
http://arduino.cc/forum/index.php/topic,145765