How do I get a class (hardware serial) to use an external buffer?

Projects Programming
1

I am fiddling around to see if I can modify HardwareSerial. One of the things I would like to do is get it to place incoming data directly into an array outside the class so that some of the time it writes to one array and subsequently to a different one - both to save the trouble of using Serial.read() to transfer the data and so the array can be a Union with another definition of the same data.

The following code in HardwareSerial.h defines the buffer (line 87 in my copy)

unsigned char _rx_buffer[SERIAL_BUFFER_SIZE];

I'm not sufficiently familiar with C/C++ to know how I could tell the class definition to use another location.

If anyone has a simple solution I would appreciate it. I don't think I would be interested in anything complicated as I would probably prefer to create some new bare-bones code to read the USART rather than make complicated changes.

Also I am not looking for a solution that is generally applicable - this will only be used for one project.

...R

2

both to save the trouble of using Serial.read() to transfer the data and so the array can be a Union with another definition of the same data.

Keep in mind that the hardware serial buffer is a circular buffer. Read and writing advance head and tail pointers, Reading does NOT move the data within the array.

I can't see how you will accomplish the same thing using an external array, nor do I see how you can use a union effectively with a circular buffer.

3

Short of rewriting HardwareSerial I don't think you can. Better to tell us what your trying and see if there is a differant way to do it.

Mark

4

I had omitted to say I will just be sending a buffer full on each occasion so the circularity is irrelevant. But I had forgotten that I would have to fiddle with the head and tail pointers as well as eveything else.

I think the easiest thing is just to write my own specialized version of HardwareSerial. There will be a lot of the USART specific I can re-use.

If I get it to work I will post it here.

I want the PC to be able to send data which is picked up and saved by the USART directly into a useable location while the Arduino is driving some stepper motors. Then when the movement is finished there will be new data immediately ready for the next move. If the data from the PC can go into two alternating arrays the new data will always be separate from the current data.

Thanks for the comments.

...R

5

I want the PC to be able to send data which is picked up and saved by the USART directly into a useable location while the Arduino is driving some stepper motors.

It does this now. That the data is in a non-linear buffer means that it needs to be copied to a linear buffer, if the data needs to be in a linear format. I don't see a way around this.

Then when the movement is finished there will be new data immediately ready for the next move.

As opposed to the few (thousand) nanoseconds later, after Serial.read() copies it?

6

I will just be sending a buffer full on each occasion so the circularity is irrelevant...

I want the PC to be able to send data which is picked up and saved by the USART directly into a useable location.

If the data is the size of the internal buffer, why not double its size and access it directly; Its not that much different to having two buffers.

Here is some code I previously wrote to access the internal buffer: New peek function for HardwareSerial - #8 by pYro_65 - Libraries - Arduino Forum

7

pYro_65:
Here is some code I previously wrote to access the internal buffer

Thanks - I had seen that before but my mind was running in a different groove. I have taken a copy and I will look at it.

Old age is catching up with my grey cells. I discovered this morning I had done some work on this idea not very long ago (perhaps for a different reason) and figured out how I can get access to the ISR(USART_RX_vect). I suspect if I just write my own code to replace the code in HardwareSerial's Serial._rx_complete_irq(); I will achieve my requirement with very little effort.

I have already figured out how to take the relevant HardwareSerial files and modify them so I can have my own copy in my own code without disturbing the principal Library copy.

...R

8

I think I now have a good solution for my needs that doesn't interfere with the normal functioning of HardwareSerial.

However while I only need to make a small change to the code getting at the code is a real PITA because of the way the HardwareSerial code is spread over 7 files. As well as the fact that the #include HardwareSerial is buried in Arduino.h. It would not be a great hardship for Arduino users to have to put #include <HardwareSerial.h> in their sketches just as they have to put #include <Servo.h>

Somewhere in among the Arduino files I presume there must be a #include "HardwareSerial0.cpp" - or something else that is equivalent, but I cannot find it (or perhaps I don't know what to look for). All the other .cpp files have a corresponding .h file, but not this one.

Does anyone know where the reference is, or how that file gets included? If I can figure that out I expect I can greatly simplify my solution.

...R

9

On my computer, HardwareSerial.cpp is in C:\Users<my user id>\Documents\arduino-1.0.5\hardware\arduino\cores\arduino.

10

PaulS:
On my computer, HardwareSerial.cpp is in C:\Users<my user id>\Documents\arduino-1.0.5\hardware\arduino\cores\arduino.

Thanks Paul, but that is not what I am asking. I know where the files are stored.

What I don't know is how the file HardwareSerial0.cpp (not HardwareSerial.cpp) gets included in the compiled code as I can't see any reference to it in any of the other HardwareSerial files.

...R

11

What I don't know is how the file HardwareSerial0.cpp (not HardwareSerial.cpp) gets included in the compiled code as I can't see any reference to it in any of the other HardwareSerial files.

I don't understand why you think that there should be a HardwareSerial0 file, with any extension, anywhere.

12

PaulS:
I don't understand why you think that there should be a HardwareSerial0 file, with any extension, anywhere.

Because it exists and it is the file that connects to the UART0_UDRE_vect interrupt vector.

Following a bit more reading the problem may be my ignorance of the C/C++ compiling process.

I have always assumed that the code #include "myFile.h" is the thing that also brings into the project in the associated myFile.cpp. In other words, without that #include neither file would be part of the project.

But I'm beginning to think I am wrong about this and the .cpp files get compiled regardless of the existence (or not) of a similarly named .h file.

I can get what I want to happen by making copies of the relevant HardwareSerial files and renaming them. I was just hoping there might be a simpler way. If I don't rename them I get duplicate definitions even after the #include <HardwareSerial.h> is commented out in Arduino.h. If I could prevent the Library copy of HardwareSerial0.cpp from being compiled I could use my own copy without needing to rename it.

Maybe all this seems as clear as mud, but I can't think of a better way to explain myself without writing several pages.

...R

13

Following a bit more reading the problem may be my ignorance of the C/C++ compiling process.

It is more likely that it is the compiling process AS PERFORMED BY THE IDE that is the problem.

In other words, without that #include neither file would be part of the project.

That is correct.

If I don't rename them I get duplicate definitions even after the #include <HardwareSerial.h> is commented out in Arduino.h.

Is it possible that HardwareSerial.h is included in other places?

If I could prevent the Library copy of HardwareSerial0.cpp from being compiled I could use my own copy without needing to rename it.

If you are compiling in verbose mode, you should see exactly where this file is coming from. And isn't that the problem you are trying to solve?

14

Robin2:

PaulS:
I don't understand why you think that there should be a HardwareSerial0 file, with any extension, anywhere.

Because it exists and it is the file that connects to the UART0_UDRE_vect interrupt vector.

It's simple enough to find out, just search your Arduino installation. On mine, there is no HardwareSerial0 of any sort. It's all handled by HardwareSerial.cpp and HardwareSerial.h. Possibly this depends on which version of IDE you have.

Following a bit more reading the problem may be my ignorance of the C/C++ compiling process.

I have always assumed that the code #include "myFile.h" is the thing that also brings into the project in the associated myFile.cpp. In other words, without that #include neither file would be part of the project.

But I'm beginning to think I am wrong about this and the .cpp files get compiled regardless of the existence (or not) of a similarly named .h file.

Indeed, in general there is no relation between header files and source files, as far as C/C++ is concerned, naming conventions are purely for the programmer. C/C++ compilers perform separate compilation of object files, there is no concept of modules.

However the Arduino IDE does a little extra stuff to include library code if you include the header file, but that is a quite non-standard feature.

I can get what I want to happen by making copies of the relevant HardwareSerial files and renaming them. I was just hoping there might be a simpler way. If I don't rename them I get duplicate definitions even after the #include <HardwareSerial.h> is commented out in Arduino.h. If I could prevent the Library copy of HardwareSerial0.cpp from being compiled I could use my own copy without needing to rename it.

I don't think you can get around renaming, unless you go in and change how the Arduino libs are built.

One way to provide your own defintions for HardwareSerial is to put the following line before including Arduino.h :

#define  HardwareSerial_h // trick to disable the standard HWserial

The Marlin firmware uses this to override the standard hardware serial implementation.

15

bobcousins:
One way to provide your own defintions for HardwareSerial is to put the following line before including Arduino.h :

#define  HardwareSerial_h // trick to disable the standard HWserial

I like that trick.

As is common I think I have been wasting people's time by asking the wrong question too early - blame it on my skimpy knowledge of C/C++.

I have now discovered that I only need to modify one file HardwareSerial_private.h which contains the code that executes when a character is received by the USART. The standard code writes the character to the serial input buffer. With a few lines of extra code I can get it also to write the character to an array that I have defined.

The modified code is as follows (shown in quotes so I can highlight the new pieces).

extern unsigned char myRxBuf[65];
extern byte myBufIndx;

void HardwareSerial::_rx_complete_irq(void)
{
if (bit_is_clear(*_ucsra, UPE0)) {
// No Parity error, read byte and store it in the buffer if there is
// room
unsigned char c = *_udr;
uint8_t i = (unsigned int)(_rx_buffer_head + 1) % SERIAL_BUFFER_SIZE;

// if we should be storing the received character into the location
// just before the tail (meaning that the head would advance to the
// current location of the tail), we're about to overflow the buffer
// and so we don't write the character or advance the head.
if (i != _rx_buffer_tail) {
_rx_buffer[_rx_buffer_head] = c;
_rx_buffer_head = i;
}
// this works independently of the size of the serial buffer
myRxBuf[myBufIndx] = c;
myBufIndx ++;
if (myBufIndx > 5) {
myBufIndx = 0;
}
} else {
// Parity error, read byte but discard it
*_udr;
};
}

I have copied HardwareSerial_private.h into my project directory and #included it in my .ino file. No other changes to the HardwareSerial system.

It's interesting to hear that some people have no file called HardwareSerial0.cpp. I am using Arduino 1.5.6-r2

...R

16

Robin2:
It's interesting to hear that some people have no file called HardwareSerial0.cpp. I am using Arduino 1.5.6-r2

Yeah, seems to have been introduced in 1.5.6.

1.5.x is still under some development, so anything that works in other versions may not work in 1.5.6, or indeed future versions :slight_smile:

17

It's interesting to hear that some people have no file called HardwareSerial0.cpp. I am using Arduino 1.5.6-r2

For which Arduino? The 1.5+ version is for the SAM hardware Arduino's. That file doesn't exist in the 1.0 branch for the AVR hardware Arduinos.

18

PaulS:

It's interesting to hear that some people have no file called HardwareSerial0.cpp. I am using Arduino 1.5.6-r2

For which Arduino? The 1.5+ version is for the SAM hardware Arduino's. That file doesn't exist in the 1.0 branch for the AVR hardware Arduinos.

I am using it for all my Arduinos - Uno, Mega, Leonardo. I switched to it because it lists how much memory the variables use. The earlier versions don't.

Since my earlier post this project has almost got me tearing my hair out. Some of the variants I have tried WORK when I upload the code. DON'T work when I upload the unchanged code a second time. DO work when I upload it a 3rd time etc... If someone else here told me this was happening I would not have believed him.

I have reverted to an earlier version in which I have copied all the relevant HardwareSerial files into my project directory and renamed them by prepending R2 to the file names and updating the relevant #includes. The only changes to the code are as in my earlier post.

...R

19

I am using it for all my Arduinos - Uno, Mega, Leonardo. I switched to it because it lists how much memory the variables use. The earlier versions don't.

There are major differences in the core code - some good, some not so good. I don't use the 1.5+ version unless needed (for my Due). For instance, SoftwareSerial is not supported, even for the Uno, where it is known to work.

20

Update ...

I got my code to do what I want - to write incoming bytes directly into an array that is accessible to my code and which is part of a Union.

There is a small performance improvement compared with having to use a succession of Serial.read()s to transfer the data from the Serial input buffer. But, for what I want to do I don't think the difference is big enough to justify the trouble.

...R