Does anyone know why serialEvent1 is does not function on ATMega32U4 devices

Projects Programming
1

Hi,

Does anyone know why serialEvent1 is does not function on ATMega32U4 devices, like the Leonardo, or Micro etc.

i.e I've tested it on Pro Micro board (Atmega32U4) and it doesnt seem to call SerialEvent1 when there are incoming serial characters.

There is a mention in the Serial section that says SerialEvent doesn't work on the Leonard or Micro, but doesnt mention whether SerialEvent1 should work or not, and since Serial1 is the actual hardware serial on this device (as Serial is created as part of the USB PC connection), it seems odd that it doesnt work.

I took a look in HardwareSerial.cpp and this is the code for Serial1 ISR

#if defined(USART1_RX_vect)
  void serialEvent1() __attribute__((weak));
  void serialEvent1() {}
  #define serialEvent1_implemented
  ISR(USART1_RX_vect)
  {
    if (bit_is_clear(UCSR1A, UPE1)) {
      unsigned char c = UDR1;
      store_char(c, &rx_buffer1);
    } else {
      unsigned char c = UDR1;
    };
  }
#endif

so I checked whether USART1_RX_vect is defined (in my code) and it is defined,

But I can't see how serialEvent1 is actually connected to the ISR.

I'd have thought that this was accomplished by calling the serialEvent1() from inside the ISR but it doesnt appear to work this way.

Can someone give me a clue about how serialEvent works, and why it may not work in SerialEvent1 on a 32U4

Thanks :wink:

2

The serial event handling is not related to the interrupt handling at all. It is just a function serialEventRun() called from main() that calls Serial.available() for each hardware port, and calls the serialEventx() functions if the result was non-zero. In my opinion it is a fundamentally daft design and should never have been implemented. It does nothing that you can't already do in your sketch, doesn't make serial event handling any easier and doesn't fit in with any other sensible event handling scheme. I suspect the whole concept was motivated by a misguided desire to make an Arduino sketch look like a Processing application, which is neither sensible nor possible.

3

There is probably nothing to stop you from modifying HardwareSerial.cpp and fixing it to do what you want, yourself.

4

I agree with @PeterH.

There seems little point in having an input buffer for Serial data if your code gets called everytime a character arrives. The whole point of a buffer is that it allows you to get the data when it is convenient for you, rather than convenient for the computer.

SerialEvent may not be implemented as an interrupt but its impact in your code is pretty much the same. However you have the option of switching interrupts off, but not serialEvent().

...R

5

PeterH:
The serial event handling is not related to the interrupt handling at all. It is just a function serialEventRun() called from main() that calls Serial.available() for each hardware port, and calls the serialEventx() functions if the result was non-zero.

I've just been looking at the code in HardwareSerial.cpp.

It seems that it is exactly the same as having this in loop()

void loop() {

   if (Serial.available() > 0) {
        doMySerialEvent();
   }
}

void doMySerialEvent() {
   // stuff goes here
}

It seems to be even more pointless than I thought. How could someone have thought that hiding simple stuff like this would be useful?

The other thing is that it's name is misleading. Presumably if your code in serialEvent() doesn't remove data from the buffer then serialEvent() will be called in every iteration of loop() regardless of whether any event happened (i.e. more characters are received).

...R

6

It seems to be even more pointless than I thought. How could someone have thought that hiding simple stuff like this would be useful?

+1

Mark

7

Thanks for the insightful replies.

I was under the gross misapprehension that the serial event handler was called in direct response to a character arriving at the serial port, but I now see that it's not that at all.

No wonder the code didn't make any sense to me.

It looks like I'm going to need to write my own replacement for the Serial1 ISR, so that the code can get notified as soon as a character arrives.

Btw.
I didn't mention what I am doing..

I've written some web server code that works with a Serial Wifi module.
I,e it parses the incoming GET request and sends back the appropriate response data.

However the issue I'm having is if I send back a page, which contains references to other pages e.g. Style sheets which are also on the Arduino. That most browsers seem to parse the page on the fly and submit requests for the other pages and images etc before the initial page has finished being transferred.

As the GET requests from browsers are around 400 characters long, the subsequent request overflow the incoming serial buffer, with the result that the subsequent requests get lost, because my send function is still pushing data to the wifi module while the GET requests are coming in.

I don't think this can be solved correctly by increasing the serial buffer size, and anyway that would require people to modify their Arduino installation, and I was hoping to write generic code ( a version is already on GitHub and is being used by other people)

Adding code to the send function, to sporadically check for incoming data, would probably resolve the problem, but it would be messy, as I'd need to put a call to the check function in a few places.

I was hoping to split out the data input and parsing from the output, using something like SerialEvent.

That is, have some code that just operates off the serial ISR in and parses the incoming data and determines what pages ( or other data ) needs to be returned to the wifi module.
Then put these requests in an queue (array ), the have the page generation and output function operate from the main loop, I.e. poll the queue and send pages one by one until the queue was empty.

In this case the queue only needs to store what page needs to be returned e.e index.html. Along with any URL parameters e.g. DigitalWrite.html?pin=13&value=LOW

In fact I'd not use quite as verbose a name or parameters as this, but it gives an example.

Anyway, the ram taken by the queue wouldn't need to be too big as long as the pages didn't have lots and lots of links to other pages on the Arduino,

Anyway, I guess my next task is to write a replacement serial ISR and see if it gets called instead of the existing one.

Note I'm using serial1 on a pro micro, so this won't interfere with normal operations.

8

rogerClark:
I was under the gross misapprehension that the serial event handler was called in direct response to a character arriving at the serial port,

So was I. It's the lousy name they chose for the function. Just as bad as Serial.flush().

Recently I was reading the Atmega328 datasheet about the USART because I wondered if there was an on-chip serial input buffer. There isn't - which is a real shame. I had hoped it could store (say) 16 bytes of incoming data without using up any CPU time.

But from what I read the USART raises an interrupt when a character is received so it should be easy enough to use that for your own code. I guess the trick is to prevent the standard Arduino code from responding.

...R

9

@Robin2

I presume that if I write a function that uses the ISR macro with the same vector as the existing Serial ISR that it will just use my new function instead of the built in one.

I'd actually like the keep the existing functionality, but just have my own realSerialEvent() function called, but I'm not keen on hacking the IDE files, as I'd need to do it on all my machines etc, so I'm going to try sticking the code from Hardware.cpp in a test program and see if I can override it

e.g use the code from Hardware.cpp (actually I'll change Serial1, but you get the idea :wink:

#if defined(USART_RX_vect)
 ISR(USART_RX_vect)
#elif defined(USART0_RX_vect)
  ISR(USART0_RX_vect)
#elif defined(USART_RXC_vect)
  ISR(USART_RXC_vect) // ATmega8
#endif
  {
  #if defined(UDR0)
    if (bit_is_clear(UCSR0A, UPE0)) {
      unsigned char c = UDR0;
      store_char(c, &rx_buffer);
    } else {
      unsigned char c = UDR0;
    };
  #elif defined(UDR)
    if (bit_is_clear(UCSRA, PE)) {
      unsigned char c = UDR;
      store_char(c, &rx_buffer);
    } else {
      unsigned char c = UDR;
    };
  #else
    #error UDR not defined
  #endif
  }

EDIT.

Unfortunately all this stuff is marked as private in Hardware.h

So I'll need to code my own function that handles the ring buffering if I need it :frowning:

Where do I file a bug / issue with the Arduino core, as serialEvent seems like a real botch.

10

I presume that if I write a function that uses the ISR macro with the same vector as the existing Serial ISR that it will just use my new function instead of the built in one.

I would not bet on it.

I think your server code is get to complex for the AVR. Take a look at the Yun and consider putting the server on the linux processor.

Mark

11

rogerClark:
I presume that if I write a function that uses the ISR macro with the same vector as the existing Serial ISR that it will just use my new function instead of the built in one.

I don't have time to look up anything now but I wonder if it is Serial.begin() that implements/initializes the interrupt from the USART. If so, just don't call it.

Of course you could be very clever and have your ISR call the standard code after you have got what you want out of it.

...R

12

Hi Robin2

I took a look in HardwareSerial.cpp and it looks like the ISR is just declared as a function straight away, i.e it happens even if Serial.begin is called.

begin() just seems to setup the data rate e.g.

void HardwareSerial::begin(unsigned long baud)
{
  uint16_t baud_setting;
  bool use_u2x = true;

#if F_CPU == 16000000UL
  // hardcoded exception for compatibility with the bootloader shipped
  // with the Duemilanove and previous boards and the firmware on the 8U2
  // on the Uno and Mega 2560.
  if (baud == 57600) {
    use_u2x = false;
  }
#endif

try_again:
  
  if (use_u2x) {
    *_ucsra = 1 << _u2x;
    baud_setting = (F_CPU / 4 / baud - 1) / 2;
  } else {
    *_ucsra = 0;
    baud_setting = (F_CPU / 8 / baud - 1) / 2;
  }
  
  if ((baud_setting > 4095) && use_u2x)
  {
    use_u2x = false;
    goto try_again;
  }

  // assign the baud_setting, a.k.a. ubbr (USART Baud Rate Register)
  *_ubrrh = baud_setting >> 8;
  *_ubrrl = baud_setting;

  transmitting = false;

  sbi(*_ucsrb, _rxen);
  sbi(*_ucsrb, _txen);
  sbi(*_ucsrb, _rxcie);
  cbi(*_ucsrb, _udrie);
}

Unfortunately all the properties (vars) are declared as private in HardwareSerial.h

private:
    ring_buffer *_rx_buffer;
    ring_buffer *_tx_buffer;
    volatile uint8_t *_ubrrh;
    volatile uint8_t *_ubrrl;
    volatile uint8_t *_ucsra;
    volatile uint8_t *_ucsrb;
    volatile uint8_t *_ucsrc;
    volatile uint8_t *_udr;
    uint8_t _rxen;
    uint8_t _txen;
    uint8_t _rxcie;
    uint8_t _udrie;
    uint8_t _u2x;
    bool transmitting;

I can't even subclass Hardware serial and make a new class that just has an override for the ISR

and I'm not even sure if I can access the store_char function that actually puts the incomming chars into the ring buffer, as its implemented in HardwareSerial.cpp and not declared in HardwareSerial.h

I'm not sure how C treats these functions which are in the cpp file, but are not part of the HardwareSerial class

inline void store_char(unsigned char c, ring_buffer *buffer)
{
  int i = (unsigned int)(buffer->head + 1) % SERIAL_BUFFER_SIZE;

  // if we should be storing the received character into the location
  // just before the tail (meaning that the head would advance to the
  // current location of the tail), we're about to overflow the buffer
  // and so we don't write the character or advance the head.
  if (i != buffer->tail) {
    buffer->buffer[buffer->head] = c;
    buffer->head = i;
  }
}

The cleanest way I can see to do it is to copy the whole of HardwareSerial and copy it, but this will waste 128 bytes of ram, as HardwareSerial will still allocate its 2 x 64 byte buffers (not that ram is an issue for me at the moment).

Or just ditch the incoming character handling in HardwareSerial by just writing my own ISR and probably not buffer it, as I'll be parsing the characters as they arrive.

I do wonder why people write classes and mark properties a private and not protected, as it would make everyone life a lot easier if people allowed a bit more access.

13

Just as a followup.

I found this thread on this site http://forum.arduino.cc/index.php/topic,17071.0.html

Which basically suggests I can't re-assign the ISR for the Serial, so it looks like I'm stuffed.

Though I'll see if either this information is out of date or there is a work around

14

Google "avr\interrupts" This link may help avr-libc: <avr/interrupt.h>: Interrupts

Mark

15

rogerClark:
if I send back a page, which contains references to other pages e.g. Style sheets which are also on the Arduino. That most browsers seem to parse the page on the fly and submit requests for the other pages and images etc before the initial page has finished being transferred.

As the GET requests from browsers are around 400 characters long, the subsequent request overflow the incoming serial buffer, with the result that the subsequent requests get lost, because my send function is still pushing data to the wifi module while the GET requests are coming in.

The only way I can see this being a problem is if you are reusing the HTTP connection. If so you could probably avoid it by using connection:close so that the client has to open a new connection for each request. That would stop it from initiating a new request until your server has got round to accepting the next connection. You might also be able to avoid the problem by invoking flow control on the modem, if it supports that. Just having your sketch send and receive in parallel as you're suggesting is probably just going to change the problem - there's a limit to the number of 400 bytes requests you can buffer up.

16

Peter

The wifi module hides the individual connections it receives on TCPNport 80

It just sends whatever arrives to the serial port and then in return, when you send it data, it pushes the data back out the next pending TCP connection.

It relies on the client e.g the web browser to terminate the connection, so I've had to use http 1.1 responses which include the data length in the http header, ( this makes the code more complex in the first place as I'm building dynamic pages)

Anyway, flow control may be an option.

17

rogerClark:
It just sends whatever arrives to the serial port and then in return, when you send it data, it pushes the data back out the next pending TCP connection.

Well, that is a four-letter word. Don't you have any control over whether/when incoming connections get accepted? I'm struggling to see how this would work given multiple incoming requests which need responses to go back to the correct socket, and it seems hard to believe the designers would have implemented this without any means to implement flow control, given that the Arduino has such limited capability to deal with network input.

18

rogerClark:
Which basically suggests I can't re-assign the ISR for the Serial, so it looks like I'm stuffed.

Are you looking for a solution for a single project or something general that others can use in their programming projects.

I suspect, for a single solution, you could find the code in the Arduino core that assigns the Serial1 interrupt and comment it out.

And, for a more general solution, maybe there is a way to find the memory location where the address of the interrupt vector is stored and just replace it with the address of your routine. Not for the faint-hearted, but probably quite doable.

I might look at this more sometime tomorrow evening.

...R

19

Peter

My wifi module does have other operation modes, which would allow me to control the incoming connections, well it would automatically accept them, but wait for me to receive their contents one by one, ie there is a Socket Receive command.

However each of the manufacturers of these wifi modules seems to implement a different command set, hence even if I wrote the web server for the module I have, it would be useless to other people.

But most modules seem to have a basic Transparent mode, where incoming data is sent straight to the serial output, so I was hoping to write something generic that other people could use.

My module is supposed to have flow control, but I'm not sure if it actually works, or if its enabled by default.
However it's definitely worth trying.

Although its not the neatest bit of coding, the solution may be to just sprinkle calls to my serial input handler in the parts of the code that send the output data. Because the only parts of the code that are going to take a lot of time to run are mainly going to be the bits which are sending serial data.

@Robin2.
I was hoping to write code that others could use. Hence I didn't want to do any hacks or change the IDE.
If I was just doing it for myself, I would probably use the other mode my module operates in :wink:

BTW.
I had a quick look at overriding the ISR vector after my last posting, and I didn't get any compile errors, but I'm not sure whether the ISR was getting called, as I didn't have time to do any real testing

So I will try that again when I have time

20

rogerClark:
But most modules seem to have a basic Transparent mode, where incoming data is sent straight to the serial output, so I was hoping to write something generic that other people could use.

It might be the most commonly available mode, but if it works as I'm envisaging it then it is not a good basic for a web server. For one thing, web servers typically send more than they receive so you have an inherent congestion problem which cannot be resolved unless you can apply flow control to the incoming connections. For another, it's critical that the right response goes back to the right connection and as I understand the transparent mode you've described it does not enable you to achieve that. In order to work as a web server I think you really need to manage the TCP connections yourself.