Random characters on serial connection being opened

Projects Programming
1

I am working on an Arduino project for a digital multimeter interface. When the user connects to the interface via the USB virtual serial port, a version string is displayed. This is done with Serial.println(F(""verstr"); in the setup() section.

The problem was that I kept getting the string displayed in the terminal vertically one character per line prior to it finally being displayed on one complete line. Sometimes I got random characters displayed prior to the version string as well. I seemed to have solved this by placing a Serial.flush() just before the Serial.println() statement to "clear" the serial transmit buffer. This works for most part, but there is a caveat.

When you connect to the serial port, a reset signal is generated and the interface resets, firing up the bootloader. This causes a delay, which, in turn, causes problems with auto-detection with third party programs. It is possible to prevent the interface from being reset by placing a capacitor between the RESET and GND pins. This solves the auto-detection problems nicely. However, since setup() is no longer being run on reset, I can connect but I don't get the version string displayed and sometimes random characters are still displayed on connection. I figured if there is a way to detect the event of a connection being opened from the terminal on the PC, then I could use this to do a Serial.flush() and display the version string, but I can't find such a function. The nearest I found is serialEvent() although this seems to be intended to capture incoming data.

With the bootloader disabled, is it possible to detect on the Arduino when a serial connection is opened from the host PC?

2

Nope, nothing i send down to Rx so no way to know.

If you want the reset but don't want the bootloader to fire, program the Arduino without bootloader aka ISP.

3

BitSeeker:
With the bootloader disabled, is it possible to detect on the Arduino when a serial connection is opened from the host PC?

Yes, because the host will assert the DTR line to low. You can check this line on another pin on the arduino.

4

Yeah, if you go to the effort of breaking out the DTR you indeed can. But programming ISP is in my opinion easier.

5

septillion:
Yeah, if you go to the effort of breaking out the DTR you indeed can. But programming ISP is in my opinion easier.

He doesn't want to program the arduino. He wants to detect, when a client opens the serial connection to send the appropriate message.

BitSeeker:
However, since setup() is no longer being run on reset [...]

setup() still runs on reset, but opening a serial connection doesn't reset the arduino anymore.

BitSeeker:
I figured if there is a way to detect the event of a connection being opened from the terminal on the PC, then I could use this to do a Serial.flush() and display the version string, but I can't find such a function. The nearest I found is serialEvent() although this seems to be intended to capture incoming data.

The only way to make this work (without checking the DTR signal line), is to send a start/stop message from the pc to the arduino. The arduino then only sends data between start/stop messages.

6

LightuC:
He doesn't want to program the arduino. He wants to detect, when a client opens the serial connection to send the appropriate message.

No, he just don't want to wait when serial is opened and send a version string. Blocking auto reset, detecting DTR and sending a string is one way. Just using the reset and send that string in setup() but don't have a bootloader which delays it is also a possibility without breaking out DTR.

7

septillion:
No, he just don't want to wait when serial is opened and send a version string.

BitSeeker:
[...]I can connect but I don't get the version string displayed and sometimes random characters are still displayed on connection.

I might have misunderstood him, but it seems he wants exactly that: When opening the serial connection from the pc, send the version string without any delay. This has nothing to do with the way he programs the arduino (he can start avrdude and then reset the board manually to invoke the bootloader). However, i agree with you that when disabling the auto reset feature, isp programming is much less bothersome than hitting reset manually.

When third party applications are involved you might not have the ability to send a start/stop message, so breaking out the dtr signal might be the only way.

8

Thanks for the replies so far. Since the DTR signal is, in effect, defeated by being drained to GND via the capacitor, which, in turn, prevents DTR from triggering a reset via the RESET pin on the 328P, it would think it improbable that this signal could be detectable on a spare Arduino pin. If the track were cut and a switch was used between the DTR and RESET instead of the capacitor solution, then I guess that could be an option when the switch was in the open position. However, I do not want to modify the board at this point. If I find no other solution, I might consider it later.

I'm not sure what it means to "program the Arduino without bootloader aka ISP"?
I know it is possible to program the board via the Reset switch, but my attempts to do this so far have been unsuccessful. I understand that timing is critical and rather frustrating. I'm also uncertain how the board can be reset without firing the bootloader since a bootloader of some sort is required to load the program. Never really ventured into this territory, so its all a bit new to me. Any further info would be appreciated.

LightuC:
setup() still runs on reset, but opening a serial connection doesn't reset the arduino anymore.

I take your point about this. I phrased it incorrectly but that was the issue. When you connect, the serial port is opened, but clearly since the version string is not being printed, setup() is not being run. I'm not sure but it seems that the program just keeps on running via the loop() while the Arduino is powered up, so I'm guessing that after re-connection program execution just drops back into the loop() routine.

UPDATE: just noted the additional posts. Yes, the idea is to send a version string immediately on connection so that the third party program can identify the interface. More importanly though, is to prevent any spurious characters which looks messy on a terminal connection but also are likely to "confuse" any third party program. At present I just put the capacitor into the relevant pin sockets when testing with the program in question, but it otherwise remains pulled out development and uploading the program. However, at some point I will need a more permanent solution.

9

LightuC:
This has nothing to do with the way he programs the arduino

It has anything to do with the way the Arduino is programmed because that delay is from the bootloader :wink: No bootloader = no delay :slight_smile:

10

BitSeeker:
Thanks for the replies so far. Since the DTR signal is, in effect, defeated by being drained to GND via the capacitor, which, in turn, prevents DTR from triggering a reset via the RESET pin on the 328P, it would think it improbable that this signal could be detectable on a spare Arduino pin. If the track were cut and a switch was used between the DTR and RESET instead of the capacitor solution, then I guess that could be an option when the switch was in the open position. However, I do not want to modify the board at this point. If I find no other solution, I might consider it later.

Yes, this solution requires you to modify the board. You could try to solder a wire to the capacitor used to pull the reset line high again, after dtr was pulled low.

BitSeeker:
I'm not sure what it means to "program the Arduino without bootloader aka ISP"?

The microcontroller on the arduino can be programmed via the ISP interface (the six pins sticking out of your board). This programming interface is build-in.

BitSeeker:
I know it is possible to program the board via the Reset switch, but my attempts to do this so far have been unsuccessful. I understand that timing is critical and rather frustrating.

If i remember correctly, avrdude makes 10 attempts to program the arduino before giving up. Press the reset button, hit upload, wait for the first programmer out-of-sync message and release the reset button. This should do the trick.

BitSeeker:
I'm also uncertain how the board can be reset without firing the bootloader

On the arduino you can't (maybe except for a reset triggered by the watchdog, depending on the bootloader you use).

BitSeeker:
I take your point about this. I phrased it incorrectly but that was the issue. When you connect, the serial port is opened, but clearly since the version string is not being printed, setup() is not being run.

The setup function only runs after reset or by calling it manually, of course it is not run, when you open the serial connection. I just wanted to clarify this.

BitSeeker:
I'm not sure but it seems that the program just keeps on running via the loop() while the Arduino is powered up, so I'm guessing that after re-connection program execution just drops back into the loop() routine.

Nothing will happen after re-connecting to the arduino. It just doesn't care.

11

septillion:
It has anything to do with the way the Arduino is programmed because that delay is from the bootloader :wink: No bootloader = no delay :slight_smile:

Yes, the delay comes from the bootloader waiting for data on the serial interface. Removing the bootloader will result in almost zero startup delay. If you want to go that way, please note that you have to disable the BOOTRST fuse first.

But again, that doesn't solve the problem of sending a message after opening the serial connection. The solution will depend on the way, the third party programs auto detect the device.

12

Why not? Now you can send it in setup() and be done :slight_smile:

And I have to say, never thought about the BOOTRST fuse but never had trouble with it after using Upload using programmer...

13

septillion:
Why not? Now you can send it in setup() and be done :slight_smile:

Yes, but now you have to manually reset the arduino when you open the serial connection to get the version. I am pretty sure, that is a much bigger delay than the bootloaders.

septillion:
And I have to say, never thought about the BOOTRST fuse but never had trouble with it after using Upload using programmer...

That is somewhat strange, because the BOOTRST fuse tells the avr to fetch instructions from the bootloader section. If, while programming via ISP, you erase the content of the flash, the arduino will never execute anything, because it jumps to the now empty bootloader section. If you don't disable the fuse and don't erase the flash, the bootloader will be run after every reset.

EDIT: Actually, to correct myself, after some digging the opcode 0xFFFF is interpreted by the avr decode engine as SBRS r31, 7. So it will execute, until the PC wraps around to 0 and then starts executing your program.

14

Well, I've just looked up "ISP programming" and avrdude.

https://www.arduino.cc/en/Tutorial/ArduinoISP

It seems that this requires a separate programmer or Arduino to use as one. I see there are some dedicated programmers on eBay but it seems just as cheap to get a Nano. I then also realised that I had already purchased a programmer and a board off eBay some time ago for the Atmega16 which has been stuck in a box for some months and seen no use. I was able to find it amoung my purchases on eBay and its described as "ATMEGA16 ATmega32 Board + USBISP 3.3V / 5V AVR Download Programmer". The board is specifically designed for 40-pin ATMEGA chips, but the programmer looks identical to and has the same component layout as this:

https://www.aliexpress.com/item/WAVGAT-USBasp-USB-ISP-3-3V-5V-AVR-Programmer-USB-ATMEGA8-ATMEGA128-New-10PIN-Wire-Support/32578298532.html

It doesn't have the Wavgat name on it, just www.betemcu.cn printed on one side of the board. However, I'm not sure if this would work with the 328p? I'm also a little confused by all the ribbon cables on these programmers being terminated with a 10 way connector, whereas the SPI interface on the Arduino is six pins?

septillion:
Why not? Now you can send it in setup() and be done :slight_smile:

I wasn't sure that I followed this, but I have just looked up the information on Arduino fuses here:

http://www.martyncurrey.com/arduino-atmega-328p-fuse-settings/

So am I correct in thinking that if I set the BOOTTRST fuse to 0 then on connecting to the serial port a reset will be triggered, but, instead of loading the bootloader, the Arduino will just boot to the program code and therefore run Setup()? Am I also correct in thinking that the Arduino can still be programmed via the SPI connector using an AVR programmer (although obviously not using the USB port) after this modification?

If so, then I don't mind trying this as I am curious by how much this will reduce the delay. However, I still need a little help with deciding on which programming approach is best and which programmer hardware to use?

15

BitSeeker:
Well, I've just looked up "ISP programming" and avrdude.

https://www.arduino.cc/en/Tutorial/ArduinoISP

It seems that this requires a separate programmer or Arduino to use as one. I see there are some dedicated programmers on eBay but it seems just as cheap to get a Nano. I then also realised that I had already purchased a programmer and a board off eBay some time ago for the Atmega16. Not sure if this would work with the 328p? I'm also a little confused by all the ribbon cables being terminated with a 10 way connector, whereas the SPI interface on the Arduino is six pins?

Yes, ISP requires an additional programmer. They should come with a 6-pin connector and a 10-pin connector. Which programmer did you buy? And yes, the arduino can be used as an ISP programmer as well, there is an example sketch for that.

BitSeeker:
I wasn't sure that I followed this, but I have just looked up the information on Arduino fuses here:

Arduino / ATmega 328P fuse settings – Martyn Currey

So am I correct in thinking that if I set the BOOTTRST fuse to 0 [...]

The fuse has to be unprogrammed (value 1). Please refer to the datasheet for your avr (search for BOOTRST).

BitSeeker:
then on connecting to the serial port a reset will be triggered, but, instead of loading the bootloader, the Arduino will just boot to the program code and therefore run Setup()?

Yes.

BitSeeker:
Am I also correct in thinking that the Arduino can still be programmed via the SPI connector using an AVR programmer (although obviously not using the USB port) after this modification?

Via the ISP connector, yes.

BitSeeker:
If so, then I don't mind trying this as I am curious by how much this will reduce the delay. However, I still need a little help with deciding on which programming approach is best and which programmer hardware to use?

Please note, that changing the fuses can brick your microcontroller (especially the RSTDISBL). You will not be able to recover it without a high-voltage programmer.

The delay can be calculated and set by the CKSEL0 and SUTx fuses. After that time the startup code will run and after that setup() will be called.

16

Just updated my previous post to provide more details about the programmer. It definitely has a 10 way socket at each end of the ribbon cable and there is no other cable. They seem to be quite cheap so happy to buy another. Just need to know which one so as to be sure I get the right tool for the job.

Regarding bricking the Arduino, I did note that and it does make me a bit nervous but if I am to learn then the only way to do that having researched the matter thoroughly is to then carefully experiment. I guess it might be just as well to buy a couple of spare Arduino boards while I'm at it.

17

BitSeeker:
Just updated my previous post to provide more details about the programmer. It definitely has a 10 way socket at each end of the ribbon cable and there is no other cable. They seem to be quite cheap so happy to buy another. Just need to know which one so as to be sure I get the right tool for the job.

In that case i recommend to get yourself another arduino to do the job. I am using the ALL-AVR ISP-Programmer from Diamex, but this costs a lot more (around 30 €).

BitSeeker:
Regarding bricking the Arduino, I did note that and it does make me a bit nervous but if I am to learn then the only way to do that having researched the matter thoroughly is to then carefully experiment. I guess it might be just as well to buy a couple of spare Arduino boards while I'm at it.

You can also calculate the fuses with an online calculator and double check in the datasheet. Just make sure not to touch the RSTDISBL fuse.

18

Well, if I'm going to think about spending around 30 € (or 30GBP), then I was wondering whether a BusPirate might be a better option? I had considered purchasing one in the past for accessing the SPI bus on various other devices, but never got around to it. I see that the Bus Pirate is supported as an AVR programmer by avrdude (provided one has the correct versions) as well as in the Arduino IDE. So would this be a good option?

19

LightuC:
Yes, but now you have to manually reset the arduino when you open the serial connection to get the version.

Why? Loading a program via ISP does not magically disable the auto-reset...

LightuC:
EDIT: Actually, to correct myself, after some digging the opcode 0xFFFF is interpreted by the avr decode engine as SBRS r31, 7. So it will execute, until the PC wraps around to 0 and then starts executing your program.

That was something that I guessed after my last reply. Pretty stupid actually! Time to add a "no bootloader 328" to the boards file.

20

septillion:
Why? Loading a program via ISP does not magically disable the auto-reset...

You are absolutely right. Don't know what I was thinking ...

So the way to go then is:

  1. Disable the BOOTRST fuse to make the arduino start fetching from address 0x0000
  2. (Optional:) Change the boot section size to increase the usable flash
  3. Program the arduino via ISP
  4. If the delay is still to large, you can go bare-bones. First I would try to avoid global variables, as they will increase the startup time. You may also want to set the CKSEL0 and SUTx fuses appropriately.