Possible Bug when sending "!!!"

Hi everyone,

I just wrote my first program with the wonderful webduino lib when I've started to pull my hairs out.
My problem: every time when I upload my program it compiles fine but stops at "Uploading..."
I'm able to upload again after I've killed the "avrdude.exe" process. (Same problem again afterward)

finally I've found the reason, I was sending a text that ended with !!!

server.print("!!!");

It's working for 1, 2, 4, 5 exclamationmarks, but not for 3 or 6 (didn't test 9)

The same thing happens with the serial.print:

Serial.print("!!!");

Is it a local problem or can someone confirm this?

(Win 7 64Bit, Arduino Mega 2560, Ethernet shield)

Rai

Known feature.
Of course, it begs the question - why would you want to print more than one exclamation mark?
Or indeed, except for ellipsis, any more than one of any particular punctuation mark consecutively.

AWOL:
Known feature.
Of course, it begs the question - why would you want to print more than one exclamation mark?
Or indeed, except for ellipsis, any more than one of any particular punctuation mark consecutively.

To be honest, it doesn't beg those questions at all. The question it begs is why the designer chose to make the bootloader incapable of dealing with an arbitrary string, without either documenting it in any way that typical users could be expected to notice, or providing any means to warn developers who inadvertently used that string in their code.

:wink:

You can never have enough exclamation marks if your website tries to tell a female user how to program the heating system.

Is there a bugtracker around where such "features" are documented?

Rai

PeterH:

AWOL:
Known feature.
Of course, it begs the question - why would you want to print more than one exclamation mark?
Or indeed, except for ellipsis, any more than one of any particular punctuation mark consecutively.

To be honest, it doesn't beg those questions at all. The question it begs is why the designer chose to make the bootloader incapable of dealing with an arbitrary string, without either documenting it in any way that typical users could be expected to notice, or providing any means to warn developers who inadvertently used that string in their code.

It was a feature explicitly added by the Mega bootloader developer to 'trigger' a monitor feature in the bootloader code. This feature is probably not used by nearly anyone (other then the developer maybe? :wink: ), but none the less the author needed to use some escape code to activate the monitor feature and therefore choose !!!. It was documented by the author of the bootloader, but the Arduino team did not do a very good job communicating it to the general population when it released the original mega board. I believe they have a new bootloader code ready (which also will handle WDT resets correctly) for the mega board but have not yet started to distribute it with the current IDE release and I have no idea if present mega boards are shipping with the new bootloader installed, but I bet no. Such is the case with a platform that relies on so many open source libraries and code modules not written by the platform developers. The problem was not with the original bootloader code developer but the Arduino team failure to properly communicate the 'feature' and their very slow process of implementing an updated bootloader for the mega board, even though it's been known and available for well over a year I believe.

Lefty

To be honest, it doesn't beg those questions at all. The question it begs is why the designer chose to make the bootloader incapable of dealing with an arbitrary string, without either documenting it in any way that typical users could be expected to notice, or providing any means to warn developers who inadvertently used that string in their code.

Where, exactly, would you expect the bootloader to do that?

PaulS:
Where, exactly, would you expect the bootloader to do that?

Firstly, as a user it's not up to me to find the solution - I'm just complaining about the problem. I'm not familiar with the implementation of the bootloader and the protocol the IDE uses to talk to it. The whole point of the Arduino project is that it hides us users from needing to know this stuff.

However, I would be astonished if no solution was possible - which is what you seem to be implying by your question.

As far as I can see, the problem is caused by an undesirable feature in the bootloader which should ideally have been removed, or not introduced in the first place. In other words, it's a bug. It may have been introduced knowingly, but it's still a bug.

If the bug can't be removed from the bootloader for some reason, it may be possible for the bootloader to indicate to the IDE when it has entered this diagnostic mode, which would enable the IDE to report it to the user in some meaningful way and not just leave them poking around in the guts of hung processes trying to guess what's broken.

If the developers can't figure out a way to achieve that, then surely at least it ought to be possible for the IDE to notify the user when they were attempting to download a sketch which the bootloader is not expected to handle correctly.

And if they can't even manage that, then would it be asking too much to describe the problem in the trouble-shooting guide?

Firstly, as a user it's not up to me to find the solution

Agreed. But, you did suggest a solution that is completely impossible. The bootloader has no way of knowing whether the stream that contains !!! contains them because they were meant to trigger the functionality that they are a key for, or if they are embedded in the data stream as a result of the compiler finding them in a string literal.

Now, I'll grant you that using %%% to trigger that functionality would have probably been better than using !!!, since so many people are so excited when things finally work, and I can't recall a single instance where someone posted code containing "%%%".

And, I'll grant you that some documentation of the fact that !!! meant something to one of the bootloaders should have been generated.

However, I would be astonished if no solution was possible - which is what you seem to be implying by your question.

The bootloader certainly can't tell the difference between code that a knowledgeable developer generated that contains !!! and code that an overexcited newbie generated that contains !!!. There is no compiler setting that defines skill level, and, even of there were, the IDE doesn't provide a way of setting compiler options. And, it certainly can't tell.

If the bug can't be removed from the bootloader for some reason, it may be possible for the bootloader to indicate to the IDE when it has entered this diagnostic mode, which would enable the IDE to report it to the user in some meaningful way and not just leave them poking around in the guts of hung processes trying to guess what's broken.

That might be the case, but it would certainly increase the size of the bootloader, and would not really help unless the displayed message was very explicit. Something like:
"Hey, I know you're excited that your code finally does what you want, but !!! means something to me that you probably didn't intend for me to do. Signed, the bootloader".

Then, how would you convey to the bootloader that you wanted it to continue, because you knew what you were doing, vs. conveying the need to stop loading the code?

How would you restore the code that was previously there, vs. leaving the chip with no code to run?

If the developers can't figure out a way to achieve that, then surely at least it ought to be possible for the IDE to notify the user when they were attempting to download a sketch which the bootloader is not expected to handle correctly.

This presupposes that the IDE knows, or has some way of learning, which bootloader is loaded on the chip, and that each bootloader has some specific ID that is unique. I don't know whether either supposition is true.

And if they can't even manage that, then would it be asking too much to describe the problem in the trouble-shooting guide?

Certainly not, but, given how much trouble we have getting people to even be aware that it exists, is this really going to help?

Edit: Minor change to one sentence, as highlighted.

PaulS:
Agreed. But, you did suggest a solution that is completely impossible. The bootloader has no way of knowing whether the stream that contains !!! contains them because they were meant to trigger the functionality that they are a key for, or if they are embedded in the data stream as a result of the compiler finding them in a string literal.

Perhaps the bootloader can differentiate between those two cases, or perhaps it can't - I don't know. I suspect it could do, if the designer had wanted it to. However, I didn't mean to imply that the bootloader should try to differentiate between those two cases. I don't know whether the bootloader has any means of providing feedback to the loader running on the PC, but since we sometimes see sync errors and so on I suppose it probably does. This suggests to me that it may be possible to provide feedback to indicate that the bootloader has gone into 'diagnostic' mode (whatever that means). Simply telling the user that this has happened would make it hugely easier for them to figure out what they've done wrong and how to correct it. Since I don't have any idea how the bootloader works and what the diagnostic mode does this is pure speculation on my part based on what I guess is happening under the covers. Obviously a developer trying to solve this problem would have a far clearer view of what options are available, if any. But from my perspective, it is not clear that it is intractable.

If in fact it's not feasible to do that, then then having the IDE warn the user that they have used an unsupported string literal does not seem impossible. If it isn't practical what whatever reason, then documenting this known problem in the place that users are directed to go look for advice about known problems does not seem unreasonable.

I've no knowledge about the software involved or what sensible options there are to address the problem, but it does not seem to me like the sort of problem that should be intractable. I'm not at all persuaded by AWOL's suggestion that the existing behaviour is reasonable, or your implication that there is no sensible way to correct the problem.

I suspect the truth is that it simply hasn't been addressed yet, because the developers are all busy working on more important things.

AWOL:
Known feature.
Of course, it begs the question - why would you want to print more than one exclamation mark?
Or indeed, except for ellipsis, any more than one of any particular punctuation mark consecutively.

Well to be fair, this 'escape code sequence' could be also triggered by any constant binary data array table, 0x21,0x21,0x21 (or possibly even from complied instruction code?) contained in a sketch source file (which would make it near impossible to figure out most of the time), so it's not just a matter of poor ascii english usage. :wink:

I suppose it probably does.

I agree. There must be some two way communication going on, for the bootloader to be able to say that it is ready for the next page of data.

Exactly what form that communication takes, though, I have no idea.

This suggests to me that it may be possible to provide feedback to indicate that the bootloader has gone into 'diagnostic' mode

Now, that certainly sounds like a good idea.

Simply telling the user that this has happened would make it hugely easier for them to figure out what they've done wrong and how to correct it.

I'm not sure I agree. Since the communication process between the bootloader and the uploader are normally not shown, seeing "Entering diagnostic mode" on the display window, in that horrid red color on the black background, wouldn't tell me squat, and certainly wouldn't trigger an aha moment.

If in fact it's not feasible to do that, then then having the IDE warn the user that they have used an unsupported string literal does not seem impossible.

Whether the string literal is supported or not depends on the bootloader. Not all Arduinos ever produced are using the same one. Changing the bootloader that is put on future Arduinos isn't going to alter any of mine. And, it's not just string literals that can cause the problem.

char threeBangs[4];
threeBangs[0] = '!';
threeBangs[1] = '!';
threeBangs[2] = '!';
threeBangs[3] = '\0';
Serial.print(threeBangs);

will result in the same issue when the compiler gets done optimizing my attempt at foolishness. And the resulting code will trigger the bootloader to go into diagnostic mode.

I suspect the truth is that it simply hasn't been addressed yet, because the developers are all busy working on more important things.

I suspect its more that they have no control over what happens after the Arduinos are sent on their way (not counting all the clones that others are making), so fixing an issue that trips one or two people a month isn't a high priority.

Again that supposes that the Arduino team has full ownership of the bootloader code. I don't think that they do. Optiboot (the one with the issues) was not developed by the Arduino folks, if I remember correctly.

Again that supposes that the Arduino team has full ownership of the bootloader code. I don't think that they do. Optiboot (the one with the issues) was not developed by the Arduino folks, if I remember correctly.

I don't think the mega boards even presently ship with a Optiboot bootloader, it's just a mega specific bootloader? Now that doesn't mean there is not a Optiboot version out there somewhere that has been modified to run on the mega boards or not, and indeed I recall there does exist a 'fixed' bootloader for the mega available somewhere that solves both the !!! problem and handling the WDT correctly, it just hasn't been placed in the arduino IDE distribution nor is being installed in present hardware. At least that is my memory of the existing situation. My opinion is that the 'official Arduino team' has been very lacks (this problem has existed sense the release of the original mega1280 board as best I recall) at implementing a fix for this long standing situation, regardless of who the original developer was and what the best final solution should be.

Lefty