Things you always wanted to know but never dared to ask

Hi.

I was afraid of bothering a serious forum with my newbiest questions about Arduino, so I'll post them here, in the relaxed atmosphere of the Bar Sport. Feel free to join.

I'll start.

1) The bootloader. I'm not sure I fully get what is it for. If I understand correctly, once the program is compiled, it is send via USB to an USB-to-serial converter (or a microcontroller programmed to do so), and then send into the main, ATmega328 microcontroller. Correct ? Now, why is a bootloader needed to "load" the program each time the board is resetted ? Where is the program in the meantime (before it is "loaded"). And in which (presumably unusable) form is it (again, before it is loaded) ?

2) Microcontrollers beyond Arduino. Now, suppose I have a "general-prupose" microcontroller, Arduino-unrelated. As far as I understand, I can burn this bootloader into the chip and then use the Arduino compiler to burn my programs and everything will work fine. But how do you program such a chip without Arduino (generaly speaking) ?

Thanks

Bootloader monitors the serial lines, if no re-load activity is seen, then a previously loaded sketch is directed to start.

Arduino only works for a select family of Atmel chips, with some expansion supported:
www.avr-developers.com

Other chips can be programmed via Atmel AVR Studio, available at www.atmel.com

CrossRoads:
Bootloader monitors the serial lines, if no re-load activity is seen, then a previously sketch is directed to start.

But why the sketch cannot be performed directly after reset (there's a few seconds lag, due to the bootloader loading the sketch) ? I read somewhere that this is possible by burning the sketch into the chip using the ICSP. What is the difference, fundamentally, in the chip itslef ?

It is my understanding that the delay in loading the sketch gives the bootloader time to check for the previously mentioned "re-load activity". If you burn the sketch directly to the chip, with no bootloader, it doesn't wait to see if a new sketch is going to be uploaded.

I think I got it. Thank you guys.

There are many microcontrollers with different hardware and architectures. So you can't use anything from Arduino core unless you rewrite them to match the other microcontroller, which is probably doable since most microcontrollers have interrupts, UART, and counters. Beyond that there will be a lot of rewriting to do. It's not a light task but the Arduino team did get help from the wiring project so they did NOT start Arduino from scratch.

The bootloader was needed so they could add programmability (is that a word?) via USB. At the time of development USB to TTL serial chips were common and the drivers for Windows/Mac/Linux readily available so it made sense to write a bootloader that would work with that serial connection.

Nowadays there are ATmega16U2 (etc) chips replacing the USB to TTL serial chips that are cheap, have built-in USB interfaces, and could program the primary processor using ICSP without ever needing a bootloader. Given how cheap ICSP programmers are generally it would seem that they should have just implemented an ICSP programmer right on the board. But back when Arduino was being developed the costs of all those things were much higher and it wasn't economical to do things that way.

I mean, damn, that was eight whole years ago. We were playing with those neat things called "iPods" and creating accounts on this new site called "MySpace" at the time :wink:

I think I spaced out on "MySpace" stuff. What was it? Never mind I don't care, really :wink: A nice thing of having a bootloader is that someone can make a bootloader to load from SD card, ethernet, and other sources. Without a bootloader it would not be easy. I wonder if MS windows still has the 512 bytes of Boot code on bootable discs that are marked with 55 AA on the last two bytes.

Am sure they do - but try and find a 5 1/4" floppy drive on a computer these days 8)

I've seen 3 1/2" drives with USB adapters, not the larger ones (tho I haven't looked).

Good thing I kept that original luggable Compaq computer that I found in a recycle bin. Came with double sided single density DOS (2.0?) and dual 5 1/4 floppy drive. I immediately made a copy of the OS with some "new" floppies. Too bad the keyboard was made of cheap foam and they all rot away.

CrossRoads:
Am sure they do - but try and find a 5 1/4" floppy drive on a computer these days 8)

I've seen 3 1/2" drives with USB adapters, not the larger ones (tho I haven't looked).

Real programmers toggle the boot code in at the switches on the front panel using 3 finger octal or 4 finger hex 8) (and yes, been there, done that)

Lets see, I counted 46 sellers of 5 1/4" floppy drives on ebay. And for fun, 21 sellers of 8" floppy drives (plus a few more selling the controllers and not the drives).

I have a few software in 8 inch drives, including some TRS-80, which I found in the same dumpster. I still kick myself for not saving more stuff from it. Have a few hard drive platters too. If you have a working tower with multiple 3.5 5.25 drives, consider doing a phantom of floppera or play starwars with them:

there's a few seconds lag, due to the bootloader loading the sketch) ?

No, no. There's a few seconds lag while the bootloader monitors the serial port looking for a NEW sketch. If it doesn't see the sketch after "a while", it starts the existing AND ALREADY LOADED sketch.

westfw:

there's a few seconds lag, due to the bootloader loading the sketch) ?

No, no. There's a few seconds lag while the bootloader monitors the serial port looking for a NEW sketch. If it doesn't see the sketch after "a while", it starts the existing AND ALREADY LOADED sketch.

Thanks

I think asking about the inner workings and neccessity of a bootloader is a relevant question that does not need to be ferreted away in Bar Sports ...

When the microchip is powered up/reset - what is it to do?

The "reset" does a few things. It zeros some hardware (not the RAM, it will contain random bytes) and sets some registers to know values, in particular the Program Counter (the address pointed to a the reset vector which is at 0000). In short, it starts the program at a defined location.

If that is your program, fine, it runs. You now want to change the program. Forgot to put in a code to allow your program to be overwritten? Your program has a fault? Damn - we're locked out.

The way that problem is solved on the Arduino, is that the program that executes is The Bootloader. It first checks if the Serial line is sending a new program. If not, it goes to the start of "your program".

"Your program" is just two functions, setup and loop. The program, which the bootloader jumps to once it is done, is the "Arduino core", which starts by initialising the defined RAM variables to the values you have given and zero if you did not (so "your RAM" does not contain random bytes), setting ports, timers and interrupts to good/known values. Then it calls setup and loop as documented and your sketch is running.

The "New program?"-test blinks LED 13 (dont know why) while it waits for some Serial input. A few bytes are exchanged so the Loader program on the pc and the Arduino match and then the program bytes are transferred and stored in your chip. When it is done, it jumps to "Your program" the same way as if no bytes had been transferred. My experience is that if there is no USB hooked up at all, the bootloader determines quickly nothing is comming and the the main program starts quicker.

Note that the bootloader code, which also sits in the same flash as your program, is NOT overwritten. It executes on every reset, and is never changed during normal Arduino IDE programming and operation.

As the others have written, there is another solution. It is to solve the problem - how did the bootloader get in there? After the chip is manufactured all the flash bytes contain the same value (all zeros or ones, doenst matter which, useless). So there must be method whereby some bytes are set. This is handled by an internal hardware bootloader (nothing to do with the bootloader described previously). It is triggered by special pulsing Reset and the pins 11-12-13 (ie. the SPI interface), and then the bytes being loaded through the SPI with a simple protocol (if any)

So really when you hit reset
1: Circuitry inside chip set to known values
2: Is the hardware bootloaader triggered?
Yes: 3: starts reading bytes at SPI port and stores in flash.


No
3: chip program starts, begins at 0000->vector->Bootloader
4: Bootloader checks Serial line/USB, includes blinking the 13 LED
5: New program?
Yes: 6: start loading bytes

no
6: Jump to arduino core main()
7: Arduino core does more initialisation of ports and memory
8: call setup()
8b: call loop() again and again, forever

Additions and corrections welcome.

hi things i always wanted to know but dare not ask...what does "OP" mean in this forum? i see people using it a lot...haha

Original Poster - the person who started the thread. Sort if " back to the subject" after the thread has gone a bit away from the original question

oh, i always tot it was "one person" haha

Msquare:
I think asking about the inner workings and neccessity of a bootloader is a relevant question that does not need to be ferreted away in Bar Sports ...

When the microchip is powered up/reset - what is it to do?

The "reset" does a few things. It zeros some hardware (not the RAM, it will contain random bytes) and sets some registers to know values, in particular the Program Counter (the address pointed to a the reset vector which is at 0000). In short, it starts the program at a defined location.

If that is your program, fine, it runs. You now want to change the program. Forgot to put in a code to allow your program to be overwritten? Your program has a fault? Damn - we're locked out.

The way that problem is solved on the Arduino, is that the program that executes is The Bootloader. It first checks if the Serial line is sending a new program. If not, it goes to the start of "your program".

"Your program" is just two functions, setup and loop. The program, which the bootloader jumps to once it is done, is the "Arduino core", which starts by initialising the defined RAM variables to the values you have given and zero if you did not (so "your RAM" does not contain random bytes), setting ports, timers and interrupts to good/known values. Then it calls setup and loop as documented and your sketch is running.

The "New program?"-test blinks LED 13 (dont know why) while it waits for some Serial input. A few bytes are exchanged so the Loader program on the pc and the Arduino match and then the program bytes are transferred and stored in your chip. When it is done, it jumps to "Your program" the same way as if no bytes had been transferred. My experience is that if there is no USB hooked up at all, the bootloader determines quickly nothing is comming and the the main program starts quicker.

Note that the bootloader code, which also sits in the same flash as your program, is NOT overwritten. It executes on every reset, and is never changed during normal Arduino IDE programming and operation.

As the others have written, there is another solution. It is to solve the problem - how did the bootloader get in there? After the chip is manufactured all the flash bytes contain the same value (all zeros or ones, doenst matter which, useless). So there must be method whereby some bytes are set. This is handled by an internal hardware bootloader (nothing to do with the bootloader described previously). It is triggered by special pulsing Reset and the pins 11-12-13 (ie. the SPI interface), and then the bytes being loaded through the SPI with a simple protocol (if any)

So really when you hit reset
1: Circuitry inside chip set to known values
2: Is the hardware bootloaader triggered?
Yes: 3: starts reading bytes at SPI port and stores in flash.


No
3: chip program starts, begins at 0000->vector->Bootloader
4: Bootloader checks Serial line/USB, includes blinking the 13 LED
5: New program?
Yes: 6: start loading bytes

no
6: Jump to arduino core main()
7: Arduino core does more initialisation of ports and memory
8: call setup()
8b: call loop() again and again, forever

Additions and corrections welcome.

Very informative, thank you very much for having taken the time to thoroughly answer my question.

DMond:
oh, i always tot it was "one person" haha

Could also mean the original post, I think. So again, back to original topic.