Go Down

Topic: Creating Custom Boards for Board Manager (Read 10368 times) previous topic - next topic

pert

The very word "core" starts to make me feel like a headache is approaching, so I am going to please myself with the thought I don't need to worry about such things yet (I'm not sure I even comprehend what all a core entails - thankfully).
When I say core I mean the files inside the cores folder in the hardware package, which contains the source files for the standard Arduino API, basically just a collection of libraries and the main function definition. You will also see the term core used to refer complete hardware packages (e.g. "ESP8266 core for Arduino") but I think that's confusing because a hardware package may not even contain a cores folder and instead reference the core from a different package, as does the OpenPanzer package. So I use the term package to refer to a collection of files used to add boards (or programmers in rare cases) to the Arduino IDE.

I appreciate again all your help and I do hope this thread is useful to some other poor sap out there who has graduated to this level...
Glad I've been able to help. I know it's been a bit of work for you to get this set up but I do think it will make your project significantly easier for the users to install and update and will reduce the number of support requests. I've seen authors of other packages be a bit skeptical at first about the value of adding Boards Manager support but they definitely came around on the subject over time.

Yvanox

#16
Mar 13, 2019, 12:35 am Last Edit: Mar 13, 2019, 12:36 am by Yvanox
This thread is the most helpfull thing i've found in the whole internet, you guys rock.


If someone is still there i just want to ask...   
(where did you learned how to develop/code your own bootloader)

CrossRoads

You could start with the datasheet. For example, in the Nov 2016 datasheet for the Atmega328P: 

30.8.13. Simple Assembly Code Example for a Boot Loader

which I won't copy here as it's a bit lengthy with comments and all.
Designing & building electrical circuits for over 25 years.  Screw Shield for Mega/Due/Uno,  Bobuino with ATMega1284P, & other '328P & '1284P creations & offerings at  my website.

LukeZ

#18
Mar 13, 2019, 09:40 am Last Edit: Mar 13, 2019, 09:41 am by LukeZ
This thread is the most helpfull thing i've found in the whole internet, you guys rock.
That's pretty much how I felt when Pert helped me through this molasses.

If someone is still there i just want to ask...   
(where did you learned how to develop/code your own bootloader)
The best way to understand the bootloader is to examine closely ones that already exist, and figure out what part that you need to modify for your own version. The other thing is to question yourself as to whether a custom bootloader is really needed or necessary, in most cases it is not, but there are exceptions. 

The readme on this GitHub page has a link to the bootloader we based ours off of (applicable only to the ATmega2560), and information about compiling and flashing a custom bootloader, maybe some of it will be useful.

LukeZ

#19
Aug 01, 2019, 05:59 am Last Edit: Aug 01, 2019, 06:02 am by LukeZ
Well two years have passed and during that time I've been trucking along like a little song, but then I had to upgrade my Arduino IDE from 1.8.1 to 1.8.5 due to some Teensy stuff that didn't compile in 1.8.1, and that has unleashed new problems on this old (but still ongoing) Open Panzer project.

The issue now is that, although the IDE loads my custom boards file just fine, and it shows up in Board Manager, I can't compile my project nor flash the custom bootloader.

When I try to compile my code using my custom board definition, I get the error:
recipe.preproc.macros pattern is missing

If I try to burn the bootloader, I get the message:
Could not find tool avrdude

Now our firmware is running on an ATmega2560 processor and the firmware compiles just fine if I select the Arduino-standard ATmega2560 board (and runs fine). I am also still able to flash my custom bootloader by using avrdude from the command line, so for now I have assumed that my firmware and bootloader hex are fine. Since the package is also loaded without error by the IDE I'm also assuming that my json file is also still fine and working as it should. 

I've tried rolling back to IDE 1.8.1 but the damage was done, now I get the same errors regardless. Tried also installing 1.8.1 fresh on a different computer and same deal, so either I was really lucky that it worked in the old days, or something has changed about Arduino's boards requirements and my package is no longer up to spec.

What I am going to guess is that the problem is related to my boards.txt file and/or my platform.txt file, which can be seen in the "avr" folder of this GitHub repo, but the contents of which are also copied below:

boards.txt
Code: [Select]
# Open Panzer TCB Board Configuration
#
# For more information see:
# - https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification
# - https://github.com/arduino/Arduino/wiki/Arduino-Hardware-Cores-migration-guide-from-1.0-to-1.6
# - http://forum.arduino.cc/index.php?topic=457405


##############################################################

optcb2560.name=Open Panzer TCB (Mega 2560)

optcb2560.bootloader.tool=avrdude
optcb2560.bootloader.extended_fuses=0xFD
optcb2560.bootloader.high_fuses=0xDA
optcb2560.bootloader.low_fuses=0xF7
optcb2560.bootloader.unlock_bits=0x3F
optcb2560.bootloader.lock_bits=0x0F
optcb2560.bootloader.file=openpanzer_boot/optcb2560_boot.hex

optcb2560.build.mcu=atmega2560
optcb2560.build.f_cpu=16000000L
optcb2560.build.core=arduino
optcb2560.build.variant=mega
optcb2560.build.board=AVR_MEGA2560

optcb2560.upload.protocol=wiring
optcb2560.upload.maximum_size=258048
optcb2560.upload.maximum_data_size=8192
optcb2560.upload.speed=115200
optcb2560.upload.tool=avrdude


platform.txt
Code: [Select]
# Open Panzer TCB Platform Configuration
# --------------------------------------------------------------------------->>
# We are leaving it up to the IDE to decide what platform specifics to use,
# but we still create this file so we can have the header name in the boards
# list (name value below)
# --------------------------------------------------------------------------->>

name=Open Panzer Boards
version=1.0.3


You can see these are rather simple files. We discussed this earlier in the thread, and it was my understanding then (and it worked then) that by specifying for example "upload.tool=avrdude" or "build.core=arduino" that we would simply tell the IDE to use whatever it already knew about such things. Perhaps we need to be more explicit now?

From my preliminary searching it looks like the documentation on these topics is the same now as it was two years ago, and it all refers to 1.5 or 1.6. Could there be un-documented changes (or documents I have not found) related to more recent developments? 

I'm happy to do as much work myself as I can but after pulling my hair out for the past couple days I realize I need some pointers about where to be looking, and I'm hoping Pert or someone will be able to assist.

Thank you in advance for what I know will be a very helpful response!



pert

Hi again!

When I try to compile my code using my custom board definition, I get the error:
recipe.preproc.macros pattern is missing
Add this line to your platform.txt:
Code: [Select]
preproc.macros.flags=-w -x c++ -E -CC
recipe.preproc.macros="{compiler.path}{compiler.cpp.cmd}" {compiler.cpp.flags} {preproc.macros.flags} -mmcu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {compiler.cpp.extra_flags} {build.extra_flags} {includes} "{source_file}" -o "{preprocessed_file_path}"


If I try to burn the bootloader, I get the message:
Could not find tool avrdude
You need to reference the resources from the Arduino AVR Boards hardware core, as documented here:
https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification#referencing-another-core-variant-or-tool

Change this line in boards.txt from:
Code: [Select]
optcb2560.bootloader.tool=avrdude
to:
Code: [Select]
optcb2560.bootloader.tool=arduino:avrdude

This line:
Code: [Select]
optcb2560.build.core=arduino
to:
Code: [Select]
optcb2560.build.core=arduino:arduino

This line:
Code: [Select]
optcb2560.upload.tool=avrdude
to:
Code: [Select]
optcb2560.upload.tool=arduino:avrdude

You need to restart the Arduino IDE before changes you make to platform.txt and boards.txt will take effect.

LukeZ

Thanks so much for your help Pert, this has fixed everything. For what it's worth, this is what happened.

I was in fact already using the "arduino:xxx" nomenclature since 2017, for every entry except for the bootloader tool which I had accidentally originally designated simply "bootloader.tool=avrdude"

However for whatever reason the past few years this had worked (or maybe I wasn't flashing the bootloader that often anyway). When I made some recent changes the bootloader ability from within the IDE no longer worked.

Then the past few days I was trying to figure out what was going on, I looked at some other people's boards.txt files and noticed many of them did not have the "arduino:xxx" nomenclature so I made a test package yesterday removing all that. Which only broke things worse.

Setting everything back to "arduino:xxx", including the bootloader tool, has fixed my issues. For anyone following along or wishing to make their own boards.txt file, here is my corrected version:

boards.txt
Code: [Select]
# Open Panzer TCB Board Configuration
#
# For more information see:
# - https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification
# - https://github.com/arduino/Arduino/wiki/Arduino-Hardware-Cores-migration-guide-from-1.0-to-1.6
# - http://forum.arduino.cc/index.php?topic=457405


##############################################################

optcb2560.name=Open Panzer TCB (Mega 2560)

optcb2560.bootloader.tool=arduino:avrdude
optcb2560.bootloader.extended_fuses=0xFD
optcb2560.bootloader.high_fuses=0xDA
optcb2560.bootloader.low_fuses=0xF7
optcb2560.bootloader.unlock_bits=0x3F
optcb2560.bootloader.lock_bits=0x0F
optcb2560.bootloader.file=openpanzer_boot/optcb2560_boot.hex

optcb2560.build.mcu=atmega2560
optcb2560.build.f_cpu=16000000L
optcb2560.build.core=arduino:arduino
optcb2560.build.variant=arduino:mega
optcb2560.build.board=AVR_MEGA2560

optcb2560.upload.protocol=wiring
optcb2560.upload.maximum_size=258048
optcb2560.upload.maximum_data_size=8192
optcb2560.upload.speed=115200
optcb2560.upload.tool=arduino:avrdude


Whether the extra lines you suggested to platform.txt were strictly necessary or not I don't know, the above may have been my only problem, but I added them for good measure.

LukeZ

#22
Aug 01, 2019, 11:35 pm Last Edit: Aug 01, 2019, 11:37 pm by LukeZ
I should mention one other thing - when goofing around with board packages I have found it insufficient to load the board from Board Manager, then close the IDE, and then reopen the IDE. These actions alone may still lead to errors when you try to test whatever you were changing.

What I have found necessary, and we may have discussed this before, is to manually delete all the old package information from Windows first, then install the new package using Boards Manager, then re-starting the IDE.

The file locations are not in the Arduino install directly as they were in the ancient past. They are now located in the user directory, something like this:
C:\Users\JohnDoe\AppData\Local\Arduino15\

Within that folder there are three places where files need to be deleted - in that folder itself you will find the json file, possibly multiple versions of it (one with a sig.tmp extension). Delete those.

Then in the "packages" subfolder you will find the actual custom packages themselves, delete the one you are trying to change.

Finally in the "staging\packages" subfolder you will find the .tar.bz2 or zip file that contained the package and which Arduino downloaded, delete that as well.

Now go back and open the IDE, go to Boards Manager and re-install the package you have changed, then restart the IDE. Now your changes should really and truly have taken effect.

Thanks again Pert for being such an invaluable assistance!

pert

Thanks so much for your help Pert, this has fixed everything.
You're welcome. I'm glad to hear it's working.

I looked at some other people's boards.txt files and noticed many of them did not have the "arduino:xxx" nomenclature
You can define your own tools in the Boards Manager package (or manually install them) and provide your own core. In that case, it's not necessary to reference anything from another package. However, in this case we know that the AVR tools and core will always be installed already with Arduino AVR Boards and that these resources will work fine for your package. So it makes sense to share the resources. That means less maintenance effort for you and a faster installation and less disk space usage for the users. Win all the way around. In some cases, a package will need a custom core, specific tool versions, or resources not already provided by another package. In that case, they can't use the reference system.

I should mention one other thing
That is a tricky thing, which I'm sure has caused people a good deal of confusion over the years. Thanks for taking the time to share your findings.

Go Up