Arduino Core Libraries Edit [SOLVED]

Hello all,

I've been trying to establish a hardware serial connection between two Arduino Unos over jumper cables (eventually other capable mediums) with a baud rate of 480600. That part has worked fine. The issue I've run into is making my data packet have 9 data bits. I didn't want to edit the Arduino libraries myself if I didn't have to, and fortunately, I found another thread that offered the modified files. The author received lots of praise for the modifications so I assumed they were a good place to start.

Unfortunately, upon including them in Arduino core libraries and trying to use some of the alterations, the Arduino IDE did not recognize these changes. I wasn't sure if the issue was the library (I didn't think so) or my inclusion of these modifications into my Arduino library. So I tried to make a simpler edit to the Serial library myself.

I changed the return line of the Serial.read() call to always return the value 62. I figured this way, I'd know very quickly if my Arduino system was using the updated files. Of course, upon running my code, it was still working using the original read method, so I know that I'm not correctly updating my Arduino core libraries.

TLDR: I am trying to update my Arduino for libraries and cannot seem to find a reliable method to do so.

System Specifics:

  • I am running Arduino IDE 1.8.5
  • I am using macOS 10.13.6
  • I am running my code on an Arduino Uno connected to an Arduino Leonardo by jumper cables. The Uno sends a few integers over a Serial connection. The Uno gets this signal and displays it on a DFRobot LCD Keypad Shield.
  • I am running at 460800 baud. I can successfully accomplish this reliably. The issue is when I try to call a method from within the alterations of the HardwareSerial.cpp file. If the alteration is a new method, the compiler does not recognize it. If the method is simply changed, the compiler does not recognize alternate method arguments.

Thank you in advance for your help. I feel like I can figure out how to get the 9-bit Serial transfer pretty easily if I can actually edit the Arduino hardware serial library.

Thanks!

You might have multiple copies of the Arduino AVR Boards hardware package on your computer. You need to be sure you're editing the one

The easiest way to find the active hardware package location is as follows:

  • Select a board from the hardware package from the Tools > Board menu
  • File > Examples > SPI > BarometricPressureSensor
  • Sketch > Show Sketch Folder
  • Move up folder levels until you reach the one that contains boards.txt. The core library will be under the cores subfolder

http://forum.arduino.cc/index.php?topic=520248.msg3546389

Thank you both for such quick replies!

Based on what you said, Pert, I checked to make sure I was editing the right packages and I was in the right place.

I referenced the link that Juraj sent, and it confirmed that I had been following the right method to get to the files.

Any other ideas?

I like to use #error or #warning preprocessor directives as a sanity check to be sure my edits to a file are being seen by the compiler.

Hi @Pert,

I am not sure what either of those directives are, what they do, where to include them, and what the expected results would be from using them. I searched them on Google with little useful result. Could you explain what those are/how I can use them or recommend a resource for me to learn that for myself? Thank you!


I don't think that the libraries are correctly being imported because I changed my Serial.read function to always return the constant 62, and it still returns a value from the RX line, which it should not. That, along with other symptoms, reasonably implies that my libraries are not being updated.

Do I have to restart the IDE or my machine or create a new project when altering those files? I have not done this before and I feel like there must be some step I'm missing.

-Rhodes C.

For #error (and general understanding of preprocessor directives):
http://www.cplusplus.com/doc/tutorial/preprocessor/

#warning is not documented in the above link because it's not in the C++ standard so here's the documentation for it (and #error too):

The names pretty much say it all. #error causes an compilation error. #warning causes a compilation warning.

Be aware that these files contain code that is switched via preprocessor conditionals (#if, #elif, #else). If you put the #error or #warning directives inside of conditionals that are not true then of course they will have no effect.

rhodesconover:
Do I have to restart the IDE or my machine or create a new project when altering those files? I have not done this before and I feel like there must be some step I'm missing.

You should not need to do that. The Arduino IDE does cache the compiled core but it should recompile when the timestamps on any of the files are newer than the cache. The cache will also be cleared whenever you change to a different selection in the Tools > Board menu or restart the Arduino IDE. There is a setting File > Preferences > Aggressively cache compiled core but you shouldn't need to disable that. Do the #error test first off to make sure you're working with the right files before worrying about the cache.

Thank you Pert!

This is a really well written out and detailed response. I took a read through those links and they were certainly helpful.

I did as you suggested and put the directive #error at the very top of the HardwareSerial.cpp library, expecting my Serial line to fail in future uses of the serial port, but it still works as it always as, suggesting that the compiler is not seeing my updates.

It's odd because I'm fairly certain I'm right about the location in the file directory based on previous posts and my own intuition/research, but at this point, I'm not sure anymore.

Thanks!

You know, if you turn on the preference for "verbose" compilation, the log will show you EXACTLY which source files are being compiled...

Thank you @westfw, that was a really good suggestion.

I checked using the verbose compilation mode, and it definitely seems to be compiling the correct file. Specifically, one of the many read outs ends with the file name "HardwareSerial.cpp.o", among others. If I understand correctly, that reasonably implies the system is creating an object file from the HardwareSerial.cpp file (the file I edited) so it seems like the edits should take effect.

After uploading to my board again, it's clear that the edits have not taken effect.

All I've been doing is opening the files to edit and change some lines of code and save them. If I reopen them, the changes are still there. But then I just go back to the Arduino IDE and upload my code to the board. Is there more I need to do? Perhaps I need to find someway to compile my changed file before uploading my sketch? I can't find anything on the Internet to suggest that but it sort of makes sense to me?

Thanks all!

That only means that it compiled a file named HardwareSerial.cpp. You might have multiple files of that name on your computer. You need to make sure that the path of the file you modified matches the path of the file used by the Arduino IDE, not just that the filenames are the same.

Another thing that sometimes tricks people is that Windows hides known file extensions by default. So if you have a file named HardwareSerial.cpp.txt Windows Explorer will show the filename as HardwareSerial.cpp.

Hi Pert,

I took a pretty deep dive through my computer for other versions of the files because I think that's a very valid concern (multiple files). I am using macOS so I searched the entire Arduino package and could only find this one (edited) version of HardwareSerial.cpp. I know that when searching my device, macOS does not show files inside packages to protect the user from themselves.

So I have to go inside an app/package and search within to find the alleged alternate file. I did not find an alternate file in the Arduino package, so I do not believe this is the issue.

Do you have any other ideas or other recommended ways to check if this is the issue?

Thanks again for your help!

Please do this:

  • File > Preferences > Show verbose output during: > compilation (check) > OK
  • Sketch > Verify/Compile
  • After the compilation fails you'll see a button on the right side of the orange bar "Copy error messages". Click that button.
  • Paste the error messages in a reply here USING CODE TAGS (</> button on the toolbar).

I am using macOS so I searched the entire Arduino package

Ahh. But did you search ~/Library/..., which is where recent versions of Arduino stash board packages and updates? You need to look at the whole path name. (for instance, I have a portable installation, so I see:

"/Applications/Arduino-1.8.5.app/Contents/Java/portable/packages/arduino/tools/avr-gcc/5.4.0-atmel3.6.1-arduino2/bin/avr-g++" -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10805 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR   "-I/Applications/Arduino-1.8.5.app/Contents/Java/portable/packages/arduino/hardware/avr/1.6.207/cores/arduino" "-I/Applications/Arduino-1.8.5.app/Contents/Java/portable/packages/arduino/hardware/avr/1.6.207/variants/standard" "[color=teal]/Applications/Arduino-1.8.5.app/Contents/Java/portable/packages/arduino/hardware/avr/1.6.207/[color=black]cores/arduino[/color][/color]/[color=red]HardwareSerial.cpp[/color]" -o "/var/folders/gv/zn3wcml52jq0vvjnd95j8g6h0000gp/T/arduino_build_474806/core/HardwareSerial.cpp.o"

Even though the "original" install would have been [color=teal][color=red][color=teal]/Applications/Arduino-1.8.5.app/Contents/Java/hardware[/color]/arduino/avr/cores/arduino/HardwareSerial.cpp[/color][/color]

@Pert,

Here

Arduino: 1.8.5 (Mac OS X), Board: "Arduino Leonardo"

/Applications/Arduino.app/Contents/Java/arduino-builder -dump-prefs -logger=machine -hardware /Applications/Arduino.app/Contents/Java/hardware -hardware /Users/rhodesconover/Library/Arduino15/packages -tools /Applications/Arduino.app/Contents/Java/tools-builder -tools /Applications/Arduino.app/Contents/Java/hardware/tools/avr -tools /Users/rhodesconover/Library/Arduino15/packages -built-in-libraries /Applications/Arduino.app/Contents/Java/libraries -libraries /Users/rhodesconover/Documents/Arduino/libraries -fqbn=arduino:avr:leonardo -vid-pid=0X2341_0X8036 -ide-version=10805 -build-path /var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916 -warnings=none -build-cache /var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_cache_886334 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.avrdude.path=/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avrdude/6.3.0-arduino9 -prefs=runtime.tools.avr-gcc.path=/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avr-gcc/4.9.2-atmel3.5.4-arduino2 -prefs=runtime.tools.arduinoOTA.path=/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/arduinoOTA/1.1.1 -verbose /Users/rhodesconover/Desktop/Work/Code/Arduino/Bluetooth Classic/Second Era/Speed Testing/UNO-S/UNO-S.ino
/Applications/Arduino.app/Contents/Java/arduino-builder -compile -logger=machine -hardware /Applications/Arduino.app/Contents/Java/hardware -hardware /Users/rhodesconover/Library/Arduino15/packages -tools /Applications/Arduino.app/Contents/Java/tools-builder -tools /Applications/Arduino.app/Contents/Java/hardware/tools/avr -tools /Users/rhodesconover/Library/Arduino15/packages -built-in-libraries /Applications/Arduino.app/Contents/Java/libraries -libraries /Users/rhodesconover/Documents/Arduino/libraries -fqbn=arduino:avr:leonardo -vid-pid=0X2341_0X8036 -ide-version=10805 -build-path /var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916 -warnings=none -build-cache /var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_cache_886334 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.avrdude.path=/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avrdude/6.3.0-arduino9 -prefs=runtime.tools.avr-gcc.path=/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avr-gcc/4.9.2-atmel3.5.4-arduino2 -prefs=runtime.tools.arduinoOTA.path=/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/arduinoOTA/1.1.1 -verbose /Users/rhodesconover/Desktop/Work/Code/Arduino/Bluetooth Classic/Second Era/Speed Testing/UNO-S/UNO-S.ino
Using board 'leonardo' from platform in folder: /Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21
Using core 'arduino' from platform in folder: /Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21
Detecting libraries used...
"/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avr-gcc/4.9.2-atmel3.5.4-arduino2/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics  -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=10805 -DARDUINO_AVR_LEONARDO -DARDUINO_ARCH_AVR  -DUSB_VID=0x2341 -DUSB_PID=0x8036 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="Arduino Leonardo"' "-I/Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/cores/arduino" "-I/Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/variants/leonardo" "/var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916/sketch/UNO-S.ino.cpp" -o "/dev/null"
Generating function prototypes...
"/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avr-gcc/4.9.2-atmel3.5.4-arduino2/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics  -flto -w -x c++ -E -CC -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=10805 -DARDUINO_AVR_LEONARDO -DARDUINO_ARCH_AVR  -DUSB_VID=0x2341 -DUSB_PID=0x8036 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="Arduino Leonardo"' "-I/Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/cores/arduino" "-I/Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/variants/leonardo" "/var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916/sketch/UNO-S.ino.cpp" -o "/var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916/preproc/ctags_target_for_gcc_minus_e.cpp"
"/Applications/Arduino.app/Contents/Java/tools-builder/ctags/5.8-arduino11/ctags" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives "/var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916/preproc/ctags_target_for_gcc_minus_e.cpp"
Compiling sketch...
"/Users/rhodesconover/Library/Arduino15/packages/arduino/tools/avr-gcc/4.9.2-atmel3.5.4-arduino2/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -mmcu=atmega32u4 -DF_CPU=16000000L -DARDUINO=10805 -DARDUINO_AVR_LEONARDO -DARDUINO_ARCH_AVR  -DUSB_VID=0x2341 -DUSB_PID=0x8036 '-DUSB_MANUFACTURER="Unknown"' '-DUSB_PRODUCT="Arduino Leonardo"' "-I/Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/cores/arduino" "-I/Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/variants/leonardo" "/var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916/sketch/UNO-S.ino.cpp" -o "/var/folders/6n/c11_qk6958q305ftkv5g7sc00000gn/T/arduino_build_615916/sketch/UNO-S.ino.cpp.o"
/Users/rhodesconover/Desktop/Work/Code/Arduino/Bluetooth Classic/Second Era/Speed Testing/UNO-S/UNO-S.ino: In function 'void loop()':
UNO-S:14: error: 'class Serial_' has no member named 'write9bit'
     Serial.write9bit(1);
            ^
exit status 1
'class Serial_' has no member named 'write9bit'

Above is the error message I received when trying to call a method "write9bit" from the altered HardwareSerial.cpp file. If I don't attempt to call a new method, but instead, alter a previously present method, it compiles correctly, but does not use the altered method when running the program.

@westfw,

westfw:
Even though the "original" install would have been /Applications/Arduino-1.8.5.app/Contents/Java/hardware[/color]/arduino/avr/cores/arduino/HardwareSerial.cpp

This is the path I've been using to show where my Arduino HardwareSerial library has been. However, I do not have a "portable" folder within my "Java" folder in this path:

westfw:
1.8.5.app/Contents/Java/portable/packages/arduino/hardware/avr/1.6.207/cores/arduinoHardwareSerial.cpp"

What do you mean by "I have a portable installation"? I feel like having a little more context on that might provide me some context and I might be able to do some poking around in my device, with perhaps a little more success.

Thanks again all! I'm new to a lot of this stuff so I really really appreciate all of your help.

From this output, the files you should be editing are at Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/cores/arduino.

rhodesconover:
What do you mean by "I have a portable installation"? I feel like having a little more context on that might provide me some context and I might be able to do some poking around in my device, with perhaps a little more success.

Typically the Arduino IDE stores some files in a folder outside its installation folder (Users/rhodesconover/Library/Arduino15 in your case). In some situations (such as an installation on shared computers at a school, running from a flash drive, or when you want to use multiple isolated versions of the Arduino IDE), that is not desirable. So the Arduino IDE has a "portable mode" feature, where the files are stored in the installation folder instead of externally. This allows you to have a completely self contained installation. You can find details here:

but really this has absolutely nothing to do with your current situation. westfw was only explaining why the path shown in their output was a bit out of the ordinary.

pert:
From this output, the files you should be editing are at Users/rhodesconover/Library/Arduino15/packages/arduino/hardware/avr/1.6.21/cores/arduino.

This is what I had thought, but I could not find that directory. It did not seem to exist at "Library." But after hearing you confirm my thought that it would be there caused me to do some cursory Google searches and found out the macOS completely hides this path from me and I had no idea it had existed. I've since found it and the correct files and I've been able to edit the libraries.

Thank you all for your help!

Is there a way for me to mark this post as solved? I think I've seen other people do that.

Now all I have to do is get my actual project working!

Thanks again!

rhodesconover:
It did not seem to exist at "Library." But after hearing you confirm my thought that it would be there caused me to do some cursory Google searches and found out the macOS completely hides this path from me and I had no idea it had existed.

That's one of the reasons for the process I described in reply #1.

rhodesconover:
I've since found it and the correct files and I've been able to edit the libraries.

Glad to hear it!

rhodesconover:
Is there a way for me to mark this post as solved? I think I've seen other people do that.

That would be helpful. You can find instructions in the "How to use this forum - please read" post:

If you consider the issue solved, please edit your original post (the first one in the thread) and add "[solved]" to the subject line. That helps people know the issue is resolved.

Thank you again for your help! The post has been edited.

For future readers, TLDR:

I couldn't edit the libraries because I was editing the wrong files. They were actually installed inside the hidden Library directory on my Mac. Once I found those files and edited them, all was well!

And why didn't you read the post I linked in replay #2? You could read there "It took me hours and hours of searching to discover that current MAC OS hides the Library folders and even makes them non-searchable - to protect you from yourself. "