Arduino Forum

Development => Suggestions for the Arduino Project => Topic started by: BrendaEM on Apr 03, 2019, 12:49 am

Title: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Apr 03, 2019, 12:49 am
Hi,


I think that in practice, adding a 1-bit with bit-packing as well as an 24-bit variables would be great additions, not only to Arduino but general purpose programming as well. While the user can manually work with each, code readability suffers.

There are a great many single-bit variable uses. The Arduino has both a limited program and variable space, and having a solid and efficient single-bit library support would save on memory. Basically, the user could specify an array of bits, such as:

bits options[120];

And set a bit:
options[5]=1;

Or the user could specify single-bit variables:

bits dog_walked, trash_out, teeth_brushed;

And then set a bit:
dog_walked = 1; // Or true, if you will

Both RGB color and DAC sound often use 24 bits. For many applications, 16-bit integers are too small, while 32-bit numbers are too slow. They could be called Int24, or better-yet, or "unsigned mid" or something like that, referencing "long".

In Arduino's microcosm, it seems like there are almost inexhaustible resources for getting anyone's device going. What I am asking for is a fundamental improvement for the language that will likely benefit all users.

Thank you for your consideration,
-- BrendaEM
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: westfw on Apr 03, 2019, 06:06 am
Arduino uses standard C/C++, and thus is significantly at the mercy of the standards for those languages.
C is ... quite opposed to data types that you can't have a pointer to (preferably with hardware support), so this eliminates the possibility of 1-bit data types.  (There was a brief window where some ARM chips supported pointers to single bits, but no one jumped on that enough for compiler vendors to notice.)   You should be able to find assorted C++ libraries that support bit groups as objects of some sort, and would allow a syntax similar to what you suggested.

On the plus side, avr-gcc has supported 24bit integers for several releases now, and you should be able to "just use" the appropriate data types "__int24" and "__uint24", even in arduino code.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Apr 27, 2019, 05:46 pm
I don't the there is any benefit to hiding behind C's shadow.


Beyond the code efficiency and compactness gain, if 24 bit, and compacted 1bit data types were in the Arduino library the code that uses it would be much more readable.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: AWOL on Apr 27, 2019, 06:25 pm
Quote
if 24 bit, and compacted 1bit data types were in the Arduino library
Maybe you missed the part about 24 bit datatypes already being implemented.
They just need a few extra overloads to things like the print methods to make them easier to use.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on Apr 28, 2019, 04:22 pm
I don't the there is any benefit to hiding behind C's shadow.
It's not a matter of "hiding behind C's shadow". After some minimal preprocessing, the .ino files of Arduino sketches are compiled as C++, using a standard C++ compiler.

Arduino already uses the Byte data type, which is non-standard, but helpful
Non-standard, yes, but it's just a typedef:
https://github.com/arduino/ArduinoCore-avr/blob/1.6.23/cores/arduino/Arduino.h#L126 (https://github.com/arduino/ArduinoCore-avr/blob/1.6.23/cores/arduino/Arduino.h#L126)
Code: [Select]
typedef uint8_t byte;

Arduino doesn't require pointers, which is certainly not standard, but good.
C and C++ don't require pointers either, so that is definitely standard. It's a feature of the language. You can use it if you like.

C does not have many of the hardware specific statements that Arduino uses.
They're just functions from a library. It might not seem like it because you don't need to add an #include directive to the .ino file. That's because it's automatically added during sketch preprocessing.

Arduino's "Setup" functionality is non-standard, but helpful.
It's just a function called from main():
https://github.com/arduino/ArduinoCore-avr/blob/1.6.23/cores/arduino/main.cpp#L43 (https://github.com/arduino/ArduinoCore-avr/blob/1.6.23/cores/arduino/main.cpp#L43)
main() is hidden away in the core library, but really it's just standard C++.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: AWOL on Apr 28, 2019, 08:26 pm
I was never keen on Arduino's choice of "byte" as "uint8_t" - it confuses the Java programmers.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on Apr 29, 2019, 03:39 am
It's interesting because the origin of Wiring's (and thus Arduino's) byte type is the Processing language, which is based on Java and has the same signed byte type as Java. My understanding is that Wiring's goal was to provide a familiar programming experience to Processing users, so the same types and API were used when possible. Considering that, it's very strange that Arduino's byte type is unsigned.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: westfw on Apr 29, 2019, 05:06 am
"Byte" (in various capitalizations) was a very commonly defined type in embedded C programming LONG before wiring or processing.  The various variations were one of the reasons that the int8_t and etc were added to the standard...
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on Apr 29, 2019, 05:19 am
How about using "boolean" instead of bool?
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: AWOL on Apr 29, 2019, 09:32 am
How about using "boolean" instead of bool?
Not keen on that either.
If you're going to have "boolean", why not go the whole hog?
"integer", "character", "floating_point", "structure" etc.  :D
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on Apr 29, 2019, 03:01 pm
Not keen on that either.
Me neither. In fact, I did a big campaign to discourage the use of the boolean type:
https://github.com/arduino/Arduino/issues/6657 (https://github.com/arduino/Arduino/issues/6657)

The reason I brought up boolean is because it seemed like westfw was disagreeing with my theory that Arduino's byte type is due to emulating the Processing language. I feel that Arduino's boolean type is more evidence in favor of my theory, since I doubt that is a long established tradition in embedded C programming.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on May 08, 2019, 08:35 pm
Is it a valid statement that there are no special language accommodations need to be made when designing a programming language for a 8-bit processor that has access to 2KB of RAM?

Would people condemn the idea of wringing the last bit of functionality and efficiency from these micro-controllers?

Other the the effort to add it to the library, which I am sure is substantial, what would be the disadvantages of adding a 1-bit auto bit-packed variable and a 24 bit integer variable to the standard Arduino library?
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: AWOL on May 08, 2019, 08:53 pm
I don't condemn wringing out performance, as long as it is not at the expense of legibility and maintainability.

As has already been mentioned, the compiler already supports 24 bit integer variables, even if the Arduino libraries haven't quite caught up with this advance.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: westfw on May 09, 2019, 02:12 am
Quote
Is it a valid statement that there are no special language accommodations need to be made when designing a programming language for a 8-bit processor that has access to 2KB of RAM?


These days, when space is so tight that you need to think about bit and 24bit variables, you'd be better off just getting a bigger processor.  (ATmega4808: 3x the RAM and 50% more flash than an ATmega328p.  And cheaper, too.  Without even need to look at ARM chips.)
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: CrossRoads on May 09, 2019, 03:45 am
Or a '1284P. 16K SRAM, 128K flash, dual hardware serial ports, 32 IO total, and supportable by the IDE with this core file addition

https://github.com/MCUdude/MCUdude_corefiles (https://github.com/MCUdude/MCUdude_corefiles)
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on May 09, 2019, 11:16 am
First, "Arduino" is not a programming language.  It's more of a "framework" based on standard C/C++
I'm very interested in this topic. When I first started with Arduino I though .ino files were written in Arduino language. As I realized how awesome microcontrollers are and made the decision to dive deeper into this world, I started looking at alternatives to Arduino. I felt that learning a proprietary "toy" language would be a waste of my time in the long run, as I reached the limits of what I could do with the language and decided to move on. Then I read that the Arduino language was C++. That was a big "aha!" moment for me, which made me confident that sticking with Arduino was the right decision.

When I've seen people talk about the Arduino language, it's usually done by beginners or people denigrating the Arduino project. I was aware that somewhere on the Arduino site they talk about the Arduino language, but I thought of that as perhaps just an attempt to make things seem less intimidating to a beginner who might just shut down at mention of C++. My belief was further reinforced by a Hackaday article from Elliot Williams, who always writes excellent articles:
https://hackaday.com/2015/07/28/embed-with-elliot-there-is-no-arduino-language/ (https://hackaday.com/2015/07/28/embed-with-elliot-there-is-no-arduino-language/)

Over the years, I've corrected people talking about the Arduino language here several times. I've thought that the impression that there is an Arduino language and an "Arduino compiler" was harmful to the learning process and led to misconceptions (perhaps like this discussion).

Recently, I submitted a proposal to Arduino about adding some text about C++ to the Arduino Language Reference pages. I was informed by a couple of very knowledgeable Arduino employees, whose opinions I respect, that .ino files are written in Arduino language, not C++:
https://github.com/arduino/reference-en/issues/476#issuecomment-461491446 (https://github.com/arduino/reference-en/issues/476#issuecomment-461491446)

I understand the sketch preprocessing process, but I never thought that it made enough of a difference to qualify the code written in .ino files as a new programming language. This made me realize that I have no clue where the line is on what becomes a new programming language. Although I've seen well thought out arguments about why there is no Arduino language, I've never seen one arguing the contrary. It would be interesting to see something like that written by someone knowledgeable on the subject of programming languages.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: AWOL on May 09, 2019, 11:25 am
I don't think of it as a new language, more as a pidgin or creole - it isn't exactly C++, but one should just about be able to make oneself understood to a native speaker.
(Among the Koorie people from the Victoria outback in Australia, a solar eclipse is "Kerosene lamp b'long Jesus gone bugger-up". source: The Telegraph)

I don't think it qualifies as a dialect.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on May 20, 2019, 08:06 pm
The libraries that allow Arduino to cross-compile and load the code are quite and achievement. Dozens of boards use the Arduino's IDE. Because Arduino's language is unique, it is language. It may mostly be C, but that's not all it is.

My purpose in asking for the data types is not only for efficiency, but for readability. It could be said that manual bit packing is clever, but its code is ugly, and distracting from the larger program. A compiler could easily pack, in this case, 8 bits into a byte.

The problem comes with code trans-portability and readability when moving to a processor with a larger word length. If there was an auto-bitpacked variable, the same code would just be more memory efficient because the library/compiler would know the word length.

If 1-bit auto-bit packed variables as well as 24-bit integers were integrated into Arduino's...vocabulary--would you avoid using them, or would you use them?
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on May 20, 2019, 09:55 pm
Because Arduino's language is unique, it is language.
I wrote a library with some unique functions and typedefs. Did I create a new programming language?

It may mostly be C
There's more C++ than C.

If 1-bit auto-bit packed variables as well as 24-bit integers were integrated into Arduino's...vocabulary--would you avoid using them, or would you use them?
First of all, it's already been explained to you that we have 24 bit variables. Use them if you like. Case closed on that.

As for 1 bit variables, please provide a demonstration of how this should be implemented in Arduino's core library. Remember that we are using a standard C++ compiler. Arduino is not in the business of writing compilers (nor would you want them to be).
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: westfw on May 21, 2019, 02:27 am
"bool" or "boolean" could be a single bit, except for C's desire to have basic types be pointable-to (and probably promotion issues, too.)   Then it would just work the same way that "bool" works now.
I'm not sure which Arduino library functions would actually need to behave differently for single-bit values, or which ones would benefit from being able to (what good is having a 1-bit variable if it takes a full register to pass to subroutines, and/or extra instructions to extract from RAM/etc?)

It's sort of a shame that none of the big-name compiler/;anguage gurus jumped on ARM's "bit-banding" feature as a way to provide full-featured 1-bit types.  (It seems it's been dropped from the ARM due to lack of interest.)  OTOH, the lack of 1-bit variables in any standard computer language that I know of probably means that it has seven more "problems" than the ones I've mentioned.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Jul 13, 2019, 10:29 pm
The problem with Booleans is that they very-likely have no automatic bit-packing, so each Boolean likely takes an entire byte, not counting the memory pointer.

I am writing bit-packing into my application, but the result will be sloppy, in that the user will be exposed to the Byte/Bit offsets.

I thought that in a manner vaguely similar to how const's are handled that the IDE could behind-the-scenes, create maintain a table for bytes and bits used, more efficiently and more readable than a person could, because I thought this would be more readable:

bits subscribed, paid up;
if ( subscribed == 1)
{
// Blink Kiosk Light
}

// Or perhaps...

bits quadcopter_area_collision_map [12, 12, 8];

// Or perhaps...

bits eepromblockflags[16];

While single-bit auto-bit-packed variables have storage advantages--especially in chips that have only 2k or 4k of RAM.

I think that the user's program will read better by not exposing the reader to saving techniques, in the IDE. This kind of technique can efficiently be done at compile time--especially in a Harvard architecture type micro-controller.

If there are Int24s available, then why not support them? Why not document them?

All the colors on the screen you are seeing are represented by 24 bits, one each of: red, green, and blue.

Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on Jul 13, 2019, 10:49 pm
this kind of technique can efficiently be done at compile time
You keep saying this, yet you refuse to explain how it can be done.


If there are Int24s available, then why not support them?
What do you mean by "supported"? You can use them; what more support do you want?
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Jul 14, 2019, 12:19 am
Pert, where exactly did I refuse anything in this thread? You may not know this, but that's likely slander.

I am registered and have used some 82 other forums. Except for trying to share things with people, file bug reports, and offer suggestions, I don't like coming using the Arduino forums. While people may offer differing opinions or methods to accomplish programming goals, I feel that there is an undercurrent here of unchecked cruelness to new would-be programmers. I hope that over time that will change because young, impressionable people also visit these forums--and they don't need that. Anyway....


When and if someone chooses to use a single 1-bit variable, on compile-time, the IDE could initialize a table of: variable name, byte, and bit, likely as pointers, but for the course idea is:

variablename, byte, bit
myvar, 0, 0

In the above example, the byte position would be a relative offset starting address from where the compilers would keeping the bit variables.

The compiler would deal out memory in 8-bit or 16-bit int chunks, whichever is more efficient. A maximum byte-bit tally can be kept, so the next table entry lines up to the last, in a similar manner to a filesystem freespace allocation map. Because it's done at compile time, in a static manner fragmentation is not an issue.

Using a single-bit variable would cost in program memory, in the form of a library loaded behind the scenes. In my experience Arduinos have more practical space issues with RAM than program memory.

The loaded library would include small, efficient compact routines to translate the changes to the variable, And'ing or Or'ing them for the user.

In the case of an bit array, the table could be extended to use a bit length, likely confined to a particular size. The table entry would look could this:

variablename, byte, bit, length
myvar, 0, 0, 32, 16

Without the compile-time bit locations, the user would either have to use global variables for it's housekeeping, (which doesn't personally bother me, because I am working on my projects alone, and I enjoy organizing namespaces and structures) or they need to lean toward creating an object-oriented approach, which I feel is too abstracted for efficient micro-controller programming.


Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: pert on Jul 14, 2019, 12:38 am
Pert, where exactly did I refuse anything in this thread?
I asked you two months ago:
As for 1 bit variables, please provide a demonstration of how this should be implemented in Arduino's core library. Remember that we are using a standard C++ compiler. Arduino is not in the business of writing compilers (nor would you want them to be).
I interpret no response to that request after such a long time to be a refusal. If you think that's slander, go right ahead and sue me.

I'm glad you're now providing some details of your proposal, but I still don't understand how it would work. You mention a library, which is great, but I don't see how 1 bit variables can be accomplished by a library alone. Certainly we can use a function or macro to read and write bits of a variable, and Arduino already provides this via the bitRead() and bitWrite() functions. But I don't think that's what you're talking about.

I reject any proposal that has Arduino creating their own compiler, or doing some crazy sketch preprocessing in the Arduino IDE like what they do with function prototypes. If you want to advocate that C++ compiler engineers add a 1 bit type to the existing compilers like GCC, great, but it's very unlikely any of them hang out in the Arduino Forum so I take this as being a proposal directed at the Arduino developers or the Arduino community.

I completely understand why you want a 1 bit type, and I'm very much in support of it if it's possible to do in a feasible, portable, and user-friendly manner.
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Jul 14, 2019, 04:05 am
[Pert, I am not expecting your to be a courteous, but it's bad form to write that someone stated something--when they did not.]

Oddly, I am not a big fan of C++ because I feel that burns memory too fast and easily for micro-controller work.

While 1-bit auto-bitpacked variables more complicated than the const's, it wouldn't be rocket science. If you could see why I would want them, then they aren't so crazy after all.

[For a data transmission project, I already have a single-bit flag function that takes an array of bytes that will use global named constants as bit-pointers. For now I am just using enums, but they are costly, being ints. I will be adding another function and another byte array and another bitmap for another series of flags. Though, it occurs to me, that this would be useful if it were implemented in the language, in a generic sense. For my purposes, I can write something a little more generic, though, users don't need to see the mechanisms, and they may want to declare these positions locally.]

It would seem that single-bit variables would carry with them a complex pointer, a pointer that points to a byte and a bit.

Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: bobcousins on Jul 14, 2019, 10:01 am
Single-bit variables are never going to happen, so you may as well forget about that.

In the rare event you have a lot of bools and want to save space, it is trivial to write a user library to deal with them as a packed array.

Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Jul 14, 2019, 08:36 pm
It's not about sending data through a library, it's about the final readability of the code.

So, if a single-bit bit-packed variable was available in Arduino, you wouldn't use them?
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: westfw on Jul 14, 2019, 11:32 pm
Quote
if a single-bit bit-packed variable was available in Arduino, you wouldn't use them?
You know, probably not.  They cases where I need a bunch of individual bits in quantity enough to impact memory usage (or program readability) are essentially non-existent.
Arrays of bits, sure.  Single bit values as a part of some higher-level structure - those too.  But those are all things that are likely to be abstracted in better ways than having a single-bit basic data type and treating them like I would in a ForTran program...
(These conversations are difficult if you only drive by every couple months to participate...)
Title: Re: 1-Bit and 24-Bit Integer Variables, Please
Post by: BrendaEM on Jul 16, 2019, 09:36 pm
There is another way to look at the one-bit variable. Arduino's Booleans are stored inefficiently, and in an inflexible manner.

I am curious why the 24-bit variables aren't included in the documentation.

In all the forums that I have been to, I have never had anyone complain about my timing, much less two, people in the same thread. Very interesting. I am not having difficulty with my timing.