Bit-banging and bit stuffing: an easy approach

Projects Libraries
1

A question that pops up quite regularly is how to isolate bits in a word or byte to set or read. I think most novices struggle with the enigmatic "myVar & (1 << 4)" notation. The standard functions and libraries are at a loss beyond the highByte() and lowByte() macros.

Long story short, here's a deceptively simple union (a cousin of a struct) that help one to bit-bang, nybble, byte, and int your way through the same data.

I've included detailed instructions in the header file and a complete example that runs well on my Leonardo.

I hope you have some fun with it.

Some uses:

  • Extracting high- and low-byte from an int
  • Shifting four-bit quadrature nibbles for rotation measurement
  • Setting up registers
  • Sixteen boolean flags for your program all in one place
  • I hope you'll give us some more examples!

Extract the .zip file in your \Arduino\libraries\ folder.

Credit where credit is due: This blog entry helped a lot with some of the details while I was starting out with creating the union. The version I've shared had some pretty complicated and inefficient predecessors!
http://www.utopiamechanicus.com/article/data-splitting-union-and-struct-c/

exploder_h.zip (4.17 KB)

2

I like it because of its simplicity, and the compiler will do the access code & masking etc.

it is a pity that one cannot make an array of 4 nybbles (or ?)

Dit you also consider the 565 structs for images (16 bits, used by several small color LCD's)

    struct {
        unsigned char R:5;
        unsigned char G:6;
        unsigned char B:5;
    } fiveSixFive;

but maybe that should be another lib?

And missing too is the 32 bit version?

3

I thought about a 32-bit version, but decided I don't truly need it. If there's some interest, I'll create one.
Unfortunately, bit fields don't support arrays or I would be all over them! I tried very hard to get a set one could iterate over but alas, not possible.

I didn't even think of the LCD, that's another application of the principle. Good one!

4

If you would write a class you could overload the [] operator.

I might give it a try - proof of concept.

I'll be back :slight_smile:

5

check this class

//
//    FILE: bithelper.ino
//  AUTHOR: Rob Tillaart
// VERSION: 0.1.00
// PURPOSE: demo [] operator
//    DATE: 2015-11-07
//     URL: http://forum.arduino.cc/index.php?topic=357709
//
// Released to the public domain
//
uint32_t start;
uint32_t stop;

class bithelper
{
  public:
    bithelper(long val)
    {
      _value = val;
    };
    byte operator[] (byte idx)
    {
      return (_value >> idx) & 0x01;
    };
    byte bitcount()
    {
      long v = _value;
      byte bc = 0;
      while (v > 0)
      {
        bc += v & 1;
        v >>= 1;
      }
      return bc;
    };
    byte nybble(byte idx)
    {
      return (_value >> (idx * 4)) & 0x0F;
    };

  private:
    long _value;
};


void setup()
{
  Serial.begin(115200);
  Serial.print("Start ");
  Serial.println(__FILE__);
  Serial.println();

  bithelper bh(1234567890L);

  Serial.println("\nbit test");
  for (byte i = 32; i-- != 0;)
  {
    Serial.print(bh[i]);
  }
  Serial.println();

  Serial.println("\nbitcount test");
  int bitcount = 0;
  for (byte i = 32; i-- != 0;)
  {
    bitcount += bh[i];
  }
  Serial.println(bitcount);
  Serial.println(bh.bitcount());
  Serial.println();

  Serial.println("\nnybble test");
  long val = 0;
  for (byte i = 8; i-- != 0;)
  {
    byte n = bh.nybble(i);
    Serial.println(n);
    val *= 16;
    val += n;
  }
  Serial.println(val);
  Serial.println();

  bithelper bh2(12345L);
  Serial.println("\nprintHex test");
  for (byte i = 8; i-- != 0;)
  {
    byte n = bh2.nybble(i);
    Serial.print(n, HEX);
  }
  Serial.println();
  Serial.println(12345L, HEX);
  Serial.println();

  Serial.println("\ndone...");
}

void loop()
{
}
6

If you want to read and write bits easily, I have my BitBool library in the library manager, or here: GitHub - Chris--A/BitBool: The ultimate C++ bit manipulation tool

Its a drop in replacement for a bool array (read and write using array notation), and very simple.

BitBool<16> bits = { B00001111, B10101010 };

//...

bool val = bits[11]; //Read

bits[4] = true; //Write;
7

Here is the Cosa variant of this; BitSet, Cosa/BitSet.hh at master · mikaelpatel/Cosa · GitHub
Below is a snippet from the example sketch; Cosa/CosaBitSet.ino at master · mikaelpatel/Cosa · GitHub

enum {
  RED,
  GREEN,
  BLUE,
  YELLOW,
  MAGENTA,
  CYAN
};

void setup()
{
  BitSet<68> a, b;
  ...

  // Check that the bitset is empty
  ASSERT(b.is_empty());
  ASSERT(a == b);

  // Add and remove elements
  ASSERT(!b[GREEN]);
  b += GREEN;
  ASSERT(b[GREEN]);
  b -= GREEN;
  ASSERT(!b[GREEN]);
  b += RED;
  b += CYAN;
  ASSERT(b[RED] && b[CYAN]);
  trace << PSTR("B0:") << b << endl;

  // Assign and check that they are equal
  a = b;
  ASSERT(a == b);
  trace << PSTR("A0:") << a << endl;

  // Add element and another bitset
  a += a.members() / 2;
  trace << PSTR("A1:") << a << endl;

  // Try adding element outside the bitset
  b += 67;
  b += 68;
  b += 100;
  trace << PSTR("B1:") << b << endl;
  ASSERT(b[RED] && b[CYAN] && b[67]);
  ASSERT(!b[68] && !b[100]);

  // Remove bitset and the last element
  b -= a;
  b -= 67;
  trace << PSTR("B2:") << b << endl;
  ASSERT(b.is_empty());
  for (uint16_t i = 4; i < b.members(); i += 5)
    b += i;
  trace << PSTR("B3:") << b << endl;
}

The Cosa common types also includes a universal type for general type mapping (univ16_t and univ32_t). Same principle as the initial post in this thread: Cosa/Types.h at master · mikaelpatel/Cosa · GitHub

/**
 * Universal type union, 16-bit.
 */
union univ16_t {
  uint16_t as_uint16;
  int16_t as_int16;
  uint8_t as_uint8[2];
  int8_t as_int8[2];
  const void* as_ptr_P;
  void* as_ptr;
  struct {
    uint8_t low;
    uint8_t high;
  };
};
typedef univ16_t univ_t;

/**
 * Universal type union, 32-bit.
 */
union univ32_t {
  float32_t as_float32;
  uint32_t as_uint32;
  int32_t as_int32;
  univ16_t as_univ16[2];
  uint16_t as_uint16[2];
  int16_t as_int16[2];
  uint8_t as_uint8[4];
  int8_t as_int8[4];
  const void* as_ptr_P[2];
  void* as_ptr[2];
  struct {
    uint16_t low;
    uint16_t high;
  };
};

Cheers!

8

all those codes seem too complex and slow.
I prefer the classic version

//set
v |= 1 << nbit;
//unset
v &=!( 1<< nbit);
//revert
v ^= 1 << nbit;

but if Nbit is a variable then I prefer to use the masks.

9

So many solutions to the same problem!
I like Rob's class approach, but the memory overhead is too much.
The classic C shift approach is good, but not as flexible as the union method for converting from bit patterns to other forms.

10

vbextreme:
all those codes seem too complex and slow.
I prefer the classic version

//set

v |= 1 << nbit;
//unset
v &=!( 1<< nbit);
//revert
v ^= 1 << nbit;



but if Nbit is a variable then I prefer to use the masks.

Luckily this is your assumption. If you tried my class, you'd actually be surprised (did you look at the code?, it just does shifts...).
Then the working function does this:

if( copy ) data |= index;
else  data &= ~index;

Notice the similarities.

11

vbextreme:
all those codes seem too complex and slow.
I prefer the classic version

//set

v |= 1 << nbit;
//unset
v &=!( 1<< nbit);
//revert
v ^= 1 << nbit;



but if Nbit is a variable then I prefer to use the masks.

I wonder if this one works as intended

v &=!( 1<< nbit);

the not operator makes everything not zero -> zero

12

quick test

void setup() 
{
  Serial.begin(115200);
  Serial.print("Start ");
  Serial.println(__FILE__);

  byte x = 0xFF;
  Serial.println(x, HEX);
  x &= !(1 << 3);
  Serial.println(x, HEX);
}

void loop() 
{
}

output:

Start sketch_nov08b.ino
FF
0

QED

13

sorry, I meant ~ not !, keystroke error.

v &=~(1 << nbit); //unset

@Mike:
but not as flexible as the union method for converting from bit patterns to other forms

you can convert without union. is so simple.

@pyro,too complicated , too expensive for a simple set / unset bit that is the ABC of programming .

14

vbextreme:
too complicated , too expensive...

That's why I mentioned, its obvious you either haven't tried it or do not understand it. Claiming its expensive shows this. As for complicated, most people are comfortable with using arrays.

Also large applications where tens or hundreds of bits are being manipulated, the code looks far nicer without loads of shifts littering the code. There are reasons why people wrote the _BV, bitSet, bitRead, etc... macros. My class is just an extension on these.

And you don't have the chance to make silly mistakes... like confusing '!' with '~'.

15

expensive, you check every time if is copy or not, this cost two clock for check, simple example set 8bit with your code increas as minimum 16 clock.
but I'm sure if we put to better analyze the code we can find even more problems.
More people think that the Arduino is simple, but they forget that the C / C ++ are among the more complex languages.

_BV or CBI,SBI or another Macro is equal to my code and not is expensive or complicate.

confusing, I said yesterday about fifty threads using the phone, I think a typo can happen.
I'm not a super hero.

16

vbextreme:
expensive, you check every time if is copy or not

What do you think happens when the value being assigned is a constant?

There is more to this than meets the eye... :slight_smile:

this cost two clock for check, simple example set 8bit with your code increas as minimum 16 clock. I'm sure if we put to better analyze the code we can find even more problems.

and if we stopped assuming things and actually tested it... Well, that's an entirely different bucket of fish altogether. Trying to calculate a high level language as exact clock cycles is quite a naive way of approaching new code.

17

you have dissasemble code?

I remain of the view that it is an unnecessary complication.

#define set |=
#define unset &=~
#define reverse ^=
#define isset &
#define bit_op(VAR,OP,BIT) ((VAR) OP (1 << (BIT)))


void setup()
{
    uint8_t like = 0;

    bit_op(like, set, 0);
    bit_op(like, reverse, 7);

    if ( bit_op(like, isset, 0) && bit_op(like, isset,7) && !bit_op(like, isset, 3) )
    {
        Serial.print("@_@");
    }
    
  
}

Portable, C/C++, easy.

translate this with your class :

void strtolow(char* s)
{
    while( *s ) 
      bit_op(*s++, set, 5);
}

but I like

void strtolow(char* s)
{
    while( *s ) 
      *s++ |= 0x20;
}
18

Sorry I overlooked your reply, but I'm happy to provide what you requested.

vbextreme:
you have dissasemble code?

I remain of the view that it is an unnecessary complication.

You can get this yourself, right?

A novel idea vs ridiculous defines... hmm I do not see how using something which is an extremely common feature (arrays) can be more complicated than your custom set of defines. Many newbies have used my library without much effort at all.

Portable; not really considering that the identifiers 'set', 'unset', 'reverse', and possibly 'isset' are far too useful names to remove from not only the global namespace, but everywhere. Your code becomes a cause of confusing errors if used... anywhere else.

Your request to re-create your function is no problem. Yeah my example may not be as simple as your second version, but you need to realize I posted the option of my class in response to the OP's intention of manipulating many bits. This is where it becomes very useful.

The key to successful coding is using tools appropriate for the job. Using a tool for handling many bits in a piece of code that only needs one is possibly an overkill... however here it is:

#include <BitBool.h>
typedef BitBool<8>* bptr;

void strtolow(char* s)
{
  bptr b = (bptr)s;
  while( *b->data ) (*b++)[5] = HIGH;
}

And guess which version compiles larger (cos that's all you're doing... guessing).

Now, if used to recreate any example slightly more involved, you'll see the benefits, lets take your other example for instance:

void setup()
{
    BitBool<8> like = {};

    like[0] = true;
    like[7].invert();

    if ( like[0] && like[7] && !like[3] )
    {
        Serial.print("@_@");
    }
}

As anyone can see, it is far less involved than using a selection of obscure defines. And it is 100% pure portable C++.

19

pYro_65:
Sorry I overlooked your reply, but I'm happy to provide what you requested.

no problem , the world will not end for that

pYro_65:
A novel idea vs ridiculous defines...

that ridiculous definition derives from a macro widely used.

From: Karl Heuer, Interactive System

#define strrel(a,R,b) (strcmp(a,b) R 0)

and use

char str[...];
char str2[...];
if ( strrel(str, ==, str2) )
...

and my code is simple porting of that idea.
but if you don't like a name can change, it's very simple to change...

pYro_65:
And guess which version compiles larger (cos that's all you're doing... guessing).

Tested both versions.
identical, memory and speed.
but I do not know if in more complex cases the compiler may fail in optimizing your class.

pYro_65:
As anyone can see, it is far less involved than using a selection of obscure defines

as I said I do not use this macro , I find simply useless and hard to read the code format other than canonical.
I think it would be studied more useful 3 bitwise operators that crazy behind yet another class that tries to hide complexity of C/C++
At this rate we will end up with code like this

Number a;
    a.equal(1);
Number b;
    b.equal(2);

a.sum(b);

or with my example

int a,b;
a equal 1
b equal 2
a sum equal b

ORRIBLE!

I like this

int a,b;
a = 1;
b = 2;
a += b;

classic, simple.
the novices have to tell him to study , not to give illusions.

20

behind yet another class that tries to hide complexity of C/C++

Seriously, I do not think anyone uses this to justify any piece of code they write. Using a library to do something repetitive in a shorter/smarter way is in no way preventing you from using C++ properly. You could argue the same point for the whole Arduino API.

In real world situations using libraries of functionality is a good thing! By having discrete functionality encapsulated away from the code where everything comes together as a solution (sketch) makes the code shorter, more concise, and has a single distinct point where to look for errors. Good libraries used appropriately provide shorter development times than recreating the wheel (or unnecessary repetition).

but I do not know if in more complex cases the compiler may fail in optimizing your class.

The way the code is generated is a result of the class design, not just the compiler. It only supports reading, setting & inverting bits; there is no room for anything to become inefficient. Not only that, it is designed to be used very effectively in complex situations:

#include <BitBool.h>

union Flags{
  unsigned long value;
  BitBool<32> bits;
} flags = { 0 };

[b]//Might only need to work with discrete bits: so create a variables referencing the ones you want!.[/b]
auto sigBit = flags.bits[4];

void setup() {

  Serial.begin(9600);
  Serial.println( flags.value ); //Original val

  //Write referenced bit high
  sigBit = true;

  Serial.println( flags.value ); //After setting bit

  [b]//Create a non sequential collection of bits.[/b]
  decltype(sigBit) randomBits[] = { flags.bits[0], flags.bits[3], flags.bits[7], flags.bits[30] };

  [b]//Easily traverse the collection.[/b]
  for( auto bit : randomBits ){
    bit.invert();
  }

  Serial.println( flags.value ); //After modifying collection
}

void loop(){}

There is more to the lib than replicating a bitfield :wink: