Help needed Handling Data in odd sized words (not 8 bit or 16 bit etc)

Hi

I am workingh with a RAM that is connected on SPI and has unusual size words. The actual word size varies dependant on the page of RAM I am loading

I know how to address the RAM, it is addressed one word at a time then I need to send data as a sequence of bytes together with some padding bits at the end

The RAM is big endian

What I am not sure is how to handle the variables in my program as they cross byte boundaries etc and I have no previous experience of this

Here is an example of what Ineed to do

Object RAM
Depth 64 Words
Word Size 49 bits
Word Size 7 bytes
Padding Bits 7

Description of single element
Bits 0:11 H Size
Bits 12:21 V Size
Bits 22:26 Priority
Bits 27:38 H Pos
Bits 38:48 V Pos

As you can see the parameters i need to deal with have no respect for byte or word boundaries lol

What I need is to create function that will fill in a structure that I then send on SPI as 7 bytes

I do also have some example code like this

#define WORD_SIZE 7 // word size expressed in bytes

typedef struct {
    union
    {
//walc327        __packed
        struct {
            unsigned int pad1:7;
            unsigned int ypos:10;                   // vertical position
            unsigned int xpos:12;                   // horizontal position
            unsigned int priority:5;                // priority (0=lowest, 31=highest)
            unsigned int height:10;                 // vertical size
            unsigned int width:12;                  // horizontal size
        } __attribute__((packed));

        UINT8 raw[WORD_SIZE];
    };
//walc404 } Fillbox_s;
} __attribute__((packed)) Fillbox_s;

That is fairly obviously the same structure in a way (though it seems to declare everyting as unsigned int (16 bits?) I guess this attribute packed is doing something.?

I can't compile that unless i change UINT8 to unsigned char then it will compile but is that the same thing?

I need a function that I can pass the height width priority vpos and xpos as integers, which the function then uses to fill in this structure correctly

I then need to send the 49 bit structure as 7 bytes (MSB first) using SPI

Can one of you more experience C programmers point me in the right direction

cheers
Rich

Today's microprocessors will give you 8-bit, 16-bit, 32-bit, and sometimes 64/128/256-bit quantities. You will occasionally find places where a non power of 2 number of 8-bit bytes are used, such as the 80-bit floating point format used by the 80387, and the 40-bit multiply/add accumulators found on some microprocessors. No mainstream chip will give you 7/49-bit values.

What you need to do is write your data structure as you normally would with 8-bit bytes, and then when you are writing the bits for SPI transmission, you will decode the structure using shifting/masking to get each bit as needed by the target. Since your target is big endian, you will also have to manually swap bytes. When you are reading from SPI, you do the reverse, read a bit at a time, and manually build up the structure with ORs.

Another block of the same devices memory is 27bit

Again some parts of the data straddle byte boundaries

All the other blocks of memory seem easier to deal with, though they may be 80 or 120 bit wide at least the data structure stays within byte boundaries

The Union I posted was from a C++ class that is supposed to help program this device

What does this mean with the :n after the integer name?

unsigned int pad1:7;
unsigned int ypos:10;

It looks to me like it is saying

unsigned int pad : 7 (bits)
unsigned int ypos : 10 (bits)

As though it is setting the bit size for each item in the union. Certainly the numbers after the colon on the integer declaration match the bit sizes for the parameters in the devices RAM

Is that correct. I did a bit of a google on C Unions and It says they are areas of RAM a bit like structures but can hold different sized data at different times (so you could store a 32 bit word in the union - but the load a char - which would overwrite the first byte of the 32bit value)

Sorry I don't have enough direct knowledge to fully understand this, I'm guessing based on what I can find to read. I couldn't find another example like this with the colons in the int definitions

I take on board what you say about having to bit shift (not nice) but what are those numbers after the colons on the int declarations in the Union doing if they are not packing the data? Certainly the above Union will compile OK

All I can find about attribute((packed)) is that it ensures that structure fields align on one-byte boundaries.

Rich

dicky96:
Another block of the same devices memory is 27bit

Again some parts of the data straddle byte boundaries

All the other blocks of memory seem easier to deal with, though they may be 80 or 120 bit wide at least the data structure stays within byte boundaries

The Union I posted was from a C++ class that is supposed to help program this device

It can lay out the bits, though in doing so, it will slow down accesses to those fields at runtime, since the compiler has to generate code to load one or more bytes, do appropriate shifting and masking to isolate the field. All of the common microprocessors can only deal with things that are multiple of 8-bit bytes. So all compilers will round up structures to be a multiple of 8-bits, and if you have any fields that aren't char, unsigned char, signed char (or aliases like uint8_t), the compiler will align it up to 16/32/64-bit boundaries, depending on the types within the structure, and the machine the compiler is generating code for.

When you are dealing with structures locally, they will always be larger than this weird machine you are communicating to. Thus when you are transferring stuff to/from this machine, you have to write a converter routine that takes a structure that the compiler lays out on your machine, and translates it bit-by-bit to send to the remote machine, and do the opposite when reading it back. You cannot use the more normal interfaces that send bytes. Instead you have to get your hands dirty and deal with raw bits.

If you have control of the remote target, perhaps it might be simpler to find something else that is easier to program that uses 8-bit bytes and is little endian. Or you will need to spend a lot of time coming up to speed to learn enough about low level programming to deal with this difficult target.

dicky96:
What does this mean with the :n after the integer name?

unsigned int pad1:7;
unsigned int ypos:10;

It looks to me like it is saying

unsigned int pad : 7 (bits)
unsigned int ypos : 10 (bits)

Yes, but as I said, the compiler is adding padding to the structure to make it fit into bytes on the machine. Depending on the machine you are generating code for, the compiler may have padding between the pad and ypos fields.

dicky96:
As though it is setting the bit size for each item in the union. Certainly the numbers after the colon on the integer declaration match the bit sizes for the parameters in the devices RAM

Is that correct. I did a bit of a google on C Unions and It says they are areas of RAM a bit like structures but can hold different sized data at different times (so you could store a 32 bit word in the union - but the load a char - which would overwrite the first byte of the 32bit value)

Again, most processors can only deal in units of 8-bit bytes, and not arbitrary groups of bits. There is no instruction to load say 7 bits. Instead what the compiler has to do in the worst case is load two adjacent bytes into registers, and use two shifts, two ands, and an or instruction to give you the 7 bit value.

Would using C's bit fields inside structures be an aid?

retrolefty:
Would using C’s bit fields inside structures be an aid?

http://www.cs.cf.ac.uk/Dave/C/node13.html

That’s what the OP is doing. However, the compiler is padding out the structure to be at least a multiple of 8-bits, possibly 16-bits.

Michael
Thaks for the answer, I appreciate you taking time to help me

I'm fine with getting my hands dirty and shifting bits / performing boolean ANDs and ORs to get seven bytes storing what I want to send via SPI, I just hoped I wouldn't have to. If Iwas programming this in assembler I would be shifting bits, carrying, masking and ORing etc, I just hoped the compiler would know what I wanted to do by those numbers after the colon and generate the code for me to save me the hassle lol :smiley:

And if I may add if I was programming in assmebler I would also know EXACTLY what data sizes I was working with. This is a criticism I have with compilers, they mask what is going on so sometimes you aren't sure of what you are actually doing :stuck_out_tongue:

From what you say these bit sizes in teh structure appear to be pretty useless (well at least in this case)

OK a few quickies to do with syntax to make sure I know the size of the datatypes i am gonna be bit shifting etc and then I'll get on with it

is a UINT8 same thing as a unsigned char?
is a UINT16 same thing as an int?
Is an int same size on Arduino Uno and Arduino Mega 2560?

Arduino IDE don't seem to like UINT data types

I need to make sure I am handling 8 and 16 bit data types before I start shifiting bits

Also when calling a function (or more specifically a method in a C++ Class if it is done like this example

loadFillBox(UINT8 word, Fillbox_s& fillBox)

What is the '&' for after Fillbox_s ?
Fillbox_s is a structure of a few UINT16s and a UINT8

Is that sort of like a pointer as in char* or some other sort of indirection? I have c++ example CODE that will help me with the bit shifitng but the method is called with that '&' on one of the arguments. I can't copmpile the C++ Class as there are includes missing, so I want to be sure I implement the C++ method as a C funtion correctly

cheers
Rich

dicky96:
Michael
Thaks for the answer, I appreciate you taking time to help me

I'm fine with getting my hands dirty and shifting bits / performing boolean ANDs and ORs to get seven bytes storing what I want to send via SPI, I just hoped I wouldn't have to. If Iwas programming this in assembler I would be shifting bits, carrying, masking and ORing etc, I just hoped the compiler would know what I wanted to do by those numbers after the colon and generate the code for me to save me the hassle lol :smiley:

From what you say they appear to be pretty useless (well at least in this case)

OK a few quickies to do with syntax to make sure I know the size of the datatypes i am gonna be bit shifting etc and then I'll get on with it

is a UINT8 same thing as a unsigned char?
is a UINT16 same thing as an int?
Is an int same size on Arduino Uno and Arduino Mega 2560?

Arduino IDE don't seem to like UINT data types

The types are uint8_t, uint16_t, etc. These are not standard C++ types, but defined in one of the headers that the IDE includes for you automatically. If you are making a library, you should include Arduino.h to get these definitions.

uint8_t is an alias for unsigned char.

uint16_t is an alias for unsigned short. It might be unsigned int on AVR processors, but not on ARM processors like the DUE.

On any Arduino compatible that uses the AVR chipset such as Uno, Mega2560, etc. the size of short and int is 16 bits. On processors that use the ARM chip, such as the Due, Teensy 3.0, Mbed, etc. the size of int is 32 bits while the size of short is 16 bits.

dicky96:
I need to make sure I am handling 8 and 16 bit data types before I start shifiting bits

As I said, the compiler will do the appropriate shifting, etc. behind your back for bit fields, it is just it needs to pad out the structure to be a multiple of 8/16/32-bits. So you will never get a 49-bit data item. However note, based on whatever software convention the compiler is targeting, it may not allow int fields to straddle boundaries, and so you might get extra padding between fields. So in this case, it is probably better if you do it explicitly.

dicky96:
Also when calling a function (or more specifically a method in a C++ Class if it is done like this example

loadFillBox(UINT8 word, Fillbox_s& fillBox)

What is the '&' for after Fillbox_s ?
Fillbox_s is a structure of a few UINT16s and a UINT8

Is that sort of like a pointer as in char* or some other sort of indirection? I have c++ example CODE that will help me with the bit shifitng but the method is called with that '&' on one of the arguments. I can't copmpile the C++ Class as there are includes missing, so I want to be sure I implement the C++ method as a C funtion correctly

Yes, the & says to pass the value by reference, which allows you to modify parameters passed to the function without doing explicit dereferences. This is not available in C.

So in C++ if you write:

void loadFillBox(uint8_t word, Fillbox_s& fillBox)
{
  fillBox.foo = word;
}

// ...

void doit (void)
{
  Fillbox_t box;
  loadFillBox (5, box);
}

You would need to do in C:

void loadFillBox(uint8_t word, Fillbox_s *p_fillBox)
{
  p_fillBox->foo = word;
}

/* ... */

void doit (void)
{
  Fillbox_t box;
  loadFillBox (5, &box);
}

OK Michael

I maybe didn’t quite describe this right - I am not actually dealing with a 49bit wide RAM it is 56bit wide at this RAM page, it’s just that the last 7 bits are padding, the relevant data is 49 bits. The data is passed on SPI as 7 bytes (56 bits)

Now I’ve tried really hard but I just don’t get this pointer thing :~ :~ :~

The bit shifting seems OK

Here is what I have - I think this is all the relevant stuff:

// ****************************************************************************
// ** Fill Box Structure
// ****************************************************************************
#define FILLBOX_WORD_SIZE  7         // word size expressed in bytes

struct Fillbox_s
    {
    int  xpos;       // horizontal position
    int  ypos;       // vertical position
    int  height;     // vertical size
    int  width;      // horizontal size
    unsigned char priority;   //0=lowest, 31=highest
    } fillBox;

unsigned char fbox_addr;

............
..............
.............

int loadFillBox(unsigned char fbox_addr, FillBox_s *p_fillBox)
{
    unsigned char       buf[FILLBOX_WORD_SIZE+2];


    buf[0] = 0x03;  // DRAM Page and Write flag
    buf[1] = fbox_addr;  // offset into DRAM page (address of the fill box)

    // format for OSD RAM
    buf[2] = ((fillBox.width>>4)&0x7F);
    buf[3] = ((fillBox.width<<4)&0xf0) | ((fillBox.height>>6)&0x0f);
    buf[4] = ((fillBox.height<<2)&0xfc) | ((fillBox.priority>>3)&0x03);
    buf[5] = ((fillBox.priority<<5)&0xe0) | ((fillBox.xpos>>7)&0x0f);
    buf[6] = ((fillBox.xpos<<1)&0xfe) | ((fillBox.ypos>>9)&0x01);
    buf[7] = (fillBox.ypos>>1);
    buf[8] = ((fillBox.ypos<<7)&0x80);

//    return_value = spiWrite(FILLBOX_WORD_SIZE+2, buf);

//    return return_value;
      return 0;
}

...........
.............
............

void setup()
{
  Serial.begin(9600);   // activate the serial output connection
  pinMode(ss, OUTPUT);   // SPI Slave Select
  SPI.begin();
  SPI.setBitOrder(MSBFIRST);
  Wire.begin();
 ......
.......
.....
 
  fbox_addr = 1;
  fillBox.xpos = 100;
  fillBox.ypos = 200;
  fillBox.height = 50;
  fillBox.width = 400;
  fillBox.priority = 5;
  loadFillBox(fbox_addr, *fillBox)

}


v0_06:44: error: 'FillBox_s' has not been declared
v0_06:165: error: 'FillBox_s' has not been declared
v0_06.ino: In function 'void setup()':
v0_06:216: error: no match for 'operator*' in '*fillBox'

Also tried

loadFillBox(fbox_addr, fillBox)
loadFillBox(fbox_addr, FillBox_s)
loadFillBox(fbox_addr, FillBox_s* fillBox)

and various option in the declaration of loadFillBox(…)

If I comment out the loadFillBox function and also the call to it in setup() then all compiles OK. It’s as soon as I uncomment the function declaration then all of a sudden Fillbox_s has not been declared anymore which seems nonsense to me as teh structure declaration is the same as before

I seem to have got into a mess with pointers it just does not seem to make sense.

I perfectly understand what I am trying to do here at a lower level - passing the start address of a structure in RAM to a function so it can access the data in the structure. That makes perfect sense, I could do it no problem in low level assembler (in a register or on the stack) but as hard as I try I can’t do it here in C ]:smiley:

Rich

Got it - I think

At least it compiles…

#define FILLBOX_WORD_SIZE  7         // word size expressed in bytes

struct Fillbox_s
    {
    int  xpos;       // horizontal position
    int  ypos;       // vertical position
    int  height;     // vertical size
    int  width;      // horizontal size
    unsigned char priority;   //0=lowest, 31=highest
    } fillBox;

unsigned char fbox_addr;


int loadFillBox(unsigned char fbox_addr, struct Fillbox_s *fillBox)
{
    unsigned char       buf[FILLBOX_WORD_SIZE+2];


//    buf[0] = 0x03;  // DRAM Page and Write flag
//    buf[1] = fbox_addr;  // offset into DRAM page (address of the fill box)

    // format for OSD RAM
    buf[2] = ((fillBox->width>>4)&0x7F);
    buf[3] = ((fillBox->width<<4)&0xf0) | ((fillBox->height>>6)&0x0f);
    buf[4] = ((fillBox->height<<2)&0xfc) | ((fillBox->priority>>3)&0x03);
    buf[5] = ((fillBox->priority<<5)&0xe0) | ((fillBox->xpos>>7)&0x0f);
    buf[6] = ((fillBox->xpos<<1)&0xfe) | ((fillBox->ypos>>9)&0x01);
    buf[7] = (fillBox->ypos>>1);
    buf[8] = ((fillBox->ypos<<7)&0x80);

//    return_value = spiWrite(FILLBOX_WORD_SIZE+2, buf);

//    return return_value;
      return 0;
}


void setup()
{
  Serial.begin(9600);   // activate the serial output connection
  pinMode(ss, OUTPUT);   // SPI Slave Select
  SPI.begin();
  SPI.setBitOrder(MSBFIRST);
  Wire.begin();
............
............
............
  fbox_addr = 1;
  fillBox.xpos = 100;
  fillBox.ypos = 200;
  fillBox.height = 50;
  fillBox.width = 400;
  fillBox.priority = 5;
  loadFillBox(fbox_addr, &fillBox);

}

What I am still not totally sure of - is did I end up with an array of seven bytes with the shifted bits in buf (as it is declared unsigned char array)?

hmmmmm

Rich

dicky96:
What I am still not totally sure of - is did I end up with an array of seven bytes with the shifted bits in buf (as it is declared unsigned char array)?

It looks that way, though I did not look to see if all of the shifts and masks were correct.