Size of HEX files

I discovered together with my nephew who is programming in BASCOM (BASIC) the Arduino IDE produces HEX files a lot larger than BASCOM does.
A test with a simple sketch (pressing a button and sending an ASCII code via serial port) shows these shocking differences:
Arduino: 1868 bytes
BASCOM: 658 bytes

That's quite a lot of difference, because we wanted to use an ATTiny2313 with only 2048K! :~

We can't tell you where the problem is unless you post the code.

That's quite a lot of difference, because we wanted to use an ATTiny2313 with only 2048K!

You can't judge the size of the object code simply from the size of a bin hex file because there are many ways of creating one and some have less overheads than others.
It depends on the line length that is used, very short line lengths produce large overheads and so might give you a distorted view of how big the final code that goes into the micro actually is.

ATTiny2313 with only 2048K!

Yikes, a tiny with over 2 million bytes of flash, must be a new model :slight_smile:

As Mike said there is no direct correlation between the HEX file size and the run time code size, or at least not if different formats are used.

Can you post 3-4 lines of each file so we can see the format of each?


Rob

Also, the binary size is always much smaller than the hex file size.
Your arduino .hex file with ~1900 bytes is probably about 800 bytes of code (sketch size should have been reported during the compile...)

Whoops, 2048K haha. That would be cool!

So, the Sketch file size the IDE reports isn't the real amount of data send to the chip? Or maybe the BASCOM compiler/IDE messages are false?

Sketch uses 1.876 bytes (91%) of program storage space. Maximum is 2.048 bytes.
Global variables use 110 bytes (85%) of dynamic memory, leaving 18 bytes for local variables. Maximum is 128 bytes.

My Arduino code:

int main(void) {
  init();
  {
    pinMode(13, INPUT);
    pinMode(12, INPUT);
    pinMode(11, INPUT);
    pinMode(10, INPUT);
    Serial.begin(9600);
  }

  while (1) {
    if (digitalRead(13)) {
      Serial.print("CODE1\r");
      delay(1000);
    }
    if (digitalRead(12)) {
      Serial.print("CODE2\r");
      delay(1000);
    } 
    if (digitalRead(11)) {
      Serial.print("CODE3\r");
      delay(1000);
    } 
    if (digitalRead(10)) {
      Serial.print("CODE4\r");
      delay(1000);
    } 
  }
}

His BASCOM code (maybe not really fair to compare 1 on 1, he forgot to insert wait after a button press):

$regfile = "attiny2313.dat"
$crystal = 8000000
$baud = 9600
$hwstack = 40
$swstack = 16
$framesize = 32

Waitms 100

Config Portb = Input
Portb = 1

Config Debounce = 30
Enable Interrupts

Sw1 Alias Pinb.0
Sw2 Alias Pinb.1
Sw3 Alias Pinb.2
Sw4 Alias Pinb.3

Do
Debounce Sw1 , 0 , Sw1_on , Sub
Debounce Sw2 , 0 , Sw2_on , Sub
Debounce Sw3 , 0 , Sw3_on , Sub
Debounce Sw4 , 0 , Sw4_on , Sub
Loop

Sw1_on:
Print "CODE1" ; &H0D
Return

Sw2_on:
Print "CODE2" ; &H0D
Return

Sw3_on:
Print "CODE3" ; &H0D
Return

Sw4_on:
Print "CODE4" ; &H0D
Return

What do you think, BASIC looks ugly eh? :smiley:

BASIC looks ugly eh?

Yes it does, mostly because most BASIC programmers have never heard of indenting, 100s of lines of code all starting on the left margin is very common.

That said your code could be about half the size as well :), no need for all the if() blocks.

the Sketch file size the IDE reports isn't the real amount of data send to the chip?

No, it's the size of the program as it sits in flash memory on the processor. A Typical HEX file record breaks the data into 16-byte chunks and formats it into ASCII with various control fields added, for example

:-10-0100-00-214601360121470136007EFE09D21901-40

(- added so you can see the fields)

The block starting with 21 is the 16 bytes of data in ASCII format.

So every 16 bytes of program == 43 bytes in the HEX file. The overhead is fixed so it can be better or worse if you package more or less data into each record, but 16 is pretty much the standard.

As for what the stamp reports I have no idea. Maybe they don't include the core libraries or something.


Rob

Graynomad:
...

That said your code could be about half the size as well :), no need for all the if() blocks.

...

I would really like to know how! :slight_smile:

the Sketch file size the IDE reports isn't the real amount of data send to the chip?

Wait... The size reported by the IDE is the amount of flash storage you need.
But ! It's unrelated to the "size of the .hex file" as reported by most file commands and such.
For example, for the Blink example:

Binary sketch size: 1,116 bytes (of a 32,256 byte maximum) - 3% used
 ll *.hex
-rw-r--r--  1 staff  3168 Mar 23 01:25 Blink.cpp.hex

I would really like to know how!

That'll teach me :slight_smile:

Here's an "Arduinoized" version.

char * strings[] = 	{"CODE1\r","CODE2\r","CODE3\r","CODE4\r"};
int 	pins[]	 =	{13, 12, 11, 10};

void setup() {
    Serial.begin(9600);
}
  
void loop () {
	for (int i = 0; i < 4; i++) {
		if (digitalRead(pins[i])) {
			Serial.print(strings[i]);
			delay(1000);
		}
	}
}

I think that does the same thing, and it's easy to change the order and/or number of pins/strings.


Rob

That is indeed a really nice way of doing it, but global memory wise, it takes up a little more space :smiley: :

Sketch uses 1.704 bytes (83%) of program storage space. Maximum is 2.048 bytes.
Global variables use 126 bytes (98%) of dynamic memory, leaving 2 bytes for local variables. Maximum is 128 bytes.

So the conclusion might be, programming for the tiny's is a different ballgame. The Arduino platform is off course good for the chips it is intended for: the ATMega's with a lot more memory. And these chips don't cost that much nowadays.

But I just found it remarkable an other development tool creates a lot smaller programs to upload to the same chip and do the same thing...

Graynomad:

I would really like to know how!

That'll teach me :slight_smile:

Here's an "Arduinoized" version.

char * strings[] = 	{"CODE1\r","CODE2\r","CODE3\r","CODE4\r"};

int pins[] = {13, 12, 11, 10};

void setup() {
    Serial.begin(9600);
}
 
void loop () {
for (int i = 0; i < 4; i++) {
if (digitalRead(pins[i])) {
Serial.print(strings[i]);
delay(1000);
}
}
}




I think that does the same thing, and it's easy to change the order and/or number of pins/strings.

______
Rob

That is indeed a really nice way of doing it, but global memory wise, it takes up a little more space

The trouble is we are comparing apples with tomatoes. If I Arduinoize the original version

void setup (void) {

    pinMode(13, INPUT);
    pinMode(12, INPUT);
    pinMode(11, INPUT);
    pinMode(10, INPUT);
    Serial.begin(9600);
  }

void loop() {
  while (1) {
    if (digitalRead(13)) {
      Serial.print("CODE1\r");
      delay(1000);
    }
    if (digitalRead(12)) {
      Serial.print("CODE2\r");
      delay(1000);
    } 
    if (digitalRead(11)) {
      Serial.print("CODE3\r");
      delay(1000);
    } 
    if (digitalRead(10)) {
      Serial.print("CODE4\r");
      delay(1000);
    } 
  }
}

I get 2562 bytes as opposed to my version at 2350 bytes. If I compile your code directly I get 2494 bytes. And if I apply the loop technique to the main() version

char * strings[] = 	{"CODE1\r","CODE2\r","CODE3\r","CODE4\r"};
int 	pins[]	 =	{13, 12, 11, 10};

int main(void) {
  init();
  {
    pinMode(13, INPUT);
    pinMode(12, INPUT);
    pinMode(11, INPUT);
    pinMode(10, INPUT);
    Serial.begin(9600);
  }

  while (1) {
 	for (int i = 0; i < 4; i++) {
		if (digitalRead(pins[i])) {
			Serial.print(strings[i]);
			delay(1000);
		}
	}
  }
}

I get 2446 bytes.

And finally a less flexible version with no pin initialisation (not required for inputs)

char * strings[] = 	{"CODE1\r","CODE2\r","CODE3\r","CODE4\r"};

int main(void) {
  init();
    Serial.begin(9600);
 
  while (1) {
 	for (int i = 10; i < 14; i++) {
		if (digitalRead(i)) {
			Serial.print(strings[i-10]);
			delay(1000);
		}
	}
  }
}

Is 2268 bytes.

So all else being equal the loop will be smaller than serial code and of course the body of the code does not get any larger even if you have 100 pins. Can you apply those last two versions to your platform and see what size they are?


Rob

Graynomad:
And finally a less flexible version with no pin initialisation (not required for inputs)

char * strings[] = 	{"CODE1\r","CODE2\r","CODE3\r","CODE4\r"};

int main(void) {
 init();
   Serial.begin(9600);

while (1) {
for (int i = 10; i < 14; i++) {
if (digitalRead(i)) {
Serial.print(strings[i-10]);
delay(1000);
}
}
 }
}




Is 2268 bytes.

If you take out the references to "Serial" and compile for Tiny85 (I don't have Tiny2313 installed) you get 666 bytes.

Conclusion: The Serial library is incredibly bloated.

What else can we do? Let's start by substituting "_delay_ms()" for "delay()" like this:

#include <util/delay.h>

int main(void) {
  //init();  // Not needed if we don't use delay()
    //Serial.begin(9600);
 
  while (1) {
 	for (int i = 10; i < 14; i++) {
		if (digitalRead(i)) {
			//Serial.print(strings[i-10]);
			_delay_ms(1000);
		}
	}
  }
}

Now it's only 282 bytes instead of 666.

We could return to the original style and use "digitalReadFast()" instead of "digitalRead()":

#include <util/delay.h>

/*--------------------------------------------------
  Inline copy of digitalwritefast.h for Tiny85
--------------------------------------------------*/
// Standard Arduino Pins
#define digitalPinToPortReg(P) &PORTB
#define digitalPinToDDRReg(P) &DDRB
#define digitalPinToPINReg(P) &PINB
#define digitalPinToBit(P) P

#define pinModeFast(P, V) \
if (__builtin_constant_p(P) && __builtin_constant_p(V)) { \
  bitWrite(*digitalPinToDDRReg(P), digitalPinToBit(P), (V)); \
} else {  \
  pinMode((P), (V)); \
} 

#define digitalWriteFast2(P, V) \
if (__builtin_constant_p(P) && __builtin_constant_p(V)) { \
  bitWrite(*digitalPinToPortReg(P), digitalPinToBit(P), (V)); \
} else { \
  digitalWrite((P), (V)); \
}

#define digitalReadFast2(P ) \
((__builtin_constant_p(P) ) \
 ? (byte)bitRead(*digitalPinToPINReg(P), digitalPinToBit(P)) \
 : (byte)digitalRead(P) \
)

int main(void) {
 
  while (1) {
    if (digitalReadFast2(0)) {
      _delay_ms(1000);
    }
    if (digitalReadFast2(1)) {
      _delay_ms(1000);
    }
    if (digitalReadFast2(2)) {
      _delay_ms(1000);
    }
    if (digitalReadFast2(3)) {
      _delay_ms(1000);
    }
  }
}

186 bytes instead of 282, not too bad.

Let's put all the messages back in (nb. using flash memory instead of RAM):

#include <util/delay.h>

/*--------------------------------------------------
  Inline copy of digitalwritefast.h for Tiny85
--------------------------------------------------*/
// Standard Arduino Pins
#define digitalPinToPortReg(P) &PORTB
#define digitalPinToDDRReg(P) &DDRB
#define digitalPinToPINReg(P) &PINB
#define digitalPinToBit(P) P

#define pinModeFast(P, V) \
if (__builtin_constant_p(P) && __builtin_constant_p(V)) { \
  bitWrite(*digitalPinToDDRReg(P), digitalPinToBit(P), (V)); \
} else {  \
  pinMode((P), (V)); \
} 

#define digitalWriteFast2(P, V) \
if (__builtin_constant_p(P) && __builtin_constant_p(V)) { \
  bitWrite(*digitalPinToPortReg(P), digitalPinToBit(P), (V)); \
} else { \
  digitalWrite((P), (V)); \
}

#define digitalReadFast2(P ) \
((__builtin_constant_p(P) ) \
 ? (byte)bitRead(*digitalPinToPINReg(P), digitalPinToBit(P)) \
 : (byte)digitalRead(P) \
)

void writeChar(char c)
{
  // We need to send 'c' via the Tiny2313 serial port
  USIDR = c;      // This is for Tiny85 :-)
}
void msg(const __FlashStringHelper *s)
{
  const prog_char *m = (prog_char*)s;
  char c = pgm_read_byte(m);
  while (c) {
    writeChar(c);
    c = pgm_read_byte(++m);
  }
}

int main(void) {
 
  while (1) {
    if (digitalReadFast2(0)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
    if (digitalReadFast2(1)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
    if (digitalReadFast2(2)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
    if (digitalReadFast2(3)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
  }
}

306 bytes.... almost an order of magnitude less than the original 2268 bytes.

We'll need another 50 bytes or so to get it fully working on a Tiny2313 but it'll be way smaller than the BASIC version.

I'm not surprised about Serial but who would have though delay() would be so bloated?

Yes Arduino libs are a HAL and you pay for that abstraction. Hang on I'll fire up the assembler :slight_smile:


Rob

fungus:
Let's put all the messages back in (nb. using flash memory instead of RAM):

#include <util/delay.h>

/--------------------------------------------------
  Inline copy of digitalwritefast.h for Tiny85
--------------------------------------------------
/
// Standard Arduino Pins
#define digitalPinToPortReg(P) &PORTB
#define digitalPinToDDRReg(P) &DDRB
#define digitalPinToPINReg(P) &PINB
#define digitalPinToBit(P) P

#define pinModeFast(P, V)
if (__builtin_constant_p(P) && __builtin_constant_p(V)) {
  bitWrite(*digitalPinToDDRReg(P), digitalPinToBit(P), (V));
} else { 
  pinMode((P), (V));
}

#define digitalWriteFast2(P, V)
if (__builtin_constant_p(P) && __builtin_constant_p(V)) {
  bitWrite(*digitalPinToPortReg(P), digitalPinToBit(P), (V));
} else {
  digitalWrite((P), (V));
}

#define digitalReadFast2(P )
((__builtin_constant_p(P) )
? (byte)bitRead(*digitalPinToPINReg(P), digitalPinToBit(P))
: (byte)digitalRead(P)
)

void writeChar(char c)
{
  // We need to send 'c' via the Tiny2313 serial port
  USIDR = c;      // This is for Tiny85 :slight_smile:
}
void msg(const __FlashStringHelper *s)
{
  const prog_char m = (prog_char)s;
  char c = pgm_read_byte(m);
  while (c) {
    writeChar(c);
    c = pgm_read_byte(++m);
  }
}

int main(void) {

while (1) {
    if (digitalReadFast2(0)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
    if (digitalReadFast2(1)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
    if (digitalReadFast2(2)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
    if (digitalReadFast2(3)) {
      msg(F("CODE1"));
      _delay_ms(1000);
    }
  }
}




306 bytes.... almost an order of magnitude less than the original 2268 bytes.

We'll need another 50 bytes or so to get it fully working on a Tiny2313 but it'll be way smaller than the BASIC version.

Nice! However, I'm getting a compile error on line 34:

sketch_mar23a.ino:34: error: expected ',' or '...' before '*' token

It won't compile for Tiny2313 - it doesn't have "USIDR"

Put "UDR" instead and see what happens.

Nope. No luck. Even if I remove that line. Isn't there an error in the lines above?

#define digitalReadFast2(P ) \
((__builtin_constant_p(P) ) \
 ? (byte)bitRead(*digitalPinToPINReg(P), digitalPinToBit(P)) \
 : (byte)digitalRead(P) \
)

fungus:
It won't compile for Tiny2313 - it doesn't have "USIDR"

Put "UDR" instead and see what happens.

It does compile if I set the board to the UNO.
When putting it back to ATTiny2313 or ATTiny85 I do get the error. :astonished:

armandzwan:
It does compile if I set the board to the UNO.
When putting it back to ATTiny2313 or ATTiny85 I do get the error. :astonished:

That's weird.

I'm not sure what line 34 is after copy/paste/etc.

Is it the line with "__FlashStringHelper"? Try replacing that with "void".

void msg(const void *s)

If we are still discussing .HEX file size (?), you might run your hex files through this:

/* readIntelHex.c
 * DdelV 20140311
 * 
 * Read Intel Hex records from stdin
 * Write data to stdout in "dump" format:
 * <addr>: <data> <data> ...
 * 
 */
#include <stdio.h>
#include <ctype.h>

// types and values
#define byte unsigned char
#define bool int
#define TRUE 1
#define FALSE 0

// limits
#define MAXBYTES 32
#define HEXBYTESPERLINE 16

// forward declarations
void exit(int );
void error(int, char *);
void processLine(char *);

// static & globals
static long int oldA;
static int bc;
long int extAddr=0;

/* main could be fancier:
 * 
 * read filename from command line, open, pass to processLine(),
 * read HEXBYTESPERLINE or use default
 * (limited by hex record length)
 * 
 */
int main() {
  oldA = -1;
  bc = 0;
  char Line[200];
  while (fgets(Line, sizeof(Line), stdin) != NULL) {
    processLine((char *)Line);
  }
  exit(0);
}

// noisy output
// #define OUTBUG

long int printHex(long int a, int b) {
  
#ifdef OUTBUG
  printf("printHex(%05x, %02x), oldA = %04x, bc = %02x",a,b,oldA,bc);
#endif
  
  if ((a == oldA) && (bc < HEXBYTESPERLINE)) {
    printf("%02x ",b);
    oldA = a;
    bc++;
  } else if (a >= 0x10000L) {
    printf("\n%05x:  %02x ", a, b);
    bc = 1;
  } else {
    printf("\n%04x:  %02x ", a, b);
    bc = 1;
  }
  return ++oldA;
}
/* getHex()
 * return value of next two hex digits
 */
int getHex(char *p) {
  int i, val = 0;

  for (i=0; i<2; i++) {
    if(!isxdigit(*p)) error((int)*p,"Bad hexadecimal digit!");
    val *= 16;
    if (isdigit(*p))
    val += (*p - '0');
    else val += (toupper(*p) - 55);
    ++p;
  }
  return val;
}
/* error()
 * print number and error message,
 * then exit with non-zero status
 */
void error(int numb, char *msg) {
  printf("0x%x %d %s\n",numb, numb, msg);
  exit(1);
}

// noisy input
// #define INBUG

/* processLine()
 * process the hex record pointed to
 * calls printHex() to output
 * 
 */
void processLine(char *cp) {
unsigned int byteCnt, recType, i;
byte nextByte, sum=0, chkSum;
bool gotEOF = FALSE;
unsigned long addr;

// eat everything at start of line until colon
  while (*cp++ != ':') { ; }

#ifdef INBUG
  printf("start, ");
#endif
  
// get byte count
  byteCnt = getHex(cp);
  cp += 2;
  if (byteCnt > MAXBYTES) error(byteCnt,"Line too long!");
  sum += byteCnt;

#ifdef INBUG
  printf("byteCnt=%d, ",byteCnt);
#endif
  
//
  addr = getHex(cp);
  cp += 2;
  sum += addr;
  addr *= 256;
  addr += getHex(cp);
  cp += 2;
  sum += addr;

#ifdef INBUG
  printf("addr = 0x%05x, ",addr);
#endif
  
// read record type byte
  recType = getHex(cp);
  cp += 2;
  sum += recType;
  if (recType == 1) {
    gotEOF = TRUE;
  
#ifdef INBUG
    printf("EOF record: ");
    if(addr)printf("start addr=0x%04x, ",addr);
#endif
    
  }
  if (recType > 2) error(recType,"Unsupported Record Type!");

#ifdef INBUG
  printf("recType=0x%02x, ",recType);
#endif
  
// read byte count bytes and print them in output format  
  if(byteCnt) {
    for (i=0; i<byteCnt ;i++) {
      nextByte = getHex(cp);
      cp += 2;
        if (recType == 2 && (addr || byteCnt != 2)) 
	  error((int)addr,"Error in record type 2!");
        if (recType == 2) extAddr = (extAddr << 8) + nextByte;
	else addr = printHex(addr, nextByte);
      sum +=  nextByte;
    }
  if (recType == 2) addr += (extAddr << 4);

#ifdef INBUG
  printf("%d data bytes read, ",i);
#endif
  }
  
// get and check check sum
  chkSum = getHex(cp);
  cp += 2;
  sum &= 255;
  
#ifdef INBUG
  printf(" chkSum=0x%02x, sum=0x%02x, residue=0x%02x",chkSum,sum,255 & (sum+chkSum));
#endif

  if (255 & (sum + chkSum)) error(255 & (sum+chkSum),"Checksum error!");

#ifdef INBUG
  printf(" cp now points to 0x%02x\n",*cp);
#endif
  
  if (*cp != (char)13) error((int)*cp,"Format error!");
  if (gotEOF)printf("\n");
}

I call it readIntelHex.c It outputs a "memory dump" view of the information in an Intel Hex file.
This might show where the differences are.