Byte Order when receiving 64bit long on socket from a android

helpseeker · October 17, 2016, 11:49pm

Hi,

my android phone is sending a 64bit long value via tcp/ip to my arduino (ethernet).

I know I have to create a long long variable here and assign the incoming bytes bit by bit.

And that’s what causes troubles to me. i tried this:

long long help=0;

for(i=0; i<8; i++){
help=(help << 8 );
help=( help | (long long)client.read() );
}

Serial.println((int)help);

this works fine within a value range of 0 to 65535. But when 65536 is coming as 0 and 65537 as 1 etc…

Well… what’s wrong here?

On Android side, I get the long value (in java a long has 8 bytes and as of java 8, there is no need to declare it unsigned) from java.io.file.length();

I really appreciate some expert advices. Thank you in advance

helpseeker · October 18, 2016, 1:15am

I'm pretty much sure now that my algorithm is correct but the print method is absolutely not appropriate and messing up the good work

Do you agree?

Any ideas of how I can read out that 64bit number?

By the way I read out all single bytes coming in. The binaries seem to be correct and coming in in little endian order (MSByte first, LSByte last).

lloyddean · October 18, 2016, 1:40am

Multi-byte values used in networking are Big Endian.

MSB-LSB

CrossRoads · October 18, 2016, 2:17am

Also try unsigned long.

julyjim · October 18, 2016, 3:58am

Here is a test code to verify this issue.

My observation

Read the data in first.
I keep hampering on that , but somehow it is not getting thru.
It is up to you to do it the easy way - using blocking code API or hard way by jumping thru hoops.
Use sliding mask to extract the data chunk - 8 bits in your case. That will keep your input intact,
see point 1.
Add the extracted chunk to output the way you need it _ MSB / LSB , inverted etc.
Serial class is limited as far as size of numeric variable, its up to you to figure out how to verify the 64 bits output.

Hope this helps.
Jim

void setup() {
  Serial.begin(115200);
  uint64_t input = 0x123456789ABCDE; // emulated data input 
  uint64_t  output =0 
  uint64_t  mask  = 0xff;
  for (int i = 0; i < 8; i++) {
    //help = ( help | input ); //  & 0xff;
    Serial.print("Input ");
    Serial.println((int)input , HEX );
    Serial.print("Output  ");
    Serial.println((int)output, HEX);
    output |= (input & mask );
    Serial.print("Output  ");
    Serial.println((uint32_t )output, HEX); 
    mask <<= 8; //input = input << 8 ;
  }
  Serial.println((int)input , HEX );
  Serial.println((int) output , HEX);
  while (1);
}

void loop() {
  // put your main code here, to run repeatedly:
}

lloyddean · October 18, 2016, 5:28am

Values can be adjusted with a simple routine as in the following example.

Place the first code block into a file located in the same directory as the demo .ino file and name it “endian.h”.

The second is the .ino file as an example to show it’s use.

This technique can be used on short, long, long longs, etc …

… well except perhaps long long can’t be printed, at least the last I tried several years ago.

// File - endian.h

#ifndef endian_H
#define endian_H

#include <stdint.h>

template<typename T>
T endian_swap(T data)
{
    T           value = data;
    uint8_t*    lhs = (uint8_t*)(&value) - 1;
    uint8_t*    rhs = (uint8_t*)(&value) + sizeof(T);
    uint8_t     t;
    while ( ++lhs < --rhs )
    {
        t       = *lhs;
        *lhs    = *rhs;
        *rhs    = t;
    }

    return value;
}

#endif

#include "endian.h"

void loop()
{    }

void setup()
{
    Serial.begin(9600);
    
    {
    uint16_t  value = 0xADDE;

    value = endian_swap(value);
    
    Serial.println(value, HEX);
    }
    
    {
    unsigned long  value = 0xEFBEADDE;

    value = endian_swap(value);
    
    Serial.println(value, HEX);
    }
}

helpseeker · October 18, 2016, 9:51am

Okay thank you all for the advices.

Before I start further attemps... Is there actually any way to convert a long long to a string? That would solve the problem almost instantly.

system · October 18, 2016, 11:58am

      Serial.println((int)help);

Why are you casting a long long to an int?

helpseeker · October 18, 2016, 1:02pm

PaulS:
      Serial.println((int)help);
Why are you casting a long long to an int?

The intention was to have at least the lower 32 bits printed. But therefore, I should not have used “unsigned long”. Of course Int is comepletely wrong here.

Anyways, I think I found a solution:

//////////////////////////////////////////////////////////////////////

unsigned long long help=0; unsigned long long1=0, long2=0; unsigned char c=0;

for(i=0; i<8; i++){

c=client.read();
Serial.print(c, HEX); Serial.print(" "); Serial.println(c, BIN);
help=( help << 8 );
help=( help | c );

}

long1=(unsigned long)(help >> 32); //copying higher 4 bytes to long1
long2=(unsigned long)(help); //copying lower 4 bytes to long1

Serial.println(“Longs are:”);
Serial.print(long1,HEX);Serial.print(" “);Serial.println(long1, BIN);
Serial.print(long2,HEX);Serial.print(” ");Serial.println(long2, BIN);

/////////////////////////////////////////////////////////////////////////////

I told my android to send the long (Java 8 (?) long) number “976786878678567600” and the printout on the monitor was:

…

D 1101
8E 10001110
3E 111110
7D 1111101
AC 10101100
E6 11100110
A 1010
B0 10110000
Longs are:
D8E3E7D 1101100011100011111001111101
ACE60AB0 10101100111001100000101010110000

…

according to some web hex converters,

D8E3E7DACE60AB0 is supposed to be 976786878678567600.

Can I assume this is working perfectly?

What risks would you guys expect in using this code?

I highly appreciate any kinds of feedbacks

system · October 18, 2016, 1:05pm

I highly appreciate any kinds of feedbacks

My first response is to question why the Android app is sending an 8 byte value to the Arduino.

helpseeker · October 18, 2016, 1:29pm

PaulS:
My first response is to question why the Android app is sending an 8 byte value to the Arduino.

The app is going to send a larger file to the arduino. Therefore it has to tell to the arduino, how large that file is, before it starts sending the file data so the arduino knows when to stop writing the file and get back to casual jobs on that connection (like sending other commands). the file size is returned from java.io.file.length() as a long, which is a 8byte-format in java.

Of course, there are many other ways which could do that (like using a separate connection, converting the long value to int before sending it etc.) but it was important for me to go like this.

system · October 18, 2016, 2:05pm

the file size is returned from java.io.file.length() as a long, which is a 8byte-format in java.

You could, since it is completely unreasonable to expect the Arduino to deal with a file containing more than 4,294,967,295 bytes, cast that value to a more appropriate type.

but it was important for me to go like this.

I can't imagine why you need to send unreasonably sized values, but more power to you.

helpseeker · October 18, 2016, 10:01pm

PaulS:
You could, since it is completely unreasonable to expect the Arduino to deal with a file containing more than 4,294,967,295 bytes, cast that value to a more appropriate type.

4,294,967,295 bits, no? that would be 512mb. Since SD cards can store many gbs, it shouldnt be totally useless to deal with long long.

Whandall · October 18, 2016, 10:27pm

helpseeker:
4,294,967,295 bits, no?

No, bytes.

helpseeker · October 18, 2016, 11:08pm

Whandall:
No, bytes.

Sorry guys, my bad.

Damn, so all the work for nothing. I could have used int straightly!

Why the hell is that java code giving the file size in a 8byte long then. That's enough for a 16777215 TB File! x(

a 4 byte unsigned long can still tell file sizes up to 4 GB.

But at least the code is good, isn't it?

lloyddean · October 18, 2016, 11:26pm

Probably because desktop Java runs on a 64 bit file system.

Your original code should probably read more like -

unsigned long long  value   = 0UL;
uint8_t*            p       = (uint8_t*)&value;

for ( size_t i = sizeof(value); i--; )
{
    // values come in in network order, MSB - LSB ...
    //
    // ... so place them in the buffer (value) from the end of the array to it's
    // beginning!!!
    
    p[i] = client.read();
}

julyjim · October 18, 2016, 11:31pm

lloyddean:
Probably because desktop Java runs of a 64 bit file system.

Your original code should probably read more like -
unsigned long long  value   = 0UL;

uint8_t* p = (uint8_t*)&value;

for ( size_t i = sizeof(value); i--; )
{
// values come in in network order, MSB - LSB ...
//
// ... so place them in the buffer (value) from the end of the array to it's
// beginning!!!

p[i] = client.read();
}

lloyddean:
Probably because desktop Java runs of a 64 bit file system.

Your original code should probably read more like -
unsigned long long  value   = 0UL;

uint8_t* p = (uint8_t*)&value;

for ( size_t i = sizeof(value); i--; )
{
// values come in in network order, MSB - LSB ...
//
// ... so place them in the buffer (value) from the end of the array to it's
// beginning!!!

p[i] = client.read();
}

Nice , you just confirm that KISS is still popular. Kudos.
Jim

system · May 5, 2021, 10:07pm