Well, you said "byte array" rather than "char array". Try this:
void setup ()
{
Serial.begin (115200);
byte a, b;
byte foo = 0xB5;
a = foo >> 4;
b = foo & 0xF;
Serial.println (a, HEX);
Serial.println (b, HEX);
char bar = 0xB5;
a = bar >> 4;
b = bar & 0xF;
Serial.println (a, HEX);
Serial.println (b, HEX);
}
void loop () {}
Output:
B
5
FB
5
You are right, it sign-extends (the B not the 5). But you could do:
a = (bar >> 4) & 0xF;
That throws away the extended sign.