At first glance the code looks reasonable, the basic logic looks OK.
I never like expressions with multiple different operators where the precedence is not defined explicitly. I think in this case the & and == are close together in precedence but not identical, and I think == has higher precedence. To apply the bitwise operator first, you need to define the precedence explicitly:
if ( (state & 0xF0) == GROUP_0 )
ETA: Riva beat me to it.
I am not sure that will fix it, but it probably will. If it doesn't, then next thing to look at would be the types used in those expressions. To reduce the opportunity for the compiler to use the wrong type, I suggest you use 8 bit integers throughout (whichever type you prefer - uint8_t would be fine, and replace your #define macros with const variables so that all your constants are correctly typed, too.