Here is the deal:
To begin with, abs() is not a function. It is a macro that uses the conditional (ternary) operator:
#define abs(x) ((x)>0?(x):-(x))
The instruction under analysis then becomes
iGn = (iG%100-100) ? (iG%100-100) : -(iG%100-100);
Now, iG is declared as an unsigned int. The expression whose absolute value we are calculating, iG%100-100, is evaluated using integer arithmetic and its result is interpreted as an unsigned int.
In the signed int realm, this expression can take values from -100 to -1. In the unsigned int world, it is always positive (65437 to 65535, if you care to know).
A positive value is then used in the conditional operation to select either the first result or the second result. Always.
The conditional operation returns the first result. Always.
This value is then copied to iGn, which is a regular 16-bit integer that can have values from -32768 to +32767. That is when the negative sign appears.
Suppose iG gets from millis()* a value of 12345. The expression iG % 100 - 100 yields 65481 as an unsigned integer, (-55 using signed integer arithmetic).
* A variable getting its value from millis() should be declared as unsigned long. Unless you are ready for more fun.
Our expression becomes
iGn = 65481 > 0 ? 65481 : -65481.
Understandably, the result is the first possibility, 65481. When you assign this number to a 16-bit integer, you get a negative number! ( the bits are the same, its interpretation is not).
If in doubt about this try the following:
void setup() {
Serial.begin(115200);
Serial.println((unsigned int)(65481));
Serial.println((int)(65481));
}
void loop() {
}
I must say I learned a lot trying to figure this one out, and I was able to keep some of my hair in place.