Just a general comment on coding style: If you're representing bit patterns, do NOT write the equivalent number in decimal. Write in hex or binary (or octal, if you must). The issues with 132, 253, 147 etc that were mentioned earlier would have been more immediately apparent.
Looking at your circuit, I guess the idea is to output 0V to the negative side of an LED and 5V to the positive side; the LED will illuminate. An RGBW LED is actually 4 LEDs in one package, so you actually have 256 LEDs to control, hence you need to use all 8 bits of both shift registers. It would be helpful to know exactly what the logical arrangement of the LEDs vs shift registers is. Do you see it as 4 rows of 16 LEDs, with 4 outputs to select the colour? Or an 8x8 grid of LEDs, with a 2x2 combination to select the colour?
But, generally speaking, if you output 0x01 to the positive side of the LED matrix and 0xFE to the negative side, you should illuminate one colour of the LED at the bottom left corner of your matrix.
Beware that turning on more than one LED in a row or column may exceed the current capabilities of a shift register (35mA) and possibly destroy it. Probably you should have 8 transistors controlled by each shift register to deliver the current to the LEDs. But then you somehow need to limit the current flowing through each LED.
So, for the positive side, values you should use are one of: 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80.
For the negative side, you want one of the inverses of those: 0xFE, 0xFD, 0xFB, 0xF7, 0xEF, 0xDF, 0xBF, 0x7F.
To cycle through bits, the << operator is useful. e.g. the following (untested code!) will cycle through all your LEDs.
for (int x = 1; x < 0x100; x <<= 1) {
digitalWrite(latchPin, LOW);
//Send x to Q7 Q6 Q5 Q4 Q3 Q2 Q1 Q0 of 1st 74595
shiftOut(dataPin, clockPin, MSBFIRST, x); // shifting out the bits:
digitalWrite(latchPin, HIGH);
for (int y = 1; y < 0x100; y <<= 1) {
digitalWrite(latchPin2, LOW);
//Send inverse of y to Q7 Q6 Q5 Q4 Q3 Q2 Q1 Q0 of 2nd 74595
shiftOut(dataPin2, clockPin2, MSBFIRST, ~y); // shifting out the bits NB: inverse Y
digitalWrite(latchPin2, HIGH);
delay(1000);
}
}