Keypad code RGB

void loop()
{
  [glow]{[/glow]
//--------------This is the basic state-------------//
   while (BlackState);
  [glow]{[/glow]
//-When the key * is press, it resets to blackstate-//
    
    char key = kpd.getKey();
    if (key=='*')
    {
//---------This is what it do when you press *------//

//---------It prints the line to the serial---------//
      Serial.println("The leds is off");

Open and close curly braces should be used to define blocks of code, following statements that can have blocks, such as if, while, and for. The highlighted braces are not needed.

The ; following the while statement forms the body of that statement. If BlackState does ever happen to be true, the only way to exit that block is to reset the Arduino. Infinite loops that require reset to exit are hardly ever a good idea.

Comments should be meaningful, and explain what is going to happen, when that is not completely obvious. When the comment states what just happened, and anyone can clearly see what happened, and why, the comment is useless.

    if (key=='*')
    {
    }
    if (key=='1')
     {
     }

Consistent indenting is a good thing. Is there any value for key that can cause both of these blocks to execute? Is there any way for key to change in the block? If the answer to both questions is no, then you should be using an if/else if/else construct instead of many if statements.

When I press 9 i want it to go in a mode

The first thing you need to do is learn about functions. Create functions that have meaningful names, like lightsOut(), redOn(), greenOn(), etc. Call those functions when the appropriate key is pressed.

Then, create a function to be called when the 9 key is pressed. In that function, put the code to read the two potentiometers. The output from the analogRead functions can be used to change the PWM value used to control the voltage applied to PWM pins. If the LEDs are connected to PWM pins, you will be able to change the brightness of each color independently.