It must go in loop() in the place where your code switches off the the LED.
Yes, you need to create the variable prevAnalogValue. It may be simpler to create all the variables before the setup() function - that way they will retain their values when loop() repeats. Alternatively create them inside loop() with the "static" prefix
...R