Since they came from solar garden lights, and are about 50mm square, it's likely that they're good for about 3V, with a "full sun" output on the order of 75-100mA. This is assuming that the lights were the typical sort with a single AA battery: if they had more batteries (I have one with 4 AAs in it), the cells would probably be rated for a higher voltage and lower current.
So you're not going to harm anything or anyone by making mistakes while testing them. Unless you try to force-feed a voltage into the solar cell: in that case, you might turn it into a Smoke Emitting Diode 
Next time you go shopping for electronic parts, pick up a few Schottky diodes with a current rating on the order of .5-1A: if you're going to be doing battery charging, you'll want a diode to protect the solar cell from the battery when it's dark. The Schottky diode is better than an ordinary diode because it wastes less energy when the solar cell is charging.
A google search will turn up examples of what people have done to get more-precise measurements, but you can get a rough idea by just putting it under a bright lamp with nothing but the meter connected (the "open-circuit voltage"), then with a resistor across it that approximates its maximum load. In this case, assuming the guess of "3V 100mA" is correct, the resistor should be about 30 Ohms. Since that guess could be wrong, I'd start with about 50 Ohms, and see what voltage I get. If the voltage is low (closer to 3V than 4 or 5V), I'd try a smaller resistor to see whether the voltage holds up when drawing more current.
Ran