The WiFi library doesn't work with ESP8266 so either you're using the wrong library or you're using the wrong reference. Please post your entire sketch using code tags(</> button on the toolbar) and explain how you're using the ESP8266(i.e. as a serial module connected to a standard Arduino board, or programming the ESP8266 directly using the esp8266 Arduino core).
There are some differences so you can't assume the Arduino reference always applies perfectly. Unfortunately the enc8266 core libraries are very lacking in documentation, apparently they expect you to just dig around in the source code to figure things out.
kalex:
for some networks i get the result 255.
The question is, what means 255?.
The source for encryptionType() is at:
As you can see, there are a couple of cases where it will return -1 which becomes 255 when cast to uint8_t.