UDD_ReleaseRX should only get called when UDD_FifoByteCount is 0, therefore no data should get lost.
If it is only getting one byte in 512 it should be fairly easy to prove by sending packets such as 0 1 2 3... 254 255 and seeing what bytes get lost.
I don't have (easy) access to windows to try your code but might be able to get the VB source running in mono on Linux.