XBee API library cannot run on high baud rates

I am having troubles getting Duemilanove 328 with series 1 XBee talking to PC in API mode with this library on higher baud rates. (I will post the link in the next post, since this is my first post.)
The PC is connected to a series 1 XBee using a XBee explorer, and using the same library except the serial code is replaced with windows serial functions. Arduino talks to PC via XBee in API mode on 9600 and 19200 reliably, but doesn't work on higher baud rates.
PC 2 PC is fine on all baud rates. Arduino always times out when calling readPacket(timeout), and the reason seems to be that Serial.available() always returns false.... I am not sure why, but that seems to be the case. I tested this with flashing LEDs..

void XBee::readPacket() {
// reset previous response
if (_response.isAvailable() || _response.isError()) {
resetResponse();
}

while (Serial.available()) {
...
...
...
}
}

Anyone knows why?

Thanks

Here is the link GitHub - andrewrapp/xbee-arduino: Arduino library for communicating with XBee radios in API mode
it's referenced here Libraries - Arduino Reference under XBee.

Anyone has similar problems?