For accurate 3D orientation, it is essential to properly calibrate the magnetometer, the gyro and sometimes the accelerometer.
Subtracting a bias is not enough for the magnetometer. Best overview and tutorial: Tutorial: How to calibrate a compass (and accelerometer) with Arduino | Underwater Arduino Data Loggers
The Madgwick or Mahony filters will work well, but you may have to correct the sensor axes to ensure an internally consistent right handed coordinate system. Otherwise the derived angles will be nonsense.
The "update filter" loop must be fast -- recommended minimum 75 to 100 iterations per second. Sending the orientation out over a slow connection at every iteration is not only completely pointless, it will also result in nonsensical angles.