The BNO055 simply does not work very well. The gyro sensor calibration is obscure, unstable and cannot be controlled. Likewise for the magnetometer.
Hence, the sensor drifts.
You would be much better off using a more modern 9DOF sensor, and a good 3D fusion filter like Mahony or Madgwick, where you have complete control over the essential step of sensor calibration. The LSM9DS1, with careful calibration and a good fusion filter, works much better than the BNO005.