Absolute Orientation on Arduino Nano 33 BLE and Unity

Hi! I'm trying to develop a sketch for an Arduino Nano 33 BLE that sends absolute orientation information over BLE. The goal is to control a wand-type object in a Unity game. I want to achieve a similar controlling capability of a VR controller basically.

I tried using the atan2 methods to calculate roll, pitch, and yaw after reading from the gyroscope, magnetometer and accelerator, but the data was choppy, noisy, and I had a similar issue as someone else posted in the forum where when I performed a pitch motion, the yaw value also increased, skewing the rotation.

I tried using the Madgwick library to calculate and send quaternion values. I also performed hard iron calibration and, in my sketch, subtracted those biases. With the quaternion values applied to the game object in Unity, I was having a different problem; the orientation values were just not accurate, and I was getting a constant gliding on different directions, but definitely constant. So it wasn't choppy, but it just wasn't really responsive.

Since I need precise absolute orientation to control the wand-type object, what library is best? Is there something I'm not considering or something I'm doing wrong? I appreciate the help.

This is the excerpt of the code where I'm calculating orientation:

void loop()
{
    unsigned long now = millis();
    if (now - lastUpdate >= interval) {
        lastUpdate = now;
        
        // Reading value from Accelerometer, Gyroscope and Magnetometer in order to determine absolute orientation
        if (IMU.accelerationAvailable()) {
            IMU.readAcceleration(ax, ay, az);
        }

        if (IMU.gyroscopeAvailable()) {
            IMU.readGyroscope(gx, gy, gz);
        }

        if (IMU.magneticFieldAvailable()) {
            IMU.readMagneticField(mx, my, mz);
        }

        // Convert gyroscope units from degrees/sec to radians/sec (required by filter)
        gx *= PI / 180.0;
        gy *= PI / 180.0;
        gz *= PI / 180.0;

        // Subtract bias from magnetometer
        // This bias was calculated with a different script
        mx -= 9.455;
        my -= 9.54;
        mz -= -33.835;
        
        // Update Madgwick filter
        filter.update(gx, gy, gz, ax, ay, az, mx, my, mz);

        // Get quaternion components
        filter.getQuaternion(qx, qy, qz, qw);
        
        // Sending information over BLE 

}

For accurate 3D orientation, it is essential to properly calibrate the magnetometer, the gyro and sometimes the accelerometer.

Subtracting a bias is not enough for the magnetometer. Best overview and tutorial: Tutorial: How to calibrate a compass (and accelerometer) with Arduino | Underwater Arduino Data Loggers

The Madgwick or Mahony filters will work well, but you may have to correct the sensor axes to ensure an internally consistent right handed coordinate system. Otherwise the derived angles will be nonsense.

The "update filter" loop must be fast -- recommended minimum 75 to 100 iterations per second. Sending the orientation out over a slow connection at every iteration is not only completely pointless, it will also result in nonsensical angles.

I had come across that tutorial before. It's pretty extensive. Something I noted while reading it its that "magnetometers really do need to be calibrated inside their final operating environment". This doesn't seem to align well with what I'm trying to do.

I'm trying to achieve a similar usage of a regular game controller. Things like Phones, Nintendo Switch Joycons, VR controllers, all don't seem to need such a complicated calibration for their gyroscope modules. Are the sensors for the Arduino Nano 33 BLE so different that I wouldn't achieve a similar level of control? Whenever I can control the wand in Unity, it is very choppy and jumpy, likely due to the noise from the orientation reading. Is there any way I could ge this as smooth as these other controllers available?

Also. Would anyone know if Euler Angles or Quaternions is better for this? I was under the impression that applying Euler Angles in Unity could cause the gimbal lock issue, but now I'm not so sure.

They are inaccurate, unsuitable for navigation and inconsistent with your stated requirement "Since I need precise absolute orientation ...".

Accurate IMU calibration does take a few minutes of work, so your willingness to put effort into it is a measure of the project's importance.