Most gyroscopes precess and need to be rezeroed with reference to an independent direction finder. In VFR flight, the eyesight of the pilot is a 2nd backup to catch severe gyro offsets, and resetting the gyro compass to match the magnetic compass whilst in flight is one of those occasional tasks to annoy pilots.
The kalman filter implements noise reduction, which is a separate (desirable) feature to have apart from zero offset removal by reference to an independent standard. There is a reason why some gyro accelerometer compass units implement 9dof including magnetic compass. If you proposed a project without one then !whoops!
The gyro alone cannot be used for orientation.
The best Attitude Heading and Reference System (AHRS) package available for Arduino is RTIMUlib. You can read about my experience with it using the Altimu 10-V3.
I don't know if RTIMUlib is compatible with the V5, but Pololu's own software certainly should be.