MPU6050 values behave strangely when mpu in vertical position

I am working on a project where I extract quaternions to deduce orientation using MPU6050 (DMP). I use the Arduino and libraries (latest update 2019) by Jeff Rowberg for calibration and reading from sensor. I have had a prototype working with MPU6050 mounted the standard way(gravity along Z) and it works fine. However when I mount MPU vertically where gravity is not along Z, it is along X and run the software the values 0 out well but the values behave strangely as if the center axis drifts but then gets back to where it started. I will demonstrate it here. For the demonstration I use Yaw Pitch Roll as it's easier to visualize.

Starts (Vertical position: gravity along X, x pointing downwards, y sideways):
ypr -0.02 2.11 0.19

Moves quickly to standard flat position (Gravity along Z, x pointing forward, y sidewars)
Values go up to: ypr -0.21 60.30 0.99
But then quickly drop to: ypr 3.55 25.11 0.03 (without me moving the sensor)

Moves back to initial position (Gravity along X)
Values go down to: ypr 0.35 -44.07 1.59
But the back rise up: ypr 1.20 -1.10 0.21 (withing 3 sec)

This problem only occurs when the program initializes mpu when mpu is in vertical position, everything works fine when initialized in horizontal no matter how I rotate the device.

Any help would be highly appreciated.

The sensor calibration is done incorrectly. But we can't identify the exact problem unless you post the code properly, using code tags as described in "How to use this forum".

With the MPU-6050, yaw is not absolute, and is defined relative to orientation of the sensor at startup, which depends on which axis you call "Z" at startup. It makes no sense to initialize the sensor with X up, then use it with Z up.

There is another orientation-dependent problem called "gimbal lock" which you cannot escape with Euler angles.

To avoid the problem, start up the sensor with Z vertical. Or use a 9 DOF sensor, which with decent software, can report absolute orientation.

The code is the same as here: MPU6050_DMP6.ino
with the only difference being the calibration which is:

// supply your own gyro offsets here, scaled for min sensitivity
mpu.setXGyroOffset(97);
mpu.setYGyroOffset(-353);
mpu.setZGyroOffset(84);
mpu.setZAccelOffset(1461); // 1688 factory default for my test chip
mpu.setYAccelOffset(-125); // Here I added additional offset for Y hoping that it would help
mpu.setXAccelOffset(1970) // Here I added additional offset for X hoping that it would help
;

What I mean by gravity along x or x pointing downwards is that the sensor is placed in this position (vertically) and then the code is ran:

VERTICAL POSITION IMAGE

While standard position (Gravity along x) is this:

STANDARD POSITION IMAGE

So when the code is run in standard position everything works fine, even if I rotate it to the vertical position and back. But not the other way.

In short I would like it to output the same values when placed vertically as it does when placed horizontally, but the vertical values behave as shown above, they move in the right direciton at least but that's about it. It might be that it uses the wrong acceleration axis when calculating the result hence the offset and delay but I don't quite understand how dmp calculates it's values so I can't say for certain and how to fix it. I'm sorry for not being able to explain it well.

Hardly anyone understands how the DMP works, and since both the MPU-6050 and MPU-9250 are now obsolete, it doesn't matter. I recommend to start using open source code.

In short I would like it to output the same values when placed vertically as it does when placed horizontally

Impossible. Clearly, you don't yet understand Euler angles, or the problem with yaw and the MPU-6050. Review of Euler angles.

The yaw angle is defined for navigational purposes to be clockwise from North (either magnetic or true), in the horizontal Earth reference frame, as the rotation about the vertical axis defined as "Z".

Thus it makes sense to initialize the sensor for yaw with the sensor in the horizontal position, with the axis you define as Z vertical. You cannot change that vertical axis definition after you have initialized the sensor.

Since the MPU-6050 can't measure magnetic fields, it can only estimate a relative yaw angle, and that will drift.

Thought question: what is the yaw angle for an airplane flying straight up?

So if I understand corectly it's impossible to initialize mpu6050 vertically and make it work, right?

Because it seems kinda strange to me since if I initialize it horizontally and then rotate to vertical position, it works fine regardless how I rotate it. And since I wanted to attach it to my leg and have it measure the angle (forward and sideways) I thought it would be possible to initialize it vertically as well.

No, the problem is Euler angles aren't general, and you shouldn't be using them internally at all.
Quarternion or DCM are the representations that actually work generally. Quarternions are probably
easiest.
Euler angles are something you use when talking to humans! And you need to convert from the internal
representation appropriately - if you calibrate in a different orientation, the conversion will be different.

3D geometry is much more complex than 2D geometry, there is no getting round it.

it’s impossible to initialize mpu6050 vertically and make it work, right?

For estimating the yaw angle, it is impossible to initialize the MPU-6050 vertically, then use it in the horizontal position.