Esp32c3 Time out when connecting via i2C

Projects General Guidance
1

Hi all.
I have xiao esp32c3 and want to connect to bno085(edit: from adafruit, shouldn't require pullups) via I2C. Both parts are working as I can do bluetooth connection with xiao and when I connect the bno sensor to arduino board, it works. When I connect them together, I am not able to establish a connection. In rare occasions it connects and then I can do the readings without any problem, even under a lot of movement. So there isnt a problem with the wiring.

I am trying to run this simple code:

#include <Adafruit_BNO08x.h>

Adafruit_BNO08x  bno08x(-1);
sh2_SensorValue_t sensorValue;

void setup(void) {
    Serial.begin(115200);
    while (!Serial) delay(10); 

    Serial.println("Adafruit BNO08x test! ");
    delay(1000);
      while (!bno08x.begin_I2C()) {
            Serial.println("Failed to find BNO08x chip ");
            delay(500);
      }
      Serial.println("BNO08x Found!");

I get this error message:
I2C hardware timeout detected
probe device timeout. Please check if xfer_timeout_ms and pull-ups are correctly set up

I tried

Wire.setClock(40000);

and a lot of other stuff recomended by chatgpt and what I thought would work. For example changing and specifying adresses but, I can run the same code without problem when using arduino mega.
Any ideas how to increase the timeout?