I hooked up a BME280 temperature/pressure/humidity sensor, a DS3231 RTC, a 24C32 EEPROM and the SSD1306 OLED display. Since some of the peripherals were 3.3V only, I used a Sparkfun logic level converter to go between the 5V lines on the R4 and the 3.3V lines on the peripherals. Everything was detected just fine.
Reading the RA4M1 User's Manual, there's some interesting (and confusing on first reading) values in Table 48.4, I/O Vih/Vil.
The minimum Vih for I2C (not SMBus) is listed as VCC x 0.7, or 3.5V. For SMBus I2C, it's 2.2V. Just reading that, if your battery was using 3.3V I2C levels, that could be your problem: Vih is never high enough with a 3.3V peripheral for non SMBus I2C.
Except that the internal pullups (10-50K) are enabled by the Wire library on both SDA and SCL (so they ought to be passively pulling the pins to 5V when they're not being driven low), and, the datasheet goes on to say that the I2C (not SMBus) values are only for pins P205, P206, P400, P401, P407, and P408.
The SDA/SCL pins brought out to the headers are P100 and P101, which are listed as the I2C (SMBus) pins, which means that their minimum Vih should be 2.2V.
I think I either need more caffeine, or a long walk to think about it.