Yeah, this was something that I didn't think of. I previously used the Arduino Nano Every, so I thought switching to the Portenta H7 would be okay.
If it helps, here's a wiring diagram of how I hooked it up. What concerns me are R1-R4.