SSD1309 2.4" Oled causing weird noise/drops

Any luck? I have an ssd1306 128x32 display and it is causing similar noise/drops either on 5v or 3.3v when connected to Arduino Uno. But when I connect the OLED to a breadboard power supply's 3.3v, the signal is clean. But that kills the portability of my project.