I think it's possible that the SD introduces interference. If the text itself on the LCD isn't affected (as in no new text added) then it probably has to do with interference on the power lines rather than the data lines, so the 4050 has nothing to do with it.
You should remember that the Arduino's 3.3v line can only supply about 50mA max, and SD cards can chew up much more than that, causing voltage to sag and cause all sorts of disruptions.
Now that I think about it I had a "flickering" issue when I ran an SD card and a 5V HD44780 display. When the card was reading or writing the display would go dim and sort of flicker.
In my next project which will involve using an SD card, I will be using a dedicated 3.3v voltage regulator just for that reason.