I'm not doing any waiting or delays etc but when I write to the SD card it sometimes takes quite a time before it returns
Writing to the SD card IS a delay. Just not in the usual sense of "do nothing else until" or "do something else until". Writing to the SD card takes time.
but don't have anything faster at the moment to test with.
fat16lib has posted some faster libraries, for the Mega particularly, where double buffering is possible, making writes a lot faster. I'd check out some of his work.