Is the 1280 (or 2560) energy efficient enough to compete with the arm chip / be comparable to the 328?
Yes. But you won't have the many small premade and sold module packaged options to use like the Pro-mini or Nano package to use. You might want to look at the Tensey 3.1 board as it's ARM based, very small and has software libraries and arduino IDE support files to work under the IDE.