How to secure/authenticate Arduino <> BLE module <> App

Hi guys,

I'm developing a gadget which is configured by an app via BLE - using a HM-19 BLE module to speak UART between the Arduino and the app.

Unlike Bluetooth Classic - with BLE there is no pairing, so any phone within range can connect to the BLE module and speak to the Arduino via serial, hijacking the device.

Are you guys aware of any ways to create authentication so only a certain bluetooth device can communicate with the Arduino?

I thought maybe if the Arduino can query the HM-19 for the mac address of the connected phone and hash it into a short code, app does the same, and every time the app sends data it can prefix the hash.
Arduino checks for this this code every time a command is received.

But this adds a lot of processing overhead and would pretty much double (if not more) my packet size, and I'm trying to keep things as fast and low latency as possible.

Thank you for any help