Wireless File Transfer Library, Arduinos with SDs

Does anyone know of a library or example that demonstrates the packetised transfer of files, via RF of some sort, from one Arduino to another ?

The initial application is sending JPG files, stored on SD, with Arduino compatibles via LoRa over long distances, 100km or so. But maybe other file types too.

Although I am primarily interested in using LoRa, seeing an example on another RF system would help.

Yes, you can use SerialTransfer.h to send packets of serial data robustly between Arduinos. Here is an example of the library transferring ~2kbyte JPEG images from one ESP32 to another: example

Note: As of SerialTransfer.h release 2.1.0, the above example is slightly out of date and would need minor updates to remain bug free. In release 2.1.0 I updated the order of arguments in "rxObj()" and "txObj()" methods.

srnet:
Although I am primarily interested in using LoRa, seeing an example on another RF system would help.

I use the lib in my own RF applications (RC airplane control) and can verify it works for all serial-interfacing RF systems

srnet:
The initial application is sending JPG files, stored on SD, with Arduino compatibles via LoRa over long distances, 100km or so.

LoRa range, at best is 15-20km.

Power_Broker:
Yes, you can use SerialTransfer.h to send packets of serial data robustly between Arduinos. Here is an example of the library transferring ~2kbyte JPEG images from one ESP32 to another: example

Note: As of SerialTransfer.h release 2.1.0, the above example is slightly out of date and would need minor updates to remain bug free. In release 2.1.0 I updated the order of arguments in "rxObj()" and "txObj()" methods.

I use the lib in my own RF applications (RC airplane control) and can verify it works for all serial-interfacing RF systems

Are there any examples for a complete file transfer ?

The LoRa devices to be used are the SPI based modules, not the serial UART versions.

blh64:
LoRa range, at best is 15-20km.

Current LoRa UHF ISM record stands at 835km, and LoRa 2.4Ghz at 89km.

srnet:
Are there any examples for a complete file transfer ?

The example does a complete file transfer.

srnet:
The LoRa devices to be used are the SPI based modules, not the serial UART versions.

Hmm, that’s a problem then. I might make versions of the library to work with I2C and SPI in the coming weeks, though…

EDIT: I added SPI file transfer functionality to the lib.

Power_Broker:
The example does a complete file transfer.

Hmm, that's a problem then. I might make versions of the library to work with I2C and SPI in the coming weeks, though...

I will take a look at the file transfer, thanks.

I am planning to use the SPI based modules in this case as they do provide some features that you cannot manage with the UART ones, such as fine control of the LoRa parameters.

At the byte level filling a LoRa devices packet buffer via SPI is not much different to sending the packet via Serial, although in the case of RF sytems there would need to be some form of send and acknowledge.

srnet:
I am planning to use the SPI based modules in this case as they do provide some features that you cannot manage with the UART ones, such as fine control of the LoRa parameters.

I'm planning a big update with my SerialTransfer.h library: extending it's functionality from just UART to UART, I2C, and SPI.

I'll post here once the update is done and live, but for now, if you want to look at the code while it's in development, check out the current dev branch.

So far the dev branch works for UART and I2C, but SPI is still a work in progress.

PJON?

Riva:
PJON?

Thanks for that, not heard of it before. I will study it.

Initially I see it does include support for LoRa, but only the SX127x devices, which would be a significant limitation for the intended application.

My own LoRa library supports the SX126x (UHF) and SX128x (2.4Ghz) LoRa devices, both of which have significant advantages for sending files in that they can be operated at 100% duty cycle in a lot of places. SX127x devices are really limited to 10% duty cycle, so throughput is significantly lower.

@srnet,

As of release 3.0.0, you can now use SerialTransfer with UART, I2C, and SPI. With this update, you can now transfer entire files over SPI. I've even included an examples of how to transmit/receive 2kb files over UART and SPI.

GitHub is down rn, but if you download the current release via the Arduino Libraries Manager, you can look at the file transfer example sketches.

I hope you find it useful and please let me know what you think of it.

@Power_Broker

I can see what you have done and that would appear to very useful for addressing groups of Arduinos over SPI or I2C.

Not quite what I was looking for, I need to have a mechanism for checking that packets have been received, and they may be at long distance and there can be other stations on-air creating their own packets. Whilst LoRa has its own internal CRC checking, that does not prevent it receiving the wrong packets.

I was expecting to find some examples of file transfer using say the RadioHead reliable datagram, you might think its a common requiirement, but perhaps not.

I have started on my own solution. Progress so far is a transmission with packettype, destinationnode and source node identifiers. The TX routine calculates the 16 bit CRC of the packet on the fly, which the receiver responds with its own calculation as an ACK. Thus there is a reasonable certainty the transmitted packet has been received correctly.

srnet:
I was expecting to find some examples of file transfer using say the RadioHead reliable datagram, you might think its a common requiirement, but perhaps not.

You mean "a common requirement" in the RadioHead library or mine?

srnet:
I have started on my own solution. Progress so far is a transmission with packettype, destinationnode and source node identifiers. The TX routine calculates the 16 bit CRC of the packet on the fly, which the receiver responds with its own calculation as an ACK. Thus there is a reasonable certainty the transmitted packet has been received correctly.

Fair enough, but the lib has CRC calculations built-in and ACKs can still be implemented at the sketch level. Also, the file transfer example uses a file index as part of all the packets that transfer actual file data. You can add extra logic to check if you've dropped a packet by seeing if the file index "jumps" by a factor of (MAX_PACKET_SIZE - 2). Then, you could go back and request all of those dropped file index packets and recoup the missing data.

Power_Broker:
You mean "a common requirement" in the RadioHead library or mine?

I meant in general, as in transferring files over RF in general but excluding WiFi.

And also excluding UART front ended LoRa modules.