I have a working application whereby a file\image arrives on a Arduino via LoRa and is saved to SD. Files are circa 64Kbytes.
I would like to have the Arduino transfer the received file\image via its serial connection to appear in a folder on the PC. Applications such as XYModem etc spring to mind.
Is anyone aware of working solutions to this sort of application which will be part Arduino and part PC ?
Straight to the PC requires a program on the PC to handle that side.
That was explained on the old Arduino site before ghits seemingly trained by Microdick went in a screwed it all up. There info's there, just try and find it.
You can get there with Python, Java, Delphi, Processing and many languages but it's a PITA with C/C++.
Sure, but maybe you could use an PC serial application that supports XYModem and/or scripting.
My Python would not be good enough to write an application in a reasonable period, so it was maybe wishful thinking that someone somwhere had already done it.
Perhaps you will find this C program useful. It receives ASCII formatted 8 bit grey scale image data from the ESP32-CAM and writes a .bmp image file on a windows PC.
Currently, the program is configured to read a Teraterm log file, resulting from using Teraterm as the "serial monitor" for the ESP32-CAM. Developed using the Code::Blocks IDE.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
const int BYTES_PER_PIXEL = 3; /// red, green, & blue
const int FILE_HEADER_SIZE = 14;
const int INFO_HEADER_SIZE = 40;
void generateBitmapImage(unsigned char* image, int height, int width, char* imageFileName);
unsigned char* createBitmapFileHeader(int height, int stride);
unsigned char* createBitmapInfoHeader(int height, int width);
char buf[200]; //line input buffer
int main ()
{
int width = 320;
int height = 240;
int nlines = 0;
unsigned char image[height][width][BYTES_PER_PIXEL];
char* imageFileName = (char*) "image.bmp";
char* formattedFileName = (char*) "teraterm.log";
FILE *fpin, *fpout;
// read formatted image data produced by ESP32-CAM
fpin = fopen(formattedFileName, "r");
if (fpin == NULL) {printf("Input file not found"); return 0;}
// get the number of lines in the file
while(fgets(buf, 100, fpin) != NULL)
nlines++;
rewind(fpin);
printf("%d lines in input file\n",nlines);
// read till '*', beginning of image data
buf[0]=0;
nlines = 0;
while (buf[0] != '*') {
fgets(buf, 100, fpin);
nlines++;
}
printf("%d lines skipped at beginning\r\n",nlines);
nlines = 0;
int i=0, j=0;
int h_one = height-1; //bitmap indexing, upside down for .BMP format
unsigned char raw;
for (i = 0; i < height; i++) {
for (j = 0; j < width; j++) {
fgets(buf, 100, fpin);
nlines++;
// printf("%s",buf);
// if (buf[0] == '*') printf("* encountered at line %d\r\n",nlines);
raw = atoi(buf);
image[h_one-i][j][2] = (unsigned char) ( raw ); //red
image[h_one-i][j][1] = (unsigned char) ( raw ); //green
image[h_one-i][j][0] = (unsigned char) ( raw ); //blue
}
}
generateBitmapImage((unsigned char*) image, height, width, imageFileName);
printf("Image generated!!");
}
void generateBitmapImage (unsigned char* image, int height, int width, char* imageFileName)
{
int widthInBytes = width * BYTES_PER_PIXEL;
unsigned char padding[4] = {0, 0, 0, 0}; //bug fix SJR
int paddingSize = (4 - (widthInBytes) % 4) % 4;
printf("padding bytes %d\r\n", paddingSize);
int stride = (widthInBytes) + paddingSize;
FILE* imageFile = fopen(imageFileName, "wb");
unsigned char* fileHeader = createBitmapFileHeader(height, stride);
fwrite(fileHeader, 1, FILE_HEADER_SIZE, imageFile);
unsigned char* infoHeader = createBitmapInfoHeader(height, width);
fwrite(infoHeader, 1, INFO_HEADER_SIZE, imageFile);
int i;
for (i = 0; i < height; i++) {
fwrite(image + (i*widthInBytes), BYTES_PER_PIXEL, width, imageFile);
// if(paddingSize > 0)
fwrite(padding, 1, paddingSize, imageFile);
}
fclose(imageFile);
}
unsigned char* createBitmapFileHeader (int height, int stride)
{
int fileSize = FILE_HEADER_SIZE + INFO_HEADER_SIZE + (stride * height);
static unsigned char fileHeader[] = {
0,0, /// signature
0,0,0,0, /// image file size in bytes
0,0,0,0, /// reserved
0,0,0,0, /// start of pixel array
};
fileHeader[ 0] = (unsigned char)('B');
fileHeader[ 1] = (unsigned char)('M');
fileHeader[ 2] = (unsigned char)(fileSize );
fileHeader[ 3] = (unsigned char)(fileSize >> 8);
fileHeader[ 4] = (unsigned char)(fileSize >> 16);
fileHeader[ 5] = (unsigned char)(fileSize >> 24);
fileHeader[10] = (unsigned char)(FILE_HEADER_SIZE + INFO_HEADER_SIZE);
return fileHeader;
}
unsigned char* createBitmapInfoHeader (int height, int width)
{
static unsigned char infoHeader[] = {
0,0,0,0, /// header size
0,0,0,0, /// image width
0,0,0,0, /// image height
0,0, /// number of color planes
0,0, /// bits per pixel
0,0,0,0, /// compression
0,0,0,0, /// image size
0,0,0,0, /// horizontal resolution
0,0,0,0, /// vertical resolution
0,0,0,0, /// colors in color table
0,0,0,0, /// important color count
};
infoHeader[ 0] = (unsigned char)(INFO_HEADER_SIZE);
infoHeader[ 4] = (unsigned char)(width );
infoHeader[ 5] = (unsigned char)(width >> 8);
infoHeader[ 6] = (unsigned char)(width >> 16);
infoHeader[ 7] = (unsigned char)(width >> 24);
infoHeader[ 8] = (unsigned char)(height );
infoHeader[ 9] = (unsigned char)(height >> 8);
infoHeader[10] = (unsigned char)(height >> 16);
infoHeader[11] = (unsigned char)(height >> 24);
infoHeader[12] = (unsigned char)(1);
infoHeader[14] = (unsigned char)(BYTES_PER_PIXEL*8);
return infoHeader;
}
You can add an SD module to an Arduino, write the file to that and use a media player on the PC to read them, SD is like the floppy disk to Arduino. When we moved floppies between computers 30-40 years ago it was called sneaker-net and it was faster at times than the 19200 baud serial (but.. a very solid 19.2!) we had rigged.
Am I missing something here? I do this on a phone whereby the filename is sent from a terminal via Bluetooth, and Arduino responds accordingly. Surely you can do the same with RealTerm or the like on a PC? Indeed, I'm sure I did once. It's just data-on-demand stored in a declared folder, rather than simply streamed.
I do that already, but want to avoid having to take the SD out of the Arduino and plugging it into the PC every time a image arrives.
You would think so, all the Arduino needs is an implementation of XYModem or similar and be able to push the file\image over to the PC.
Hence the question if there is a known implementation ........
The Processing language uses a form of Java, but as Java is implemented in C it looks almost like C and I would be surprised if anything would be a problem to you.
OK, so do you know if there is a file transfer library\example for serial for Arduino and Processing ?
I can sort of think how it could be done in a simple way, start with a header that says here is x bytes with a CRC of Y coming, send them, and wait for an OK.
But if that wheel has already been invented ...........
I found the Dumpfile example included in the IDE was all that is required. You send the filename, and Arduino sends the file. No libraries, no pythons, and no winkling SD out of the slot.
The file would go to Excel. I believe later versions of Excel can access a file directly.
I had not noticed that sketch, but looking at it all it appears to do is send (write) every byte of an SD file to serial. Not sure how the receiver end would know when to start or stop or error check.
At 2 attempts to implement xmodem and ymodem on Arduino. On Linux the rz/sz commands supports x, y, and z modem transfers.
I had to #include the serial library in an earlier version of Processing.
Here is another one for the Arduino side of things: GitHub - gdsports/XYmodem: Arduino XYMODEM file transfer protocol
Ah, maybe that is what I was missing, because that is what I thought you wanted. As for the start, stop, and error check, I'm not sure either, and have never felt the need to ask.
I did mention applications such as Xmodem in the first post.
Whether a simple blast down the serial port would be reliable enough, I am not sure. I guess I could test it Arduino to Arduino with some form of timeout to signal transfer end.
There would still would need to be some form of CRC check. A corrupt byte might be obvious in a jpg image bit no obvious in a data file.
Clearly you need to implement SMB over Netbios over TCP over IP over PPP over Aysnc HDLC! 
Not sure but it is simple, you just read the bytes coming in from the serial into a buffer, then write that buffer out to a file. There is an example of writing a file.
This is the first one so you can get the feel of how the language works:-
/**
* SaveFile 1
*
* Saving files is a useful way to store data so it can be viewed after a
* program has stopped running. The saveStrings() function writes an array
* of strings to a file, with each string written to a new line. This file
* is saved to the sketch's folder.
*/
int[] x = new int[0];
int[] y = new int[0];
void setup()
{
size(200, 200);
}
void draw()
{
background(204);
stroke(0);
noFill();
beginShape();
for (int i = 0; i < x.length; i++) {
vertex(x[i], y[i]);
}
endShape();
// Show the next segment to be added
if (x.length >= 1) {
stroke(255);
line(mouseX, mouseY, x[x.length-1], y[x.length-1]);
}
}
void mousePressed() { // Click to add a line segment
x = append(x, mouseX);
y = append(y, mouseY);
}
void keyPressed() { // Press a key to save the data
String[] lines = new String[x.length];
for (int i = 0; i < x.length; i++) {
lines[i] = x[i] + "\t" + y[i];
}
saveStrings("lines.txt", lines);
exit(); // Stop the program
}
Note it is also generating some data to save. The second file example is
/**
* SaveFile 2
*
* This file a PrintWriter object to write data continuously to a file
* while the mouse is pressed. When a key is pressed, the file closes
* itself and the program is stopped.
*/
PrintWriter output;
void setup()
{
size(200, 200);
// Create a new file in the sketch directory
output = createWriter("positions.txt");
frameRate(12);
}
void draw()
{
if (mousePressed) {
point(mouseX, mouseY);
// Write the coordinate to a file with a
// "\t" (TAB character) between each entry
output.println(mouseX + "\t" + mouseY);
}
}
void keyPressed() { // Press a key to save the data
output.flush(); // Write the remaining data
output.close(); // Finish the file
exit(); // Stop the program
}
You still have to do that, although with an import command and not a # include. Here is the included example of a simple serial read
/**
* Simple Read
*
* Read data from the serial port and change the color of a rectangle
* when a switch connected to a Wiring or Arduino board is pressed and released.
* This example works with the Wiring / Arduino program that follows below.
*/
import processing.serial.*;
Serial myPort; // Create object from Serial class
int val; // Data received from the serial port
void setup()
{
size(200, 200);
// I know that the first port in the serial list on my mac
// is always my FTDI adaptor, so I open Serial.list()[0].
// On Windows machines, this generally opens COM1.
// Open whatever port is the one you're using.
String portName = Serial.list()[0];
myPort = new Serial(this, portName, 9600);
}
void draw()
{
if ( myPort.available() > 0) { // If data is available,
val = myPort.read(); // read it and store it in val
}
background(255); // Set background to white
if (val == 0) { // If the serial value is 0,
fill(0); // set fill to black
}
else { // If the serial value is not 0,
fill(204); // set fill to light gray
}
rect(50, 50, 100, 100);
}
The setup function is the same but the loop function is replaced by the draw function. In the setup this function can be optionally made to be called only once. Otherwise it is called every video frame. There are lots of other examples included in the language as well.
You also mentioned, twice, a working solution - which you were given. I recall Xmodem is just another terminal programme, but of great antiquity. I imagine people would be more inclined to use RealTerm, like me, although I do it via Bluetooth to the phone these days. RealTerm can be quite sophisticated, but you may find that you need no more than to nominate the directory.