I am doing the following code to write data to the customized TCP server that i have created on my desktop. The serial port shows data being sent in multiple batches. Not sure what am i doing wrong here?
Instead of using a traditional WIFI connection, i am doing it over LTE. This is why i am using the sim7000 LTE shield. I am also concerned about the data being consumed here. Hopefully it shouldn't be much becuase the image file is not more than 60-80 KB.
The SD card module i am using is the one from adafruit.
while (counter < 3 && !fona.TCPconnect("xx.xxx.xx.xx",1234)){
Serial.println("Couldn't connect to server");
counter ++;
delay(3000);
}
File myFile = SD.open("1.JPG");
// temp = SD.read_reg(
counter = 0; // This counts the number of failed attempts tries
if (fona.TCPconnected()) {
Serial.println("Connected to Server");
Serial.println("Start uploading...");
//!fona.TCPsend(packet, sizeof(packet))
if (myFile) {
byte clientBuf[244];
int clientCount = 0;
while(myFile.available())
{
clientBuf[clientCount] = myFile.read();
clientCount++;
if(clientCount > 243)
{
// Serial.println("Packet");
fona.TCPsend(clientBuf,sizeof(clientBuf));
clientCount = 0;
}
}
//final <64 byte cleanup packet
if(clientCount > 0) fona.TCPsend(clientBuf, sizeof(clientBuf));
// close the file:
myFile.close();
}
while (counter < 3 && !fona.TCPclose()){
Serial.println("Couldn't CLose to server");
counter ++;
delay(3000);
}
Serial.println("Server Connection Closed");
delay(5000);
}
ammarqs:
The serial port shows data being sent in multiple batches. Not sure what am i doing wrong here?
What do you expect from the output on the serial monitor? You are sending your data in batches of 244 bytes, so it should show up in batches, shouldn't it? Or am I missing something?
I had to send it batches because it wont otherwise. Or is there anyway i can send the whole thing as one file?
my server receives it as garbage. I will have to combine the data somehow so that when the server receives it, it recognizes as a combined file and not parts that it doesn't know what to do with.
I had to send it batches because it wont otherwise. Or is there anyway i can send the whole thing as one file?
my server receives it as garbage. I will have to combine the data somehow so that when the server receives it, it recognizes as a combined file and not parts that it doesn't know what to do with.
You can't buffer the whole image in memory, because 60k is way more than you have available on the arduino. Also the size parameter in the TCPsend method is of type uint8_t so you maximum package size is 255. So you will need to change your receiver to combine the blocks into an image.
I am using the mega here.
Even though i use the following code, and it sends everything to my server, it takes a lot of time for the image to send.
I correct myself, the image is only 5KB and and still it takes around 20-30 secs for this code to send data in blocks. It took around 18 blocks to send a 5Kb image.
Any other way i can do this? other than tcp server/client connection?
FTP is not an option as its not secure.
Reading 1 byte at a time is slow. Try readBytes(buf, len).
while (myFile.available() > 0) {
bytesIn = myFile.readBytes(clientBuf, sizeof(clientBuf));
if (bytesIn > 0) {
fona.TCPsend(clientBuf, bytesIn);
}
}
myFile.close();
What is the UART speed between the AVR and the fona device? Use the highest possible speed supported by both sides.
TCP is not secure by itself. FTP uses TCP so it is no more or less secure than TCP. If you need SSL/TLS crypto, that is much harder unless the fona device supports it.
5000 / 244 = 21.5 blocks so 18 is pretty close to correct. If 244 is a real limitation of the TCPsend() method, there is no way around having so many blocks.
Perhaps the problem is on the server side. I usually use netcat (also known as nc) on the server side for testing.
$ nc -l 1234 >mydata.dat
nc listens on TCP port 1234. When something connects, it writes all incoming data to the file mydata.dat. When the client closes the connection, nc closes the file and exits.
2. What is the UART speed between the AVR and the fona device? Use the highest possible speed supported by both sides.
3. TCP is not secure by itself. FTP uses TCP so it is no more or less secure than TCP. If you need SSL/TLS crypto, that is much harder unless the fona device supports it.
4. 5000 / 244 = 21.5 blocks so 18 is pretty close to correct. If 244 is a real limitation of the TCPsend() method, there is no way around having so many blocks.
5. Perhaps the problem is on the server side. I usually use netcat (also known as nc) on the server side for testing.
$ nc -l 1234 >mydata.dat
nc listens on TCP port 1234. When something connects, it writes all incoming data to the file mydata.dat. When the client closes the connection, nc closes the file and exits.
Thanks for the reply. I will try that and see what changes. I tried increasing the buf from 244 to 500 or even greater, it doesnt change anything. Not all blocks when printed on the serial port are of the same length. probably thats how an image in converted to hex. I tried it by using an online tool.
I increased the clientBuf value to 400, 600, 1000, 5000. and noticed that the blocks kept reducing, but the total values overall decreased. So i have a feeling that increasing the Buf size is affecting the data being sent to the server.
ammarqs:
Thanks for the reply. I will try that and see what changes. I tried increasing the buf from 244 to 500 or even greater, it doesnt change anything. Not all blocks when printed on the serial port are of the same length. probably thats how an image in converted to hex. I tried it by using an online tool.
The data type for bytesIn ?
As I said, the parameter type of the input buffer is of type uint8_t which ranges from 0 to 255. Setting it to 500 is actually the same as 244 (man, what a coincidence ...). The maximum amount of data you can pass to TCPsend are thus 255 bytes.
ammarqs:
I increased the clientBuf value to 400, 600, 1000, 5000. and noticed that the blocks kept reducing, but the total values overall decreased. So i have a feeling that increasing the Buf size is affecting the data being sent to the server.
Keeping in mind the parameter type you set the size to 144, 88, 232 and 136.
LightuC:
As I said, the parameter type of the input buffer is of type uint8_t which ranges from 0 to 255. Setting it to 500 is actually the same as 244 (man, what a coincidence ...). The maximum amount of data you can pass to TCPsend are thus 255 bytes.
Keeping in mind the parameter type you set the size to 144, 88, 232 and 136.
Yup i figured. I kept in mind what you said, but still wanted to try anyway. the data is being sent, but its taking too long. 18 blocks for a 5kb file, if i try increasing the resolution on the pictures, this will take forever to upload the pictures to the server.
There should be a way around. Using a POST request should work, I am limited to this:
Fona.HTTP_POST_start()
Description
High-level function to perform an HTTP POST request.
Syntax
fona.HTTP_POST_start(url, contenttype, postdata, postdatalen, status, datalen)
Parameters
url: string containing the URL
contenttype: flash string value of the HTTP Conten-Type header. Ex: F("text/plain")
postdata: uint8_t data to post
postdatalen: uint16_t length of the data to post
status: uint16_t for storing HTTP status code
datalen: uint16_t length of the server response
Returns
True if the AT command ran successfully
False otherwise
But to post an image data, encoding the body on this would still wont make any difference because of the 255 limit. So i would still have to post 18 blocks into the body to send the image.
Yes, this is the same one. TCPsend will delay the arduino for 3 seconds every time you call it. With the HTTP Post you can post 65535 bytes at the same time. Unfortunately you might not be able to buffer the data in memory, depending on the arduino you have.
Because you asked before changing your post completely: I am looking at the code. You are using the TCPsend method of some sim7000-lte library. So I googled for the library and took a look at its implementation of the TCPsend method. The other number:
Parameters
url: string containing the URL
contenttype: flash string value of the HTTP Conten-Type header. Ex: F("text/plain")
postdata: uint8_t data to post
postdatalen: uint16_t length of the data to post
status: uint16_t for storing HTTP status code
datalen: uint16_t length of the server response
You see the postdatalen parameter? Now you just check the valid range for its type, uint16_t, and it is 0 - 65535.
LightuC:
Because you asked before changing your post completely: I am looking at the code. You are using the TCPsend method of some sim7000-lte library. So I googled for the library and took a look at its implementation of the TCPsend method. The other number:
You see the postdatalen parameter? Now you just check the valid range for its type, uint16_t, and it is 0 - 65535.
OK right. So how does this affect and change what i am doing? This means i can increase the size of the clientBuf from 255 to 65K?
