Loosing single characters on FTP upload from arduino

Using Arduino Networking, Protocols, and Devices
#1

Hi,

Have been trying this now for some time.

I am uploading JPG files to server, However they loose characters and sometimes the file is larger.

Using standard arduino FTP with check for buffer and pins added but no change.

File fh;

byte doFTP()
{
	#ifdef FTPWRITE
	fh = SD.open(JPG_Filename,FILE_READ);
	#else
	SD.remove(JPG_Filename);
	fh = SD.open(JPG_Filename,FILE_WRITE);
	#endif

	if(!fh)
	{
		Serial.println(F("SD open fail"));
		return 0;
	}

	#ifndef FTPWRITE
	if(!fh.seek(0))
	{
		Serial.println(F("Rewind fail"));
		fh.close();
		return 0;
	}
	#endif

	Serial.println(F("SD opened"));

	if (client.connect(server,21)) {
		Serial.println(F("Command connected"));
	}
	else {
		fh.close();
		Serial.println(F("Command connection failed"));
		return 0;
	}

	if(!eRcv()) return 0;

	client.println(F("USER xxxxx"));

	if(!eRcv()) return 0;

	client.println(F("PASS xxxxx"));

	if(!eRcv()) return 0;

	client.println(F("SYST"));

	if(!eRcv()) return 0;

	client.println(F("PASV"));

	if(!eRcv()) return 0;

	char *tStr = strtok(outBuf,"(,");
	int array_pasv[6];
	for ( int i = 0; i < 6; i++) {
		tStr = strtok(NULL,"(,");
		array_pasv[i] = atoi(tStr);
		if(tStr == NULL)
		{
			Serial.println(F("Bad PASV Answer"));

		}
	}
	
	client.println(F("CWD public_html"));

	if(!eRcv()) return 0;
	unsigned int hiPort,loPort;

	hiPort = array_pasv[4] << 8;
	loPort = array_pasv[5] & 255;

	Serial.print(F("Data port: "));
	hiPort = hiPort | loPort;
	Serial.println(hiPort);

	if (dclient.connect(server,hiPort)) {
		Serial.println(F("Data connected"));
	}
	else {
		Serial.println(F("Data connection failed"));
		client.stop();
		fh.close();
		return 0;
	}

	#ifdef FTPWRITE
	client.print(F("STOR "));
	client.println(JPG_Filename);
	#else
	client.print(F("RETR "));
	client.println(JPG_Filename);
	#endif

	if(!eRcv())
	{
		dclient.stop();
		return 0;
	}

	#ifdef FTPWRITE
	Serial.println(F("Writing"));

	byte clientBuf[64];
	int clientCount = 0;

	/*
	while(fh.available())
	{
		clientBuf[clientCount] = fh.read();
		clientCount++;

		if(clientCount > 63)
		{
			dclient.write(clientBuf,64);
			clientCount = 0;
		}
	}
	*/
	
	    [b]int newFree = client.free();

	    Serial.print("Free Buffer:");
		Serial.println(newFree, DEC);
			
		while(fh.available())
	    {
	    digitalWrite(10,HIGH);
	    //digitalWrite(4,LOW);
		clientCount = fh.read(clientBuf,32);
	    //digitalWrite(10,LOW);
	    digitalWrite(4,HIGH);
	    newFree = client.free();

	    if(newFree < 250)
     	{
		   Serial.print("Low buffer ");
		   Serial.println(newFree, DEC);
	    }
		dclient.write(clientBuf,clientCount);
		int icount = 0;
		while (icount < clientCount)
		{
			if(clientBuf[icount]<0x10) Serial.print("0");
			Serial.print(clientBuf[icount++],HEX);
			Serial.print(" ");
		}
		Serial.println();
	    newFree = client.free();

	    Serial.print("Free Buffer:");
	   Serial.println(newFree, DEC);
		
		delay(50);
	}[/b]

	if(clientCount > 0) dclient.write(clientBuf,clientCount);

	#else
	while(dclient.connected())
	{
		while(dclient.available())
		{
			char c = dclient.read();
			fh.write(c);
			Serial.write(c);
		}
	}
	#endif

	dclient.stop();
	Serial.println(F("Data disconnected"));

	if(!eRcv()) return 0;

	client.println(F("QUIT"));

	if(!eRcv()) return 0;

	client.stop();
	Serial.println(F("Command disconnected"));

	fh.close();
	Serial.println(F("SD closed"));
	return 1;
}

the correct data comes from the SD card when printed out but the received file is loosing characters.

have highlighted code area problem.

Using atmega 2560 and Ethernet shield.

any pointers to tracking the problem would be appreciated. I am connecting over wireless from switch. But the data loss is too consistent to be wireless issue. And I can download anything from Filezilla.

Have tried connecting to local PC Filezilla for testing but unsuccessful yet. Can connect to web with no problem except for missing characters around 155 bytes in.

Have changed buffer sizes to no avail.

Any suggestion welcome?

thanks

albert

#2

What is wrong with my original code? I see you have that remarked out. Read 64 bytes from the SD, then send those. Do that over and over until all have been sent.

edit: Where did you get the this?

	    int newFree = client.free();

There is no client.free function in the ethernet library. What do you think this does?

#3

i am loosing single characters in original code and this code.

So nothing wrong with code i suspect. I think there must be an interaction between hardware/time/somewhere else.

Have been checking everywhere for possible error cause.

The function is an addition to ethernet that check the queue in the w500(?) chip. It is not guaranteed but might point out if the queue overfilled. That does not happen here that i notice from tracing.

int EthernetClient::free() {
if (_sock != MAX_SOCK_NUM)
return W5100.getTXFreeSize(_sock);
return 0;
}

The serial prints show the data comes clean from the SD card which has a correct image. Yet when I do the write to dclient the data gets corrupted somehow with lost bytes.

The code is yours and should work as proven by others but I am still trying to get it to work. I am now trying to get it to put the file on my local machine through the filezilla server to test if its the broadband or connection. Not found anything yet that really makes much difference to the occasional bytes missing from the file on the web.

have not got the permissions working on filezilla/windows yet.

Any ideas welcome.

Total code consumes about 12% memory and Data consumes 20%. So should not be issue on mega 2560 and ethernet shield.

Thanks

albert

#4

Every call to dclient.write creates and sends another packet. If you lose any characters, I would think it would be an entire line sent by the write function (the whole packet), not just a character here and there.
.
I have sent some large files using that algorithm and never lost a character. If you do find the cause, please let me know what you found.

#5

will keep you posted on testing.

#6