[ethercard] Packet response 512 bytes is too small

Hi,

I’m relatively new with Arduino, I’ve done electronics before and programming of microcontrollers also but it’s been a few years. Anyway, I’m trying to read data from an API and parse it with the ethercard library and an Arduino UNO in combination with an ENC28J60 ethernet module.

I am able to connect to the API, and I do get a response. However, the problem is that the response is larger than 512 bytes and I only saw the first 512 bytes. I’ve solved that one by adding this rule ether.persistTcpConnection(true);, now the whole response is printed in chunks of 512 bytes. However, I need to get that all those chunks in 1 string and it looks like it’s getting double packets and is mixing them up which is a little funny.

Is there anyone who can help me out here?
What I really want to accomplish here is get the response in 1 String or char array…

This Is what my code looks like at the moment:

/******************************************
 * Author: WouterDS
 * Website: http://wouterds.be/
 * Project: Communication with A JSON API
******************************************/

/*
 * Libraries
 */
#include <LiquidCrystal.h>
#include <EtherCard.h>
#include "utility/stringbuffer.h"
#include "aJSON.h"

/*
 * Variables
 */
LiquidCrystal lcd(7, 6, 5, 4, 3, 2);

static byte mymac[] = { 0x74,0x69,0x69,0x2D,0x30,0x31 };
byte Ethernet::buffer[700];
static uint32_t timer;
char website[] PROGMEM = "api.wouterds.be";

/*
 * Functions
*/
static void logString(PGM_P s) {
	char c;
	while ((c = pgm_read_byte(s++)) != 0)
	Serial.print(c);
}

static void handleResponse(byte status, word off, word len) {
	logString(PSTR("\n########################## RECEIVED CHUNK OF DATA ##########################\n"));
	Ethernet::buffer[off+512] = 0;
	Serial.println((const char*) Ethernet::buffer + off);
}

/*
 * Initializers
 */
void initLcd() {
	lcd.begin(16, 2);
	lcd.print("Init LCD..");
	delay(500);
}

void initEthernet() {
	lcd.clear();
	lcd.print("Init ethernet..");

	if(ether.begin(sizeof Ethernet::buffer, mymac) == 0)
		logString(PSTR("Failed to access Ethernet controller"));

	ether.persistTcpConnection(true);

	if(!ether.dhcpSetup())
		logString(PSTR("DHCP failed"));

	ether.printIp("Arduino IP:	", ether.myip);
	ether.printIp("Gateway:	", ether.gwip);
	ether.printIp("DNS:		", ether.dnsip);

	if(!ether.dnsLookup(website))
		logString(PSTR("DNS failed"));

	ether.printIp("\nServer ip:	", ether.hisip);
}

/*
 * Setup
 */
void setup() {
	Serial.begin(9600);

	logString(PSTR("Communication with A JSON API\n-----------------------------\n"));

	initLcd();
	initEthernet();
}

/*
 * Loop
 */
void loop() {
	ether.packetLoop(ether.packetReceive());

	if (millis() > timer) {
		logString(PSTR("\n\n############################################################################\n########################### MAKING AN API REQUEST ##########################\n############################################################################\n"));
		timer = millis() + 30000;
		ether.browseUrl(PSTR("/info"), "", website, handleResponse);
	}
}

And these are the logs I’m getting at the moment:

Communication with A JSON API
-----------------------------
Arduino IP:	192.168.2.2
Gateway:	192.168.2.1
DNS:		192.168.2.1

Server ip:	198.211.118.224


############################################################################
########################### MAKING AN API REQUEST ##########################
############################################################################

########################## RECEIVED CHUNK OF DATA ##########################
HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Mon, 23 Dec 2013 00:39:49 GMT
Content-Type: application/json
Connection: close
X-Powered-By: PHP/5.4.23-1~dotdeb.1

{"status":"success","message":"OK","data":{"firstname":"Wouter","lastname":"De Schuyter","birthday":"1992-12-13 02:30:00","age":21,"favoriteColor":{"description":"Sunglow","hex":"#FFC23F","rgb":{"r":255,"g":194,"b":63}},"luckyNumber":13,"location":{"description":"Home","cc":"BE","country":"Belgium","city":"Astene","coordinates":{"lat":50.9945976

########################## RECEIVED CHUNK OF DATA ##########################
18437,"lng":3.5643990081919},"lastKnown":[{"description":"Home","cc":"BE","country":"Belgium","city":"Astene","coordinates":{"lat":50.994597618437,"lng":3.5643990081919},"when":"12 hours 15 minutes ago"},{"description":"Taxi","cc":"BE","country":"Belgium","coordinates":{"lat":51.041127473073,"lng":3.7270855925346},"when":"22 hours 30 minutes ago"},{"description":"Frituur Sympa","cc":"BE","country":"Belgium","city":"Gent","coordinates":{"lat":51.0567600813,"lng":3.7280076742172},"when":"22 hours 59 minutes a

########################## RECEIVED CHUNK OF DATA ##########################
HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Mon, 23 Dec 2013 00:39:49 GMT
Content-Type: application/json
Connection: close
X-Powered-By: PHP/5.4.23-1~dotdeb.1

{"status":"success","message":"OK","data":{"firstname":"Wouter","lastname":"De Schuyter","birthday":"1992-12-13 02:30:00","age":21,"favoriteColor":{"description":"Sunglow","hex":"#FFC23F","rgb":{"r":255,"g":194,"b":63}},"luckyNumber":13,"location":{"description":"Home","cc":"BE","country":"Belgium","city":"Astene","coordinates":{"lat":50.9945976

########################## RECEIVED CHUNK OF DATA ##########################
HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Mon, 23 Dec 2013 00:39:49 GMT
Content-Type: application/json
Connection: close
X-Powered-By: PHP/5.4.23-1~dotdeb.1

{"status":"success","message":"OK","data":{"firstname":"Wouter","lastname":"De Schuyter","birthday":"1992-12-13 02:30:00","age":21,"favoriteColor":{"description":"Sunglow","hex":"#FFC23F","rgb":{"r":255,"g":194,"b":63}},"luckyNumber":13,"location":{"description":"Home","cc":"BE","country":"Belgium","city":"Astene","coordinates":{"lat":50.9945976

########################## RECEIVED CHUNK OF DATA ##########################
go"}]},"email":"info[@]wouterds[.]be","website":"http:\/\/wouterds.be\/","nicknames":["WouterDS","Troxia","Paradox","Paradoxj"],"social":{"twitter":{"nickname":"wouterds","website":"http:\/\/twitter.com\/wouterds"},"linkedin":{"nickname":"wouterds","website":"http:\/\/linkedin.com\/in\/wouterds"},"vimeo":{"nickname":"wouterds","website":"http:\/\/vimeo.com\/wouterds"},"youtube":{"nickname":"wouterdeschuyter","website":"http:\/\/youtube.com\/user\/wouterdeschuyter"},"github":{"nickname":"wouterds","website":

########################## RECEIVED CHUNK OF DATA ##########################
go"}]},"email":"info[@]wouterds[.]be","website":"http:\/\/wouterds.be\/","nicknames":["WouterDS","Troxia","Paradox","Paradoxj"],"social":{"twitter":{"nickname":"wouterds","website":"http:\/\/twitter.com\/wouterds"},"linkedin":{"nickname":"wouterds","website":"http:\/\/linkedin.com\/in\/wouterds"},"vimeo":{"nickname":"wouterds","website":"http:\/\/vimeo.com\/wouterds"},"youtube":{"nickname":"wouterdeschuyter","website":"http:\/\/youtube.com\/user\/wouterdeschuyter"},"github":{"nickname":"wouterds","website":

########################## RECEIVED CHUNK OF DATA ##########################
"http:\/\/github.com\/wouterds"},"facebook":{"nickname":"wouter.de.schuyter","website":"http:\/\/fb.com\/wouter.de.schuyter"},"lastfm":{"nickname":"paradoxj","website":"http:\/\/last.fm\/user\/paradoxj"}},"time":"2013-12-23 01:39:47","ip":"178.117.106.51","host":"178-117-106-51.access.telenet.be","referer":null,"keys":{"averagePerDay":22309.861566485,"total":12248114},"clicks":{"averagePerDay":3517.9635701275,"total":1931362},"requests":{"today":480,"averagePerDay":196.66666666667,"lastWeek":590,"lastMonth"

########################## RECEIVED CHUNK OF DATA ##########################
:590,"lastYear":590,"total":590}}}

########################## RECEIVED CHUNK OF DATA ##########################
"http:\/\/github.com\/wouterds"},"facebook":{"nickname":"wouter.de.schuyter","website":"http:\/\/fb.com\/wouter.de.schuyter"},"lastfm":{"nickname":"paradoxj","website":"http:\/\/last.fm\/user\/paradoxj"}},"time":"2013-12-23 01:39:47","ip":"178.117.106.51","host":"178-117-106-51.access.telenet.be","referer":null,"keys":{"averagePerDay":22309.861566485,"total":12248114},"clicks":{"averagePerDay":3517.9635701275,"total":1931362},"requests":{"today":480,"averagePerDay":196.66666666667,"lastWeek":590,"lastMonth"

########################## RECEIVED CHUNK OF DATA ##########################
:590,"lastYear":590,"total":590}}}

Thanks in advance,

Kind regards,
Wouter

bump =(

No one?

Here I am again.. Hoping for someone that can help me.. I've been stuck for a few days and searched countless hours on this..

Anyone please?

Bump

EtherCard is not designed to do full tcp and has very limited support for persistent connections.
You may want to try UIPEthernet-library if this is a requirenment.

  • Norbert