Hello, i am trying to read data from a website that is a rss feed from twitter (not the now extinct twitter own rss feed)
I am really a bit lost in controlling the data even from the start..
Could someone just advise what i would need to send in the '"client print"section to just feed the whole site one character at a time?
I really want to learn how to do this, so I want to try to work out the steps to take after i have this returned data, so just a little nudge in the right direction please... (at least for now)
Thanks in advance
ps.. looking at the link here :Twitter for @guzqty is there anything that would inherently stop arduino from reading it?
and the code based on the webclient example
/*
Web client
This sketch connects to a website (http://www.google.com)
using an Arduino Wiznet Ethernet shield.
Circuit:
* Ethernet shield attached to pins 10, 11, 12, 13
created 18 Dec 2009
by David A. Mellis
modified 9 Apr 2012
by Tom Igoe, based on work by Adrian McEwen
*/
#include <SPI.h>
#include <Ethernet.h>
// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0xDE, 0xAC, 0xBE, 0xEF, 0xFE, 0xED };
// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
//IPAddress server(74,125,232,128); // numeric IP for Google (no DNS)
char server[] = "http://roomj.com/twitter/feed.php?t=2814997314-gIEojuQIS1AKtRMYChLg85CXjKzt9fbR7V4AEv5&s=0IoEo34Tk2CfgBYNmdGOZxDCurBtHwjRhojGP0EwnaXgL"; // name address for Google (using DNS)
// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192,168,0,18);
// Initialize the Ethernet client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;
void setup() {
// Open serial communications and wait for port to open:
Serial.begin(9600);
// while (!Serial) {
// ; // wait for serial port to connect. Needed for Leonardo only
// }
// start the Ethernet connection:
// if (Ethernet.begin(mac) == 0) {
// Serial.println("Failed to configure Ethernet using DHCP");
// // no point in carrying on, so do nothing forevermore:
// // try to congifure using IP address instead of DHCP:
Ethernet.begin(mac, ip);
//}
// give the Ethernet shield a second to initialize:
delay(1000);
Serial.println("connecting...");
// if you get a connection, report back via serial:
if (client.connect(server, 80)) {
Serial.println("connected");
client.print("GET ");
client.println(); // I FEEL LIKE I AM MISSING SOMETHING FROM HERE TO REQUEST THE DATA
}
else {
// kf you didn't get a connection to the server:
Serial.println("connection failed");
}
}
void loop()
{
// if there are incoming bytes available
// from the server, read them and print them:
if (client.available()) {
char c = client.read();
Serial.print(c);
}
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting.");
client.stop();
// do nothing forevermore:
while(true);
}
}
is there anything that would inherently stop arduino from reading it?
Not that I can see.
char server[] = "http://roomj.com/twitter/feed.php?t=2814997314-gIEojuQIS1AKtRMYChLg85CXjKzt9fbR7V4AEv5&s=0IoEo34Tk2CfgBYNmdGOZxDCurBtHwjRhojGP0EwnaXgL"; // name address for Google (using DNS)
That is NOT the server. The server is roomj.com. The GET request includes the part from twitter on, with " HTTP/1.1" on the end.
Hmm.. So i updated the code as i believe was indicated and it seems from the serial output that it makes a connection, but it doesn't print anything.. Output from serial monitor:
connecting at 9600
connecting...
connected <---- from the setup
disconnecting. <--- from the loop where it says (if server is disconnected)
Can anyone explain why it would be suggesting server is disconnected? how can i remedy that?
/*
Web client
This sketch connects to a website (http://www.google.com)
using an Arduino Wiznet Ethernet shield.
Circuit:
* Ethernet shield attached to pins 10, 11, 12, 13
created 18 Dec 2009
by David A. Mellis
modified 9 Apr 2012
by Tom Igoe, based on work by Adrian McEwen
*/
#include <SPI.h>
#include <Ethernet.h>
// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0xDE, 0xAC, 0xBE, 0xEF, 0xFE, 0xED };
// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
//IPAddress server(74,125,232,128); // numeric IP for Google (no DNS)
char server[] = "http://www.roomj.com"; // name address for Google (using DNS)
// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192,168,0,18);
// Initialize the Ethernet client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;
void setup() {
// Open serial communications and wait for port to open:
Serial.begin(9600);
// while (!Serial) {
// ; // wait for serial port to connect. Needed for Leonardo only
// }
// start the Ethernet connection:
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
// no point in carrying on, so do nothing forevermore:
// try to congifure using IP address instead of DHCP:
Ethernet.begin(mac, ip);
}
// give the Ethernet shield a second to initialize:
delay(1000);
Serial.println("connecting...");
// if you get a connection, report back via serial:
if (client.connect(server, 80)) {
Serial.println("connected");
client.print("GET /twitter/feed.php?t=2814997314-gIEojuQIS1AKtRMYChLg85CXjKzt9fbR7V4AEv5&s=0IoEo34Tk2CfgBYNmdGOZxDCurBtHwjRhojGP0EwnaXgL HTTP/1.1");
client.println("Host: www.roomj.com");
client.println("Connection: close");
client.println();
}
else {
// kf you didn't get a connection to the server:
Serial.println("connection failed");
}
}
void loop()
{
// if there are incoming bytes available
// from the server, read them and print them:
if (client.available()) {
char c = client.read();
Serial.print(c);
}
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting.");
client.stop();
// do nothing forevermore:
while(true);
}
}
tried everything that i can think of to test or troubleshoot, could it be the format of the data on the site? because its xml formatted for a rss feed?
Basic client test code you can try to see if you get a response from the server.
//zoomkat 9-22-12
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test
//for use with W5100 based ethernet shields
//remove SD card if inserted
#include <SPI.h>
#include <Ethernet.h>
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
char serverName[] = "web.comporium.net"; // zoomkat's test web page server
EthernetClient client;
//////////////////////
void setup(){
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
// no point in carrying on, so do nothing forevermore:
while(true);
}
Serial.begin(9600);
Serial.println("Better client test 9/22/12"); // so I can keep track of what is loaded
Serial.println("Send an e in serial monitor to test"); // what to do to test
}
void loop(){
// check for serial input
if (Serial.available() > 0) //if something in serial buffer
{
byte inChar; // sets inChar as a byte
inChar = Serial.read(); //gets byte from buffer
if(inChar == 'e') // checks to see byte is an e
{
sendGET(); // call sendGET function below when byte is an e
}
}
}
//////////////////////////
void sendGET() //client function to send/receive GET request data.
{
if (client.connect(serverName, 80)) { //starts client connection, checks for connection
Serial.println("connected");
client.println("GET /~shb/arduino.txt HTTP/1.1"); //download text
client.println("Host: web.comporium.net");
client.println("Connection: close"); //close 1.1 persistent connection
client.println(); //end of get request
}
else {
Serial.println("connection failed"); //error message if no client connect
Serial.println();
}
while(client.connected() && !client.available()) delay(1); //waits for data
while (client.connected() || client.available()) { //connected or data available
char c = client.read(); //gets byte from ethernet buffer
Serial.print(c); //prints byte to serial monitor
}
Serial.println();
Serial.println("disconnecting.");
Serial.println("==================");
Serial.println();
client.stop(); //stop client
}
Just a little feedback, have been diverted/disheartened from this branch of my project, and relented to attempt an alternative using Temboo. Really didn't want to, but it did work quite quickly and smoothly, but soon i reached the ceiling of my free temboo account allowances, and so i have returned to this to try Zoomkat's test sketch.
So after swapping the server details in the sketch to the target from above, i instantly got results, so i will be reusing the sendGET() void in my sketch.
Thanks for the help and suggestions. I'm not very good at all the coding etc, but i do really enjoy messing around with my arduino's and the support from this community is excellent.
tried everything that i can think of to test or troubleshoot, could it be the format of the data on the site? because its xml formatted for a rss feed?