next topic »

[on:]

Good day guys,

I'm connecting my project-Arduino twitter display (Arduino +Ethernet shield+ LCD) to my school's ethernet port and of course every institution comes with proxy settings of their own.

I've submitted an application for access to bypass current proxy settings and they have given me a static IP address on top of bypassing it.

However, my program still doesn't retrieve the string of tweets and weather via GET HTTP command.

Could a firewall be in the way? How do i troubleshoot this problem?

p.s. this is a working program tested with my home internet without a proxy.

Appreciate the help!

Cheers

[Reply #1 on:]

Could a firewall be in the way?

Yes, it probably is.

How do i troubleshoot this problem?

Take your Arduino offline, give the IP to your laptop and use the telnet command to do the same as your Arduino would do. For example, accessing a web server:

# telnet www.google.com 80
GET / HTTP/1.0
Host: www.google.com


You have to finish with two presses of the return key (sending CR/LF twice).
If this returns you some HTML code you really have a proxy bypass. Else you don't.

Have you considered to use the proxy? Is it an authenticated proxy (do you have to enter username and password)?

[Reply #2 on:]

Good day,
 
The guy from IT i spoke to gave me the telnet command to try out the proxy bypass he allowed. it was working however, my laptop was using wifi whereas my arduino was using an ethernet cable.

yup it is an authenticated proxy from school. requires our student ID number as well as password.

I'll connect my laptop to the school's network via ethernet tomorrow and try the telnet again.

let you know the results when i've done so. in the mean time if you have any other suggestions, do let me know!

cheers mate!

[Reply #3 on:]

The guy from IT i spoke to gave me the telnet command to try out the proxy bypass he allowed. it was working however, my laptop was using wifi whereas my arduino was using an ethernet cable.

If it was working on your laptop over the WLAN (which probably gave you a completely different IP) is very strange. Seems that it's not really an installed bypass but a work-around of some kind. Can you post the telnet command (with all IP addresses replaced by "X" for security reasons, same for passwords and the like)?

If you post the information how you have to configure your browser for using the proxy (what value in which field, without telling the user ID and password of course), I can tell you how you can integrate the proxy into your sketch.

[Reply #4 on:]

Good day Pylon,

the it guy told me over the phone to type: "telnet yahoo.com 80" in terminal on my macbook. the school's IP is a 10.2.XX.XX

to configure proxy, using a google chrome browser:

step 1: go to settings
step 2: under Network heading, open Change Proxy Settings...
step 3: Under the Network window, select Proxies.
Step 4: on the same window, enter "http://proxy.newcastle.edu.au/proxy.pac" in the URL box.
Step 5: when you open the browser, you'll be prompted to enter Student ID: c3xxxxxx and password: xxxxxxxx

then you'll be able to access the internet.

Hope it helps!

cheers!

[Reply #5 on:]

If that's the real URL you posted, it does not forces you to use the proxy but just prefers the proxy for a list of domains, always including a failback to a direct (without proxy) connection if the proxy doesn't answer in time. That's probably why the "bypass" also worked on your laptop. I guess that the firewall is filtering out some destinations that are not allowed in your school, maybe twitter is one of them.

Try the telnet example with your twitter hostname and port 80. Does that work too?

[Reply #6 on:]

Good day Pylon,

I spoke to the it guy over the phone and he says the static IP address that he gave me {10.2.XX.XX} isn't passing any data over the connection. When i ran my program, i realised it didn't even get through the Ethernet(mac,ip) stage. What do you suggest is the problem?

Also is there a way i can get past authentication with the sketch i have? Let me know if you need to view my sketch.

Cheers mate!
Kenneth

[Reply #7 on:]

Yes, I guess it's time to post your (complete) code (replace private information with XX as you have done before). When specifying an IP (as you seem to do), no DHCP or something similar takes place, so the code should return immediately. You don't specify a gateway or DNS server resulting in the code taking the most common values.

{10.2.XX.XX}

What network mask do you have to use? If you don't have the values you may plug in a laptop and see what it gets per DHCP. In most cases it will be the same as you have to use with the fixed IP.

[Reply #8 on:]

Code:

#include <SPI.h>
#include <Ethernet.h>
#include <TextFinder.h>
#include <LiquidCrystal.h>

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(9,8,7,6,5,4);   //School's Arduino

byte mac[] = {0xA2, 0xB1, 0x9A, 0xD2, 0xB3, 0xC5}; //mac address(do not change)

byte ip[] = {10,2,XX,XX}; //static IP for school
//byte ip[] = {192,168,X,XX}; //Michael Street IP

byte server[] = {128,242,240,20}; // Twitter
char serverName[] = "api.twitter.com";  // twitter URL
char tweet[140];

EthernetClient client;

TextFinder  finder(client); 

void setup()
{
  Serial.begin(9600);
  Ethernet.begin(mac,ip);
 Serial.println("Connecting..");
}


void loop()
  {
    client.stop();
    client.flush();
    checktweet();
    checkweather();

    delay(80000);
}

this is one of the segments i implement the twitter portion using a get HTTP

Code:

  void checktweet()
  {
{
  if (client.connect(serverName,80)) {
    client.println("GET http://www.twitter.com/statuses/user_timeline/XXXXXXX.rss HTTP/1.0");
    client.println();
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("Connected");
    lcd.setCursor(0,1);
    lcd.print("to server!!!");
    delay(1500);
  }
  else {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("Connection");
    lcd.setCursor(0,1);
    lcd.print("failed...");
    delay(1500);
    return;
  }
  if (client.connected()) {
     // get the last tweet by simply parsing the item and title tags
     if((finder.find("<item>")&&(finder.getString("<title>","</title>",tweet,140)!=0)))
     {   
         lcd.clear();
         lcd.setCursor(0,0);
         lcd.print("Retrieving");
         lcd.setCursor(0,1);
         lcd.print("Tweet...");
         delay(2000);
         lcd.clear();
         lcd.print(tweet);
         
         for (int j=0; j<2; j++) {
           // first part of the tweet
           lcd.clear();
           lcd.setCursor(0,0);
           for (int i=15; i<35; i++)
             lcd.print(tweet[i]);
           lcd.setCursor(0,1);
           for (int i=35; i<55; i++)
             lcd.print(tweet[i]);

           }
     }
      else
      lcd.println("Could not find tweet");
  }
  else {
    lcd.clear();
    lcd.setCursor(0,0);
    lcd.print("Could not retrieve");
    lcd.setCursor(0,1);
    lcd.print("tweet...");
    delay(2000);
    return;
  }
  delay(1);
  client.stop();
  client.flush();
  //delay(40000); // wait a minute before next update
 /* lcd.clear();
  lcd.setCursor(0,0);
  lcd.print("Refreshing");
  lcd.setCursor(0,1);
  lcd.print("connection");
  delay(2000);*/
 
}

  }

at home, DHCP using the static IP i have is fine. but in school with all the proxy/firewall and whatnot it's almost impossible to connect to it except via mobile broadband.. such a bother!

cheers mate!

p.s. sorry i can't give my full code cus this is my fyp and i don't want someone to nick it off here. but it's just similar to the checktweet segment.

[Reply #9 on:]

I guess the problem is the network setup.

At home you have let's say an IP of 192.168.1.44. The Ethernet.begin() method then extracts the non-specified information about DNS, gateway and network mask from it, this way:

DNS and gateway is same IP with last byte set to 1: 192.168.1.1
Network mask is always 255.255.255.0

So let's say the school's fixed address is 10.2.4.67, then the Ethernet class will assume the following values:

Gateway/DNS: 10.2.4.1
Network mask: 255.255.255.0

If these values are correct for the school network, the Arduino should work. But I guess that they are not correct and that's why your Arduino fails probably. Try getting these values (from your notebook or from the school's network admin) and specify them explicitly on the Ethernet.begin() method.

[Reply #10 on:]

As Pylon said it is highly likely to be your network configuration. If your network admin at school told you the addresses were in  10.2.X.X then the mask is likely to be 255.255.0.0, but you definitely need to find the gateway address and put both of those pieces of information into the Ethenet.begin.

Paul

[Reply #11 on:]

the school's IP is a 10.2.XX.XX

Which means you are definitely going through a proxy of some sort. 10.x.x.x is a private network address range. It can't be used on the public Internet without network address translation.

[Reply #12 on:]

@Nantonos: NAT (network address translation) is not the same as a proxy. The private address just tells us that he's going over some kind of router which is doing the address translation. Sure there's still the possibility that they installed a transparent proxy but that's not very likely when they have an automated proxy setup.

@OP: Could you check the network parameters and tell us if you succeeded?

[Reply #13 on:]

Sorry have been terribly busy with work.

Anyway the school's IT services suggested using a 3G dongle instead if the problem persists (they have no idea how to solve it)

i'll look for my telecom lecturer and ask if he has any idea how to solve the problem.

In the mean time, pylon and countrypaul, i shall take your advice and ask them to provide the network mask and include into the ethernet.begin and see what happens.

Let you guys know if it works or not!

cheers!

[Reply #14 on:]

If your school uses some atrocity like Cisco Clean Access for authentication, forget about using the arduino on that network.

-j