problem with ethernet shield in only one site :S

hi, well I skiped my old project with my arduino mega for a while

so now I'm wanting to finish it... so the thing is, I need to enter on the site that do the tracking on my orders via post service... so for that I'm using an arduino mega + ethernet shield wich I did lot's of things, configuration and etc

so what happens is, I can do client stuff on any site (sparkfun, twitter, g1, anything) but not on the one I want to... wich is websro.correios.com.br... wich can track orders based on it's tracking number...

so using "nslookup" on my "cmd" pront or wireshark I saw the ip for the site is 200.252.60.71... so I made a normal http request wich works for any other site:

GET / HTTP/1.0 Host: websro.correios.com.br

but nothing works... I get nothing on serial monitor...

I get this:

connecting... connected

disconnecting.

well, I can open any other site using this stuff, I could open all that I tried to, I also can do the server aplication but not open this tracking site :S

so, if someone can help me, I would be very happy as I spent lot's of time on this yet and really don't know what to do next.

I tried using HTTP/1.1, using the entire request (wich I got from the request my browser sends to the site to get the answer) as well with a IE6 request (wich is a different request, not like the google chrome request or opera request) and lot's of other things...

another thing... there is another 2 servers wich are used by the post office guys... this one I talked about is only for tracking services, the other 2 ones (wich I found using nslookup) are for the main site of the post office site, www.correios.com.br, wich are: 200.252.60.80 and 200.252.60.24... so, using the board I can't also get data from this two servers :S

I can enter on all this 3 servers from the "cmd" telnet stuff sending the same thing... and I get all the data from the page...

the g1 site has a bunch of sites in the same server so, with the Host: server line on the request, I can enter on the site I want to

so everything works just fine, but not this websro.correios.com.br or www.correios.com.br sites :S

so if someone can help me, I would be very happy...

obs: sorry for bad english...

ah, another thing, I want everyone to understand that it's not a normal bad request stuff, because if I do a bad request on g1, like

GET / HTTP/1.0

it will give me some data back wich I can save on an .html file and open with my browser and I'm gonna see some "bad request, send one that I can understand" stuff or something

so, I really do not get anything back from it, it's just that on serial monitor

Connecting... Connected

disconnecting.

from both websro.correios.com.br and www.correios.com.br... so again, any help will be very very cool as everything that I learned about internet and http and ethernet or lan or anything was about 2 weeks of reading some stuff in order to put the board to work... and I mean, I never learned this things by my own or learn on university or anything before, it was just myself alone so probably someone here knows a LOT more than me and could solve this...

well I tried to ask this for the teacher on the university who works with this kind of stuff... but I expected to have a better answer from him but, I didn't like what I got about it, something like if what I'm doing is nothing accidentaly entering in some restricted area on the internet or something... but I don't think it's my problem here, because the websro.correios.com.br is a normal site... the extension with the tracking number should be but, it's not ... I mean, it's just put the address like this:

http://websro.correios.com.br/sro_bin/txect01$.Inexistente?P_LINGUA=001&P_TIPO=002&P_COD_LIS=RB110420549HK

so the "RB110420549HK" on the end is the tracking number and changing this I can change the order I want to check... so this should be a "restricted area" or something but it's not so... also, I tried with the hongkong post website and it works great... only this 3 servers can't be open till now by my ethernet shield :S

don't know what to do :S

thx

well as I can interface with the ethernet shield, if someone think this is a software troubleshooting, please move it to the software part if like to... but I'm not sure so I posted here

You need to snoop on the TCP session to see exactly what's happening - how to do this depends on the OS.

One thing I remember about HTTP when I was messing with it was that its complex and sending just a two-line request header is likely to cause problems - also it is vital to make sure line-endings are CRLF sequences everywhere.

It may be that the site is doing an HTTP redirect or specifying chunked encoding or doing Basic Authentication or some such complication - and only a direct snoop on the TCP stream will find this out.

Also after the request header and blank line are output they must be flushed so that a TCP packet actually gets sent - but you can't just close the stream because then you risk losing the response (this depends on the server logic I think...)

I would wager that you're breaking the HTTP spec in some way that matters for some but not all web-servers - and HTTP 1.1 is a very large spec. :(

well, you talking about the big request that my browser (for example) does when I connect to the page? something like this:

(this is my friend's request... he has a mac) GET / HTTP/1.1 Host: websro.correios.com.br User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

(and this is my request, I use windows) GET / HTTP/1.1 Host: websro.correios.com.br Connection: keep-alive User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.33 Safari/532.0 Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,/;q=0.5 Accept-Encoding: gzip,deflate,sdch Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

also there is a IE6 request wich is diferent from this two... and I tried to send the entire request a lot of times, even the IE6 one and my google chrome one... and I got nothing...

and about what you said... if that's the thing, why I can get on the page using only this on cmd:

telnet 200.252.60.71 80 GET / HTTP/1.0 Host: 200.252.60.71

also, the requests end with a '\n' character given by the "println()" function on the Client class... and this is essential, I mean, I would not be able to enter on any site without using this :S

so, somone advice?

obs: I really tought some "ethernet shield guru" or something would know this stuff or something... I really don't know a lot about ethernet stuff, HTTP or what, I learn something by reading stuff on the internet...

I get the following using a manual connection:

 tn websro.correios.com.br http
Trying 200.252.60.71...
Connected to websro.correios.com.br (200.252.60.71).
Escape character is '^]'.
GET / HTTP/1.1        

HTTP/1.1 400 Bad Request
Date: Thu, 04 Mar 2010 05:56:07 GMT
Server: Apache/1.3.41 (Unix) mod_perl/1.30
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1

169
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>400 Bad Request</TITLE>
</HEAD><BODY>
<H1>Bad Request</H1>
Your browser sent a request that this server could not understand.<P>
client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /<P>
<HR>
<ADDRESS>Apache/1.3.41 Server at 127.0.0.1 Port 80</ADDRESS>
</BODY></HTML>

0

Connection closed by foreign host.

Sending only a slight more complex request seems to get much better results:

> tn websro.correios.com.br http
Trying 200.252.60.71...
Connected to websro.correios.com.br (200.252.60.71).
Escape character is '^]'.
GET / HTTP/1.1        
Host: websro.correios.com.br

HTTP/1.1 200 OK
Date: Thu, 04 Mar 2010 05:58:21 GMT
Server: Oracle-Application-Server-10g/10.1.2.0.2 Oracle-HTTP-Server
Last-Modified: Mon, 23 Nov 2009 12:35:44 GMT
ETag: "435ab-a6e-4b0a81a0"
Accept-Ranges: bytes
Content-Length: 2670
Content-Type: text/html
X-Cache: MISS from 127.0.0.1

<script language="JavaScript">
(lots more)

hm, you used an arduino to do this? if you did, could you send me the code please?

if it’s a telnet connection or something made in a computer, is what happens when I do it from my computer using telnet, I can get nicely using telnet, but I get NOTHING with arduino + ethernet shield…

If you're talking about me, those are connections from telnet on a unix PC, not direct from an Ardunio... Do your sketch currently show EVERYTHING received from the net in some sort of debug window? That would be my next step...