[RESOLVED]Submitting google form using GPRS shield: error

Hi all!

I am having issues submitting an entry to a Google form using Arduino GPRS shield.

The problem does not seem to be the way that I have constructed the URL, because when I use it in my browser it successfully posts to the form, and I can view the entry in Google spreadsheet. The Arduino establishes connection to the server using if (client.connect(server, port)) successfully, so let me just show you the part of the code where I think the problem lies:

client.print("GET https://docs.google.com/forms/d/[my formkey goes here - keeping that private]/formResponse?ifq&entry.1554908416=HelloWorld&submit=Submit");
    client.println(" HTTP/1.0");
    client.println("Host: docs.google.com");
    client.println();

And this is the response error that I receive:

HTTP/1.0 400 Bad Request
Content-Type: text/html; charset=UTF-8
Content-Length: 1419
Date: Tue, 24 Jun 2014 19:09:51 GMT
Server: GFE/2.0
Alternate-Protocol: 80:quic

<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 400 (Bad Request)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/errors/logo_sm_2.png) no-repeat}@media only screen and (min-resolution:192dpi){#logo{background:url(//ww.google.com/images/errors/logo_sm_2_hr.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//ww.google.com/images/errors/logo_sm_2_hr.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:55px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>400.</b> <ins>Thatâ??s an error.</ins>
  <p>Your client has issued a malformed or illegal request.  <ins>Thatâ??s all we know.</ins>

I have tried using the POST command instead of GET, which then returns an error saying that I require Content-Length parameter (which I did include but didn't work)

my work is based on this guide (using GPRS shield instead of Ethernet):

please note that HTTP/1.1 vs HTTP/1.0 yield similar results, and that I don't think I'm doing anything devastatingly incorrect, it feels like I am one or three lines of code incorrect. Can anyone help please!?

Thanks in advance for the assistance :slight_smile:

client.print("GET https://docs.google.com/forms/d/[my formkey goes here - keeping that private]/formResponse?ifq&entry.1554908416=HelloWorld&submit=Submit");
That's a no-can-do highlighted there.

@PaulS you beauty!

Not fixed, but something different! See new 'error' below (I'm going to sensor potential confidential data). New entry "Eureka" was not posted in Google Spreadsheet.

HTTP/1.0 301 Moved Permanently
Content-Length: 314      
Content-Type: text/html; charset=UTF-8
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Tue, 24 Jun 2014 19:44:23 GMT
Location: https://docs.google.com/forms/d/[super-secretive-formkey!]/formResponse?ifq&entry.1554908416=Eureka&submit=Submit
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 80:quic
Proxy-Connection: close

<HTML><HEAD><TITLE>Moved Permanently</TITLE> </HEAD><BODY BGCOLOR="#FFFFFF" TEXT="#000000"><H1>Moved Permanently</H1>The document has moved <A HREF="https://docs.google.com/forms/d/[super-secretive-formkey!]/formResponse?ifq&amp;entry.1554908416=Eureka&amp;submit=Submit">here</A>. </BODY></HTML>

disconnecting.

That is HTTP-speak for "You're trying to access this using HTTP and I'm only going to let you access it using HTTPS.

HTTPS requires using SSL encryption which requires far more resources than your little Arduino has available; it's simply not feasible. This is the point that PaulS was steering you towards when he highlighted the fact that you're trying to use HTTPS.

Thanks @PeterH

Does this mean give up? I hope not!

In my original post I linked a tutorial which does an almost identical thing, but using the Ethernet Shield, not GPRS/GSM shield, so "surely" there is a way?

What I tried to do before-hand was post data to Xively.com (previously Cosm - IoT stuff) - which was successful, however data was in the form of a graph on the Xively.com site. I need it to be in spreadsheet/csv form ideally such that I can do maths/accounting on it. Google spreadsheets does allow for the function inside a cell =IMPORTDATA("URL") where URL is where Xively posts CSV data, but this function didn't work (typical!). I would rather cut out Xively entirely if need be, but it can be a messy solution to my problem

imbatronics:
Does this mean give up? I hope not!

PeterH:
it's simply not feasible

If you can access it without using HTTPS then that's fine. If you need HTTPS, it's not feasible using anArduino.

There might be some language trickery that I'm not catching here:

If you can access it without using HTTPS then that's fine. If you need HTTPS, it's not feasible using anArduino.

Are you saying (absolutely) that I do in fact need HTTPS (therefore not feasable)? Or are you supposing that there is a way for me to do it without? If so, I am all ears!

I am also unaware of the differences between what I am trying to do vs. what Boris Landoni is trying to do in the tutorial I linked in my original post. Would my using an Ethernet Shield - like he does - solve my problem? This is the URL where Boris' Arduino is posting to (note the https)

I have almost no experience in this field, so thank you for your patience :slight_smile:

Are you saying (absolutely) that I do in fact need HTTPS (therefore not feasable)? Or are you supposing that there is a way for me to do it without? If so, I am all ears!

You can NOT access that site directly using the Arduino.

What you could do is acquire your own domain and get it hosted. Then, you could run a script on your server, unsecured (that did not require HTTPS). You'd have that script collect the data, and then send it to google.docs, using HTTPS, using your credentials.

Thanks PaulS and PeterH for you help!

I still don't quite understand why Boris Landoni's (tutorial from previous posts) method works with WiFi/Ethernet shield, whereas mine does not. Do Ethernet/Wifi shields have SSL encryption dedicated IC's on there? He seems to be posting to https.

I need to be using the Arduino on campus, which might pose further complications because of proxies etc, but my phone can connect with proxy on WiFi so that might not be a problem.

Do Ethernet/Wifi shields have SSL encryption dedicated IC's on there?

No.

I still don't quite understand why Boris Landoni's (tutorial from previous posts) method works with WiFi/Ethernet shield, whereas mine does not.

Is there any proof that it does?

I need to be using the Arduino on campus, which might pose further complications because of proxies etc, but my phone can connect with proxy on WiFi so that might not be a problem.

It will be up to you to test your solution in your environment.

imbatronics:
Are you saying (absolutely) that I do in fact need HTTPS (therefore not feasable)?

All the evidence you have shown suggests that the site you're accessing is only accessible via HTTPS. You cannot access an HTTPS site using your Arduino. It doesn't work, will never work, you're wasting your time trying.

The example project you linked to accessed that site using HTTP (not HTTPS). I don't know whether that example worked then, and I don't know whether it works now. However, your sketch doing the same thing does NOT work because the site you're accessing requires HTTPS and I suspect you'd find the original sketch doesn't work either. It doesn't matter what sort of network interface you use, it isn't going to support SSL/TLS encryption so HTTPS is not available to you. There are plenty of small form factor PCs which could do what you're trying to do; the Arduino can't do it.

Is that absolute enough for you?

Thanks again guys, really appreciate it!

I have found a work-around over here:

https://productforums.google.com/forum/#!topic/docs/f4hJKF1OQOw

I'm going to give it a try, hopefully it works.

I also bought a cheap Ethernet Shield today - so will try with that too, I figured it is fun to have even though is not going to make things better for my problem. If I get things working I will make name include [RESOLVED]