NeoCat Twitter Library. Tweet fails with code 500

Hello Arduino Forum,

I'm using NeoCat's Twitter Library for Arduino (http://arduino-tweet.appspot.com/). I have previously used this without issue, until today when I've been getting the following error message:

tweet failed: code 500

According to the twitter developers site, this error indicates an internal server error (see https://dev.twitter.com/docs/error-codes-responses for more details).

I have tried tweeting with two separate Arduino devices. One an Arduino Uno R3 with Ethernet Shield; the other a Botanicalls Kit (http://www.botanicalls.com/. Both fail to tweet today. However, I did successfully tweet from the Botanicalls on 6/24.

Please let me know if you are having this same error or have any advice on the situation. Thanks in advance,

I am seeing the same thing. An Arduino here that tweets once per hour has not had a successful tweet for nine hours.

Thanks for the info Steph. I've been looking into the problem and I believe it's something to do with the arduino-tweet.appspot.com website exceeding it's quota. I wish I knew more about it.

As of right now, my device is able to tweet just fine. The problem appears to be solved.

My project's tweets started to work again last night, but it's down again as of an hour ago, with the status 500 again.

I remember a month or two ago, the arduino-tweet.appspot.com became very unreliable for 3 or 4 days, before it finally sorted out.

Thanks again Steph. My Arduino tweets aren't going through either. I guess we just have to wait and see.

My last tweet in my app is this https://twitter.com/easysoat

it was at 04:28:20 GMT -5

now my app, receives this:

HTTP/1.0 500 Internal Server Error
Content-Type: text/html
Set-Cookie: rack.session=BAh7AA%3D%3D%0A; path=/
Vary: Accept-Encoding
Date: Mon, 09 Jul 2012 20:21:20 GMT
Server: Google Frontend
Cache-Control: private

Error - com.google.apphosting.api.ApiProxy$OverQuotaException: The API call datastore_v3.RunQuery() required more quota than is available.failed : code 500

This twitter app works excellent, i don't know what to do know, i tweet a message to @NeoCat if he needs help for fixing this, all of you should do the same

Thanks for the info Mbuchelly. Good idea to send a tweet to @neocat. The arduino-tweet.appspot.com site appears to be working again, as my device has made three successful tweets today.

Not sure this is fixed. My tweets to http://twitter.com/vickisgarden1 are failing with code 503 for me.

Error
 

Over Quota

This application is temporarily over its serving quota. Please try again later.

tweet fail: code 503

Any ideas are most welcome!

You're right DrewC, the arduino-tweet.appspot.com isn't working again. Thanks for the info.

Looking at appspot.com, it is a platform for web applications running on Google's servers. arduino-tweet.appspot.com is a web app programmed by NeoCat to allow our Arduino devices to tweet. I think that maybe this web app is proving too popular for the free service offered by Google.

The page states:

Every Google App Engine application will have enough CPU, bandwidth, and storage to serve around 5 million monthly pageviews for free. You can purchase additional resources at competitive prices when you need them and you'll pay only for what you use.

It appears we Arduino Twitter users have used all of the free bandwidth provided by Google.

Any ideas for solutions?

@Gizmojoe, just saw this Twitter post on @NeoCat:

NeoCat ?@NeoCat
@mbuchelly Somebody seems sending too frequent requests. I rejected that and the server should recover in 5 hour. Thanks for your report.

6:58 PM - 9 Jul 12 via twicli · Details

So maybe this will recover sometime soon, but not sure how to manage over time if people are abusing the site. Perhaps a policy of some sort is in order, say, a limit on the number of tweets/day on this site. If you have a device that's really pounding the site then it's best to take it elsewhere, to a site that's purpose-built for that kind of sensor data stream. Thoughts?

  • Drew

If you have a device that's really pounding the site then it's best to take it elsewhere, to a site that's purpose-built for that kind of sensor data stream. Thoughts?

It seems to me that you should do this for any critical data. Once the novelty of being able to twee has worn off, you should examine WHY you are sending this data, and whether you need to continue doing so. If the answer is yes, doing it one someone else's dime is not playing fair.

PaulS:
It seems to me that you should do this for any critical data. Once the novelty of being able to twee has worn off, you should examine WHY you are sending this data, and whether you need to continue doing so. If the answer is yes, doing it one someone else's dime is not playing fair.

My project sends a status tweet once per hour, I hope it isn't excessive. But it has moved beyond the testing phase and has been 'live' for months. I would prefer a self hosted solution rather that relying on / using up Neocat's bandwidth.

Is there an 'easy to use' alternative that one could use self-hosted? A few months ago when there were similar problems I tried to find a simple replacement but came up with nothing.

Is there an 'easy to use' alternative that one could use self-hosted?

I use my Arduino to store data on a web server, calling PHP scripts that are located on the server to store the data in a MySQL database.

PHP and MySQL are relatively easy to set up, and the price is right. Of course, I have access to the server that hosts my domain, and can install whatever I need.

All that you need is your own domain, and the ability to execute scripts on the server hosting that domain. If the scripting languages include PHP, and mySQL is installed, great. If not, there are certainly plenty of ways to store the data in other formats, like flat files.

Is there an 'easy to use' alternative that one could use self-hosted?

That's a good idea Paul about using PHP and MySQL, but that does require having a server or computer setup. My tweeting Arduino devices standalone. We would also have to make significant changes to our existing Arduino setups.

Another idea would be to setup a clone of NeoCat's Arduino-Tweet site on Google App Engine. We'd need NeoCat's permission, and likely his/her help, and would have to create a new Arduino Twitter library specifically for the cloned site. Just a small group of us could use the clone site. This way we would only have to make minor changes to our Arduinos. What do you think of that idea?

What do you think of that idea?

Not much. I still think that you want someone else to pay for hosting your data. That is the part I can't understand. There is no such thing as a free lunch.

I have access to my own hosting equipment; I have a home server for private / low-bandwidth stuff, and I have a server at work which gives me all the hosting goodness I need. I've already experimented with sending data from my arduino project to my server and displaying it within a simple html / php page.

What I'd prefer though, is to just keep using twitter. It does the trick, it's convenient, and I don't have to reinvent the wheel.

Personally, I like GizmoJoe's idea. If Neocat's setup (the server side) was open source, I would try and set up a copy of it on my own server. That way I wouldn't be using Neocat's bandwidth. (I don't mind using Twitter's bandwidth, they monetize my tweets with ads and whatever else they stuff into my stream.)

As I mentioned earlier, there was a period of very inconsistent behavior a couple months ago that got me looking into other options. I made a note back then about a linux based command-line twitter client. I'm pretty sure I could utilize that by just reconfiguring how the Arduino sends its data, and setting something up on one of my servers to accept the data and pipe it into the command-line client.

When Neocat's system started working ok again, I stopped looking into the alternatives but it might be worth persuing that again now.

Cheers!

If you're interested setting on your own web app, it appears NeoCat's code for arduino-tweet.appspot.com is on GitHub. I don't understand if the code is open source or not. You could ask @NeoCat on twitter if you could use it. Here's the link:

Personally, I don't know much about setting up and hosting web apps, but I would consider trying it (with NeoCat's permission, of course). There is still the matter of modifying the Arduino Twitter library to support a server different than arduino-tweet.appspot.com, but I believe that could be done.

Arghhh...the app is down again, a victim of over-quota usage. We really do need to consider usage scenarios for our Arduino testing as long as some are abusing the privilege of the very nice setup NeoCat has created for us. Although it reset last night at midnight PDT (and will again in 27 minutes), it will surely be down once again by tomorrow evening. Scanning several Arduino-Twitter apps, I see that many (including mine) have built-in Twitter throttles to deliberately limit tweets to a reasonable max, either though rate/hr or total number in a given period. It would seem that if we all were willing to do this we might all enjoy the continued use of a great resource. Perhaps NeoCat could even put something in his Google App that could detect over-use and either de-authorize that connection or do some throttling ala Verizon or At&T who face similar issues with bandwidth hogs in their wireless services. Thoughts?

I agree with Drew that we need to be respectful of Neocat's web app and limit the amount of tweeting done by our Arduino devices. Also, I like the idea of throttling users that tweet too frequently.

I have two questions:

  1. How could we decide on an acceptable number of tweets a device is allowed per a given period of time?

  2. Is one anyone and experienced enough programmer to help Neocat setup the throttling that Drew has proposed?

I'm still looking into the idea of creating a clone of the arduino-tweet.appspot site as well. Thanks for your ideas and feedback.