next topic »

[on:]

Wondering if others' experience with ThingSpeak jives with mine. I frequently see issues with data POSTs or with the web site. This morning, since I first checked an hour ago, both have been continually returning "502 Bad Gateway". Briefer issues that last maybe only a couple minutes and result in a lost data POST or two seem to be a regular occurrence.

I have been using Pachube for some time, quite successfully, and tried ThingSpeak just out of curiosity. There are things I like about it, although like anything, it has its pros and cons. Thought I'd try it for my current project; at this point I'm pretty disappointed. I'd like to think that it's so wildly popular that they can't roll in server capacity fast enough, but given the lack of traffic on their forums, I fear it's gone moribund.

[Reply #1 on:]

From what I could tell, the outage was about seven hours. I posted an inquiry on the ThingSpeak forum and the admin confirmed that there was an outage.

[Reply #2 on:]

ThingSpeak is on the Slicehost arm of Rackspace, and as you may know, they are transitioning to Rackspace Cloud. This was our first outage this year and it didn't go unnoticed. 200 users joined since last week... just not an active forum since it's a separate login thru wordpress and who wants to join a new forum anyway?

Sorry for the trouble. We are at the beginning surge of users + developing simultaneously.

Now, on the 502 Bad Gateway error - this may be caused by a buffer on the Arduino Ethernet shield (Wiznet-based). Occasionally a character remains in the buffer from a previous response or the buffer is not ready. Our API is sensitive to the command. I can reproduce the error if the sketch does not flush the buffer occasionally. I can also have a Netduino Plus posting at 15 sec intervals and experience 0 bad gateways at the same time as a wiznet module.

Here's the chart: https://thingspeak.com/channels/9/charts/1?results=100 - the points are exactly 15 seconds apart.

We actually need a better sketch that makes it more stable for our environment.

[Reply #3 on:]

Appreciate the reply. I was not aware of the migration. If users are ramping up, then that's a good thing and I'm glad to hear it!

The only time I've seen 502 Bad Gateway was during the outage when I was also seeing it simultaneously in a web browser when trying to access the web site, so I figured the issue was not limited to my sketch, which had been working well. I am curious about the buffer condition you describe though, just so I can be aware. Is your example sketch that produces the error available?

I'm actually more concerned about posts that seem to get lost even though status 200 OK is returned from the server. I see you also replied to my inquiry regarding that, thanks, good to know, but it raises more questions as well. I've seen this happen more often than I would like with tweets. I suspect that it also happens with regular data posts, but I haven't actually trapped any evidence of that; will advise if and when I do. My application is for home monitoring and losing an occasional data post actually isn't a huge deal since I usually send one per minute. Losing a tweet could be a bigger deal if it represents an alarm condition.

Thanks again for your time and for your reply. One final question, where would you direct ThingSpeak users who want to participate in an active online forum?

[Reply #4 on:]

Any place is fine to ask questions... users have been good about forwarding us to where the conversation is happening.

Tracking down info on the Duplicate Messge error is tricky: http://stackoverflow.com/questions/4874532/twitter-error-could-not-post-tweet

I have not found it spelled out in the Twitter API, but it does exist.

For example, if you send, "The temperature is 56" and "The temperature is 60" one message will likely get disregarded. You can even add time stamps or random numbers, and will still bounce - they have some matching detection. You are safe, I think every 3-4 hours if the content is the same.

If you keep the Serial Monitor window open on the Arduino sketch, you will see what gets reported back. That's where you might see Bad Gateway or a 403 error.

[Reply #5 on:]

Any place is fine to ask questions... users have been good about forwarding us to where the conversation is happening.

Tracking down info on the Duplicate Messge error is tricky: http://stackoverflow.com/questions/4874532/twitter-error-could-not-post-tweet

I have not found it spelled out in the Twitter API, but it does exist.

For example, if you send, "The temperature is 56" and "The temperature is 60" one message will likely get disregarded. You can even add time stamps or random numbers, and will still bounce - they have some matching detection. You are safe, I think every 3-4 hours if the content is the same.

Wow, on the one hand I get suppressing spam, but as far as the IOT, that is an exceedingly bad idea on Twitter's part, automated monitoring messages will by definition tend to be very similar! Maybe it should be a user-configurable parameter, whether to filter or not. I'd even be OK with it being on by default.

If you keep the Serial Monitor window open on the Arduino sketch, you will see what gets reported back. That's where you might see Bad Gateway or a 403 error.

Yes, I am quite aware, like I said, I was seeing the 502 error simultaneously from www.thingspeak.com in my web browser, and being returned from the server in response to POSTs. The examples I posted in the ThingSpeak forum were copied from the serial monitor. BTW, I wrote my own sketch, which is why I asked for a pointer to the one you referenced that would reproduce the 502 error. Not sure mine is experiencing that.

[Reply #6 on:]

Your sketch is probably fine. If you had the error in the browser and in the serial monitor, then it was because of the outage yesterday.

Let us know if it continues.

[Reply #7 on:]

Sure will, haven't seen it since, though.

I did experiment with duplicate tweets, by just posting them manually on twitter.com. I also found an item regarding duplicates on Twitter's help center which indicates that tweets have to be exactly the same to be rejected. My testing is consistent with this, duplicates are rejected, but changing even one character will allow the tweet be posted.

I suppose it's possible that manual posts are treated differently than those sent through an API, and the help center item doesn't define "recent", so I've sent an inquiry to Twitter support, I will share what I learn here.

[Reply #8 on:]

According to Twitter support, tweets are not filtered unless they are identical, and this applies whether they're entered manually via the web site or if they come in via an app like ThingTweet.  So adding a timestamp or a unique serial number to each message should ensure everything gets posted.

They said there wasn't a specific time interval after which a duplicate tweet would be allowed, but didn't explain further. Maybe they compare against the last n tweets or something.

I've added timestamps and I'm assuming at this point that the issue is resolved; continued testing will tell for sure. I did at one point inadvertently send two tweets only milliseconds apart, and the second one got lost, but that could be for several reasons and it seems like an unreasonable thing to do anyway, so now I'm ensuring that doesn't happen.

Appreciate the input, ThingSpeak!