Ethernet sheild and GSM module as an SMS web router

Hi I am trying to make an arduino into an SMS web router
I came up with this idea based as a way of utilizing the vast amount of text messages I have available on my mobile contract that I just don’t use, but I’ll get to that in a bit. I’ve done a few projects and built a small server with the Ethernet shield but I’m not that proficient with network protocol.
I’ve had a quick read through the forum and I couldn’t find anything that answered my questions if a forum thread exists that covers this subject that I’ve missed I would be grateful if you could point me to it.

What I want to do is to enable remote access to my network, using the GSM module to transmit data as SMS messages that are then reassembled at the other end with the arduino and Ethernet shield acting like a router that reassembles the data. The way I find this easiest to understand is to think of two of these routers acting like a bridge within my network with one module connected to my LAN setup and the other going with me, then I could access it anywhere I go that has a mobile signal (like my own privet internet ).

My network isn’t connected to the internet it’s a small LAN setup I primarily use for testing purposes
I am using the standard Ethernet shield with a micro SD card port, an arduino uno R3, and a GSM/GPRS shield
http://www.ebay.co.uk/itm/221176159800?_trksid=p5197.c0.m619
http://www.sainsmart.com/sainsmart-ethernet-shield-w5100-for-arduino.html

The thing that I need help on first is, is the arduino Ethernet shield even able to act in that manner, if so has anyone got any advice on where to start with the coding,. If it is not possible with the Ethernet shield than is it possible to create an invisible link in the network that simply forwards data anonymously appearing as a link straight to my computer.
Also does anybody have any recommended reading that could help with networking with the arduino? thanks.

Did I understand you correctly, you want to have such a device on both ends? And you want it to transfer the whole traffic like a router would do? I'd guess this is not feasible. You'll need almost the whole SMS content to transmit only the header of such a packet, making this extremely inefficient. In the best case you need about 32 bytes for the header, in 7bit characters of the SMS this is about 40 characters. This is without any application level protocol stuff and without any optional header content. You then need additional sequence numbers to re-assign the SMS messages to complete packets. So you end with a payload of less than 100 bytes per SMS.

If you solved that problem the Ethernet shield poses the next one. You're not able to produce low level packets there, ARP and stuff like that is handled internally.

yes I’ve been working on the theory of developing a transfer protocol to exchange data via the SMS system I’ll outline it below but I'm still working on it the thing to remember is that I’m not looking to send the information in one message but rather many smaller ones that are then reassembled

for the sake of simplicity I will give my example with respect to a single packet transfer ignoring the network protocol side of things for now, so focusing on the GSM data transfer and reassembly.

I will refer to my two units as the sender and receiver though bi directional communication is the overall goal and if implemented network protocol would enable this.
when the packet is sent to the arduino (from the Ethernet connection) it will evaluate its size and divide it up into smaller chunks that are appropriately sized for the SMS message this could be a large quantity of chunks, these will be saved into a matrix so that they can be referenced.

the arduino will then send the packets using the SMS attaching the packet number and the number of packets, when the SMS message gets to the other end the Receiver will place the contents of the message into a matrix a set amount of time after it stops receiving data it will evaluate whether or not all the packets have been received using the number of packets and looking at the matrix contents to see if any parts are empty.

if a part is missing the receiver will send a set of 3 time date stamped messages to indicate any parts that are missing and to resend them. The reason that 3 messages are used is to help ensure that at least one gets through when the sender receives this message it will store the time date on the packet so it knows that it has serviced this request and will then resend the indicated parts. the time date that has been serviced will be stored until it is deemed to have passed the timeout point, requests will only be considered valid for a set amount of time from sending so that any that get held up don’t affect the system down the line.

when the receiver has recovered all of the packets it will reassemble them into the intended form and feed them onto the Ethernet shield at which point it will empty the matrix and return a message to the sender to indicate that it can clear it’s contents as well and continue on to the next set of data.
again this is a rough outline and will be refined and reworked the thing I need real help with is the Ethernet shield.

This is a rather simple system. What will you do with the incoming network traffic while you're sending one packet? Keep in mind that one packet may be split into up to 15 SMS messages. And you have currently no acknowledge message in your concept. This would mean that the resend SMS may arrive when the next few packet are already sent. On an Arduino you haven't enough memory to store multiple packets (sending and receiving) so you have to use at least a Mega2560 to be able to send one packet at a time bidirectionally and you need an acknowledge message then.
You may imagine that the number of SMSes you have to send get incredibly high. Does you contract include an unlimited number of SMSes and are you sure that it stays unlimited if you send ten-thousands of SMS per day?

yes the concept outline as it stands is fairly crude and will need improvements such as conformation messages and priority handling not to mention the potential for multiple simultaneous incoming requests, message integrity checking and exception handling. But I think that it might be an idea to make this a fairly rigid one way system to start with to try and show that the individual parts for this project can perform in the capacity required and if not I will have to engineer a part that can.

I think that you’re defiantly right about upgrading to the mega or possibly the due though I’ll need to look more into that. though even with that I am considering using the micro-SD card on the Ethernet board to act as a buffer to handle incoming packets so that when a packet comes in the first thing the arduino does is it writes it straight to the SD card and only loads them when it’s ready to handle them.

with regards to the number of texts being sent I’m toying with a few ways to reduce the number required though the idea of this was to use a vast number of them anyway with an unlimited contract that gives you a silly amount to play with.

Edit:
ok so here are a few ideas I’m looking at mainly to reduce the load on the arduino and the SMS system I’m looking for some feedback on them such as have you seen them implemented before, is there a better way that you know etc.

Using a predefined reference list stored on the SD card to abbreviate commands and list references.
This idea was born from the idea of writing my own library for this project (which I might still do) except instead of defining new functions the reference list would be used to either clump or abbreviate existing functions under a single heading. The same concept could be used to convert device name references into ip addresses and mac codes. This idea won’t lower the data being sent massively but will reduce the amount of miscellaneous data being sent.

Using an SD card as a buffer
I mentioned this before but I’m considering using an SD/micro-SD card to store data until the arduino can handle it.

Prioritization
This is something that I think is going to be essential to this project succeeding without a priority based management system data will inevitably build up on whatever side of the arduino is referenced second in the code because it will be servicing the first instance in addition to this I think that it may be an idea to attach a device priority to the system so that if a device has allot of data waiting to be sent or is waiting for allot of data it is reflected in the actions of the other as well.

Implementing a dedicated real time clock system
The reason for this is that I plan on implementing a time out on messages based on when they are sent this is because the SMS system isn’t the most reliable and sometimes messages are delayed or are lost completely.
Anyway if a resend message were delayed it could result I packets being resent that aren’t needed or a reference to data that doesn’t exist that could conflict with exception handlers and integrity checking. So to help ensure the reliability and accuracy of the timeout features I will need accurate clocks on each unit.

I’ve also been doing some reading on connecting using the Ethernet shield, well network protocol in general and I reckon that I can make this work it might take some more work than I had anticipated but when has a project ever run smoothly. I’ll give an update on this later however I’m quite busy this week so discussion I can do in my free time but I’m unlikely to get much actual testing done.

Edit: 19/6/13

While I am fairly sure that this topic is dead at this point for anyone who still has an interest in following my progress I have produced most of the documentation relating to the bridging protocol, I am contemplating writing this into a library so that I can call upon functions that I can later refine and rework as necessary and not need to update all the related code in my sketches.

The next step is to work out how to get the Ethernet shield to handle the incoming data, this is going to require a greater examination of the relevant data sheets and libraries to determine what needs to be done. If anyone is familiar with this I would appreciate any assistance. at this point I am confident that this is possible.

If anyone is interested in seeing my work thus far feel free to ask.