Email attachments from arduino as an SMTP client

Thanks.. This suggestion was valuable

I haven't tried sending an attachment, but sending plain text is pretty straightforward.

My problem was that I was trying to use a GMail account, and it needs TLS, something Arduino can't. But hopefully I found that Yahoo Mail doesn't need cipher.

Function would be something similar to:

// Hello World! Example: mail from foo@yahoo.com to bar@fakemail.com
client.connect("smtp.mail.yahoo.com",25);
client.print("HELO\n");
client.print("AUTH PLAIN\n");
client.print("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n");  // x: base64 encoded login: \0foo@yahoo.com\0password
client.print("MAIL FROM:<foo@yahoo.com>\n");
client.print("RCPT TO:<bar@fakemail.com>\n");
client.print("DATA\n");
client.print("From: \"Foo\" <foo@yahoo.com>\r\n");
client.print("To: bar@fakemail.com>\r\n");
client.print("Subject: Buy Cialis\r\n");  // xD
client.print("\r\n");
client.print("Hello World!\r\n");  // Lines of text
// ......................................
client.print(".\r\n");
client.print("quit\n");
client.stop()

Sending photo attachments is not a good task for the Arduino. The ATmega328 is a microcontroller not a microprocessor. Encoding, attaching, and sending a 50kb file through the poor little ATmega328 probably isn't worth the trouble. There are a lot of limitations you'll have to solve with very creative solutions.

Now, if you are trying to do this under the category of "Let's see if it can be done" that is a different story. If you are trying to accomplish any other goal (like build a system of automated webcams) the Arduino isn't a good choice.

My main objective is to transfer pictures from the JPEG cam that I posted above and I wanted to do it via ethernet shield/ arduino black widow shield. I am looking into best methods to do this.

Now that I posted here, I am able to gauge the possibilities before I hit a dead end. From what I read, the camera that I posted in my previous thread generates images from 3KB to 9KB. I do not want my arduino to be tethered to a computer via USB.

Will something like make sense for a picture transfer?

He has installed Linux inside an Asus router. Will hosting a mail client inside the router be a more sensible solution?

some cam's already have email and ftp build-in like the Trendnet TV-IP110 (cheap as in like $49)

but your issue with that model and google is ssl - there might be other cams that support ssl.

You could also look into plug computers or routers that support open source firmware (I use a reflashed unslung nslu2) then you can use bash, c or php to do what ever you want.

I looked at trying to reflash the cams but way less info and way over my head to try and be the first to do it - when compared to plug/router/nas reflashing there's a ton of info for those devices.

Am I invisible? :blush:

[quote author=James C4S link=topic=67701.msg498070#msg498070 date=1311777685]
Sending photo attachments is not a good task for the Arduino. The ATmega328 is a microcontroller not a microprocessor. Encoding, attaching, and sending a 50kb file through the poor little ATmega328 probably isn't worth the trouble. There are a lot of limitations you'll have to solve with very creative solutions.[/quote]
Pardon? What is the trouble? What limitations?

This would be how to base64 encode & send to the SMTP server a file named picture from a SD:

static const char cb64[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
void encodeblock(unsigned char in[3],unsigned char out[4],int len) {
  out[0]=cb64[in[0]>>2]; out[1]=cb64[((in[0]&0x03)<<4)|((in[1]&0xF0)>>4)];
  out[2]=(unsigned char) (len>1 ? cb64[((in[1]&0x0F)<<2)|((in[2]&0xC0)>>6)] : '=');
  out[3]=(unsigned char) (len>2 ? cb64[in[2]&0x3F] : '=');
}
void encode() {
  unsigned char in[3],out[4]; int i,len,blocksout=0;
  while (picture.available()!=0) {
    len=0;
    for (i=0;i<3;i++) { in[i]=(unsigned char) picture.read(); if (picture.available()!=0) len++; else in[i]=0; }
    if (len) { encodeblock(in,out,len); for(i=0;i<4;i++) client.write(out[i]); blocksout++; }
    if (blocksout>=19||picture.available()==0) { if (blocksout) client.print("\r\n");  blocksout=0; }
  }
}

Interesting problem. Obviously, the hardest part is to build the SMTP client state machine that implements something like section 4.2 in RFC 3030 (see link in reply by ManicDee). Assuming your SMTP server supports chunking, then all you have to do is the following:

  1. Get the binary data of the image file as a stream of octets (bytes)
  2. Invoke the state machine and follow the protocol (plus whatever cleartext auth needed) up until the BDAT part
  3. Issue a BDAT for the size of the binary data (or send them in chunks)
  4. Blast the binary data across the wire.
  5. Issue a BDAT 0 LAST
  6. Wait for confirmation of all data chunks, issue QUIT and close the connection.

You should be able to find C-based SMTP client code in the open-source realm and adapt it for the Arduino. In any case, sounds like quite a bit of work. Also, I would imagine it would be very, very hard to build an SSL stack on the Arduino.

Just my 2 cents' worth from a backseat commentator. :slight_smile:

There's no state required, SMTP is a very simple protocol and you can pre-program the entire client side of the conversation, only waiting to see "OK" from the server and dropping the connection if anything looks different to what you expect.

The only problem with trying to use the Arduino for SMTP will be those servers that insist on using SSMTP or SMTP/SSL. Base64 encoding is simple. SMTP conversation is trivial. Ethernet communication is a little harder. I expect SSL requires more data in the form of keys to be held in memory than the Arduino has memory. That might make an interesting experiment for some time in the future.

The obvious workaround is to have a plain-text SMTP server living on the network that the Arduino will connect to, which then forwards the email to wherever it's supposed to go. Of course there are probably "better" ways of getting data off the Arduino, but SMTP is certainly a convenient one (the message goes straight from the Arduino to your mail inbox) and IMHO is the simplest conversation to handle. Other options include FTP or HTTP, but both those protocols are far more complex than SMTP. FTP requires multiple connections and requires the Arduino to listen for incoming connections - no thanks! HTTP will require Base64 encoding, and involves more header generation and a more complex conversation.

I'll be fiddling with my Arduino tonight, let's see if I can't get a simple Base64 encoder working. Of course I'm currently limited to encoding about 400 bytes of data since I don't have an SD card interface yet (it's in the mail). So 400 bytes for input leads to about 530 bytes of output, which ends up leaving 70 bytes of RAM free (on my Uno I'll have more, but you might be using something different). With an SD card (or other external storage), the Base64 encoding operation only needs to use 7.25 bytes of RAM (3 bytes input, 4 bytes output, 2 bits for length of input), "some" memory for an input buffer, "some" memory for an output buffer, and lots of reading and writing over the SD Card interface to fill the input buffer and empty the output buffer.

And then there's the "clever" option of having one buffer for input and output, with the input starting 1/4 of the way through the buffer, overwriting the buffer with the output as it is generated (since we don't need the input data that we've already encoded).

There's also the question of how to get a camera, an Ethernet shield, and an SD Card shield connected to the Arduino at the same time. OH THE WIRING! OH THE HUMANITY!

Razorblade:
Am I invisible? :blush:

Seems like yes, I am.

Is this a joke? Are you really not reading me? Why the h**l are you still talking about how difficult it would be, etc... when I've written functional examples with Arduino of both sending e-mail and base64 encoding, and demonstrated that with 25 lines of code it's all done?

Razorblade:
Is this a joke? Are you really not reading me? Why the h**l are you still talking about how difficult it would be, etc... when I've written functional examples with Arduino of both sending e-mail and base64 encoding, and demonstrated that with 25 lines of code it's all done?

No you're not and yes indeed you have. Apologies for missing that post.

Sorry for the anger, but I was getting worried.

This would be the whole program. Reads picture.jpg from microSD card, sends an e-mail with picture.jpg attachment:

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

byte MAC[]={0x,0x,0x,0x,0x,0x};  // Your MAC
File picture; Client client;

static const char cb64[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
void encodeblock(unsigned char in[3],unsigned char out[4],int len) {
  out[0]=cb64[in[0]>>2]; out[1]=cb64[((in[0]&0x03)<<4)|((in[1]&0xF0)>>4)];
  out[2]=(unsigned char) (len>1 ? cb64[((in[1]&0x0F)<<2)|((in[2]&0xC0)>>6)] : '=');
  out[3]=(unsigned char) (len>2 ? cb64[in[2]&0x3F] : '=');
}
void encode() {
  unsigned char in[3],out[4]; int i,len,blocksout=0;
  while (picture.available()!=0) {
    len=0; for (i=0;i<3;i++) { in[i]=(unsigned char) picture.read(); if (picture.available()!=0) len++; else in[i]=0; }
    if (len) { encodeblock(in,out,len); for(i=0;i<4;i++) client.write(out[i]); blocksout++; }
    if (blocksout>=19||picture.available()==0) { if (blocksout) client.print("\r\n");  blocksout=0; }
  }
}

void setup() {
  Ethernet.begin(MAC); delay(2000);
  pinMode(10,OUTPUT); SD.begin(4);
}

void loop() {
  picture=SD.open("picture.jpg",FILE_READ);
  client.connect("smtp.mail.yahoo.com",25);
  client.print("HELO\nAUTH PLAIN\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\r\n");  // x: base64 encoded login: \0foo@yahoo.com\0password
  client.print("MAIL FROM:<foo@yahoo.com>\nRCPT TO:<bar@fakemail.com>\nDATA\nFrom: \"Foo\" <foo@yahoo.com>\r\nTo: bar@fakemail.com\r\nSubject: Picture attached\r\n");
  client.print("Content-Type: image/jpeg; name=\"picture.jpg\"\r\nContent-Disposition: attachment; filename=\"picture.jpg\"\r\nContent-Transfer-Encoding: base64\r\n\r\n");
  encode();
  client.print("\r\n.\r\nQUIT\n");
  client.stop();
  picture.close();
  while(1);
}

Takes less than 30 seconds to send a 50 KiB picture.

Thanks for sharing RazorBlade! It's quite interesting to see that the Ethernet shield does not have a fixed MAC address which is really weird and makes it susceptible to MAC collissions (though I doubt anybody is doing MAC-based routing much anymore). What will probably happen is that we end up with everyone using all 00-00-00-00-00-00 as their MAC. I wonder if it might be an issue if you have multiple devices with the same MAC address on the same subnet though.

Also, does the Ethernet shield have existing libraries for DNS lookup and all that?

Finally, what's the bottleneck in this system? Ethernet is really fast so why would sending a 50KB picture take 30 seconds?

Sorry for hi-jacking this thread with n00b questions.

Finally, what's the bottleneck in this system? Ethernet is really fast so why would sending a 50KB picture take 30 seconds?

Not ethernet on the Arduino. Besides, this thread started with sending the picture using a cell phone module.

Oh cool, thanks Paul!

primate:
What will probably happen is that we end up with everyone using all 00-00-00-00-00-00 as their MAC. I wonder if it might be an issue if you have multiple devices with the same MAC address on the same subnet though.

'Official' Ethernet shield comes with a sticker with an unique MAC written. It starts with 90-A2-DA-
The problem with multiple Arduinos is the same hostname "WIZnet", but it can be changed in the Ethernet library (Dhcp.h).

primate:
Also, does the Ethernet shield have existing libraries for DNS lookup and all that?

Yeah. I'm usign Arduino 1.0-beta1, which comes with DNS and DHCP support.

primate:
Finally, what's the bottleneck in this system? Ethernet is really fast so why would sending a 50KB picture take 30 seconds?.

Getting an IP (~4 seconds), reading each byte from the SD (~40 microseconds each: 6 s), SMTP traffic and load.... don't know exactly.

PaulS:
Besides, this thread started with sending the picture using a cell phone module.

? O_o

Sairam:
I am aware of the fact that it is possible to send emails to the SMTP server using the arduino ethernet shield. I want to send picture attachments along with the mail.

can you use the code above that razor blade posted above for txt files?

i placed the code with .txt instead of jpgs

but when i run it i get this

connected
220 to5email2.gprs.rogers.com ESMTP server (InterMail vM.7.09.00.04 201-2219-102-106-20090410) ready Fri, 12 Aug 2011 13:28:53 -0400
250 to5email2.gprs.rogers.com
250 Sender camarduino@gmx.com Ok
250 Recipient cam_the_man007@hotmail.com Ok
500 Command unknown: 'CONTENT-TYPE:;'
500 Command unknown: 'CONTENT-DISPOSITION:'
500 Command unknown: 'CONTENT-TRANSFER-ENCODING:'
500 Command unknown: ''
500 Command unknown: 'BWVVDYBTZW93IG1LBW=='
500 Command unknown: '.'
221 to5email2.gprs.rogers.com ESMTP server closing connection

any insight?

If I remember correctly, I believe any Headers (starting from CONTENT-TYPE) is actually part of the mesage content, and needs to come after the DATA command - but I must admit I'm a bit rusty on this.

Seems like you are only printing the responses from the server. It would help diagnostics if we can also see the outbound SMTP commands to the server too.

This is the code i am running with the headers after data, which sends an email but no attachment just places those lines of code in the email body.

the results i posted above is when the "DATA" line is removed and the subject

#include <Ethernet.h>
#include <SPI.h>
#include <SD.h>



byte mac[] = { 0, 0, 0, 0, 0, 0 }; // Arduino's artificial mac address
byte ip[] = { 0, 0, 0, 0 }; // my ip
byte server[] = { 0, 0, 0, 0 }; // my smtp server ip
int time = 5000;
int wait = 2000;

static const char cb64[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

File datatxt;
Client client(server, 25);


void setup()
{
 delay(time);
 
 Ethernet.begin(mac, ip);
 Serial.begin(9600);
 delay(1000);
 
 pinMode(10,OUTPUT); 
 SD.begin(4);
 
 Serial.println("connecting...");
 
 if (client.connect()) {
                        
                           
                        Serial.println("connected");
   
                        client.println("HELO itismeletschat"); /* say hello (statement after helo is needed but irrelevant)*/
                          delay(wait); /* wait for a response */

                        client.println("MAIL From: camarduino@gmx.com"); /* identify sender, this should be the same as the smtp server you are using */
                          delay(wait); /* wait for a response */

                        client.println("RCPT To: cameroncollie@gmail.com"); /* identify recipient */
                        delay(wait); /* wait for a response */

                        client.println("DATA");
                          delay(wait); /* wait for a response */
                        
                        datatxt =SD.open("data.txt",FILE_READ);
                        client.println("Subject: Picture attached");
                        client.print("Content-Type: txt; name=\"data.txt\"\r\nContent-Disposition: attachment; filename=\"data.txt\"\r\nContent-Transfer-Encoding: base64\r\n\r\n");
                        
                        client.println("."); /* end email */
                 
                        client.println("QUIT"); /* terminate connection */
                        delay(wait); /* wait for a response */
                        client.stop();
                        datatxt.close();
                        
  
 } else {
   Serial.println("connection failed");
 }
}

void loop()
{
 while (client.available()) {
   char c = client.read();
   Serial.print(c);
 }
 
 if (!client.connected()) {
   Serial.println();
   Serial.println("disconnecting.");
   client.stop();
   for(;;)
     ;
 }
}




void encodeblock(unsigned char in[3],unsigned char out[4],int len) {
  out[0]=cb64[in[0]>>2]; out[1]=cb64[((in[0]&0x03)<<4)|((in[1]&0xF0)>>4)];
  out[2]=(unsigned char) (len>1 ? cb64[((in[1]&0x0F)<<2)|((in[2]&0xC0)>>6)] : '=');
  out[3]=(unsigned char) (len>2 ? cb64[in[2]&0x3F] : '=');
}

void encode() {
  unsigned char in[3],out[4]; 
  int i,len,blocksout=0;
  
  while (datatxt.available()!=0) {
    len=0; 
      for (i=0;i<3;i++){ 
            in[i]=(unsigned char) datatxt.read(); 
                if (datatxt.available()!=0) len++; 
                      else in[i]=0;
      }
      if (len){
          encodeblock(in,out,len); 
           for(i=0;i<4;i++) client.write(out[i]); 
                blocksout++; }
      if (blocksout>=19||datatxt.available()==0){ 
          if (blocksout) client.print("\r\n");  blocksout=0; 
      }
   }
}

Hmmm, the only thing I can think of is that \r might not be an allowed character in the SMTP protocol, so instead of \r\n (which is CR-LF, where CR is only relevant for screen display), you can use \n only. Try breaking down the combined lines in the one print() call into separate println() commands and you might be able get more meaningful diagnostics.

Just my 2 cents' worth.

Well i just tried that code again,
it decided to work this time, now i just need to send the contents of the txt file, it didn't have any contents.
but i can send a txt file.