I need to authenticate the process of serial connection between two Arduino by using a challenge-response authentication mechanism.
I'm going to do it on Arduino Uno. It is a university project,
Note: Because of the comment that I have here I would like to say something. Again and again, I'm not asking for someone to do this for me. I need someone to guide me to the right way to do it. sometimes, when you don't have the experience you waste much time looking for the wrong things, and that I've been done during the last 6 weeks. I'm stock now
What are you actually asking? If you want advice or help, in my experience of forums like this you're more likely to get help if you show in detail what you've done so far and explain where you got stuck. Then ask for help specifically on "this" or "that".
And forgive me for saying so, if it's a "university project" should you not be able to do this without resorting to a hobbyist forum?
What are the requirements on the challenge-response authentication? What kind of standard do you have in mind, or can it be a self-made (and relatively simple) solution? Could you describe the application a bit more in detail?
It is, don't worry. But many of us are wary of the situation described as "teacher gave us homework, sounds difficult and I don't want it - you guys on the internet do it for me now". I don't think that's what we have here.
Then again, I've come across several threads that were definitely homework questions that received ample help. And I've also seen threads with legit university projects that were not homework with questions like this one here. Unless indicated otherwise, let's assume the best...
You are connecting two Arduino UNOs using Serial Port, probably, as per following diagram of Fig-1. Now, what you want for the UNOs to do? Will they exchnage data between them? For example: UNO-1 will send this message: "Hello! I am UNO-1 Sender." to UNO-2. In respones, UNO-2 will send this message to UNO-1: "Yes! I am UNO-2 Receiver."
What an amazing picture that you've attached, could you please mention the resource of this picture. also, the processing that you've mentioned, is so helpful to me to understand the process.
Honestly, I'm still looking for the authentication method. I've done some research to define one, but I'm stuck without any.
Some common way to authenticate messages between devices in the Internet is the used of AES-CMAC or AES-GCM. There must be software libraries for these two encryption protocols, albeit will be slow on the UNOs 16MHz clock.
Give each side a public and private key, then one side sends some random message to the other (encrypted) and the other replies back (double-encrypted using the message it just got, plus the senders public key). IF you want to both ends to authenticate each other, the first one can send a third message back to the other which is also double-encrypted.
If both sides were able to read and reply OK, authentication worked. If "garbage" came through, the "other side" is fake, or some replay attack is taking place.
Find a "big int" library for Arduino, and use "RSA" - that should easily fit in the limited space you've got - just remember to use RANDOM NUMBERS* as much as possible for what you encrypt (rsa is not secure to encrypt non-random content)
use timers and adc measurements to make something random.
I'm not sure where to start with this. Well, I guess I do know - I'd start with trying to get something to work without the authentication mechanism. So first just set up two Arduinos and get them to talk to each other over whichever protocol/connection is desired.