ethernet shield to send http command

im trying to figure out how to send this http command to a device on my network that uses authentication. the command is

http://10.10.10.10/cgibin/configManager.cgiaction=setConfig&VideoInOptions[0].DayNightColor=0

the webserver will accept it with the username and password in the string. but i cant figure out how to get the arduino to send it with authentication. i get this error on the arduino

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Digest realm="Login to 4E01DD5PAFC1A58", qop="auth", nonce="960946880", opaque="540ecdf4f3a4209074f652938bcaf0853fb4424f"
Connection: close
CONTENT-LENGTH: 0

so i tried to see if i could figure out how to add authentication but its not working right. here is the code i have pieced together

#include <SPI.h>
#include <Ethernet.h>


byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
char server[] = "10.0.0.65";    // name address for Google (using DNS)
// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(10, 0, 0, 99); //ipaddr of arduino

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
 // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");
  Serial.println("connected2");
    // Make a HTTP request:
//    client.print("GET /Authorization: Basic ");
//    client.print("YWRtaW46QWFhYWFhYTE==@10.0.0.65:80");
//    client.println("GET /cgi-bin/configManager.cgi?action=setConfig&VideoInOptions[0].DayNightColor=0");
//    client.println("Host: 10.0.0.65");
//    client.println("Connection: close");
//    client.println();

  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make your API request:
  client.print("GET /Authorization: dIGEST ");
  client.print("YWRtaW46QWFhYWFhYTE=@10.0.0.65:80");
    client.println("GET /cgi-bin/configManager.cgi?action=setConfig&VideoInOptions[0].DayNightColor=0 HTTP/1.1");
    client.println("Host: 10.0.0.65");
    //client.println("Connection: close");
    client.println();

  } 
  else {
    // kf you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
//  if (!client.connected()) {
//    Serial.println();
//    Serial.println("disconnecting.");
//    client.stop();

    // do nothing forevermore:
    //while(true);
  
}

please help thanks

the webserver will accept it with the username and password in the string

What does that mean? You would normally access a web server using a browser.

Where in the string? What string? Why doesn't your code try to put the user name and password in "the string"?

PaulS:
What does that mean? You would normally access a web server using a browser.

Where in the string? What string? Why doesn't your code try to put the user name and password in "the string"?

in the web browser i can just admin:admin@10.10.10.10/request. when i dont use authentication and i send the request with the arduino i get a 401 response. im not sure how to send the http commands with authentication with the arduino

heres what im trying now. i dont get a response and it just dont work,

  if (client.connect(server, 80)) {
    Serial.println("connected");
client.println("Host: 10.0.0.65");
client.println("Authorization: Digest Realm YWRtaW46QWFhYWFhYTE==");
    client.println("GET /cgi-bin/configManager.cgi?action=setConfig&VideoInOptions[0].DayNightColor=0 HTTP/1.1");
    //client.println("Connection: close");
    client.println();

it looks like im trying to do basic authentication on a server thats using digest realm. however i cant find any support on how to make this work with digest authentication

i found this but it uses the ESP8266HTTPClient.h> library. how can i do this with the ethernet shield?

notsolowki:
i found this but it uses the ESP8266HTTPClient.h> library. how can i do this with the ethernet shield?

The process for setting up the authorization data is exactly the same. The process for adding the headers is different, but I'm sure you can figure out how to do that (since you already know how to add the Host header).

PaulS:
The process for setting up the authorization data is exactly the same. The process for adding the headers is different, but I'm sure you can figure out how to do that (since you already know how to add the Host header).

do you mean i can setup the digest authentication by setting up the header in the program i have now and i dont have to do any decrypting / comparing on the arduino? a thorough gogole search about arduino webclient and digest authentication didnt come up with much except for arduino web server.

do you mean i can setup the digest authentication by setting up the header in the program i have now

Yes, that is what I mean.

i dont have to do any decrypting / comparing on the arduino?

You can do the encrypting of the credentials anywhere, IF they are fixed. If they are not, then you need to do the encrypting on the Arduino.

There is no decrypting done on the client end. The server will compare the encrypted value supplied with the expected encrypted value, and allow, or disallow, access depending on whether they match, or not.

The supplied data is never decrypted.

PaulS:
Yes, that is what I mean.
You can do the encrypting of the credentials anywhere, IF they are fixed. If they are not, then you need to do the encrypting on the Arduino.

There is no decrypting done on the client end. The server will compare the encrypted value supplied with the expected encrypted value, and allow, or disallow, access depending on whether they match, or not.

The supplied data is never decrypted.

im looking at that ESP8266HTTPClient.h library trying to figure out how they are sending the header. the credentials will be fixed but i still dont understand how i can authenticate by printing the header with my program. i dont know what the header is suppose to look like

You could go look at the source code for this method:

 http.addHeader("Authorization", authorization);

But, it's pretty easy to guess what it does. Look at your Host header. It has a name, a colon and a value. So, it is highly likely that the above method's arguments are the name and value, and all it does is print them to the server, printing a colon in between and a carriage return and line feed after them.

PaulS:
You could go look at the source code for this method:

 http.addHeader("Authorization", authorization);

But, it's pretty easy to guess what it does. Look at your Host header. It has a name, a colon and a value. So, it is highly likely that the above method's arguments are the name and value, and all it does is print them to the server, printing a colon in between and a carriage return and line feed after them.

i kind of understand what your saying so it should start with this?

client.print("Authorization: Digest");

but i still dont know where im suppose to or how to format the credentials and the rest of the header

im looking at thins but its a little advanced for me,

sendHeader(String(FPSTR(WWW_Authenticate)), String(F("Digest realm=\"")) +_srealm + String(F("\", qop=\"auth\", nonce=\"")) + _snonce + String(F("\", opaque=\"")) + _sopaque + String(F("\"")));

 String authorization = "Digest username=\"" + username + "\", realm=\"" + realm + "\", nonce=\"" + nonce +
                         "\", uri=\"" + uri + "\", algorithm=\"MD5\", qop=auth, nc=" + String(nc) + ", cnonce=\"" + cNonce + "\", response=\"" + response + "\"";

after reading this are you sure i can still just insert the header? what am i suppose to do about cnonce and opaque.

referring to this

The client auth header is:
       Authorization: Digest
        algorithm=MD5,
        username="<username>",
        realm="<realm>",
        snonce="<server-nonce>",
        cnonce="<client-nonce>",     -- must be fresh
        uri="<requested-uri>",
        request="<client-digest>",
        message="<message-digest>",
        opaque="<opaque>"            -- required if provided by server
where:
        <client-digest> := H( H(A1) + CN + SN + BI + H(H(A1) + A2) )
        <message-digest> := H( H(A1) + <client-digest> + H(H(A1) + CB) )
and:
        A1 := 'MD5' + U + R + P
        A2 := <Method> + <requested-uri>
        BI := "cbody" | "no cbody"                      -- depending
        OP := "opaque" + <opaque> | "no opaque"         -- depending
        CB := "cbody" + <message-body> | "no cbody"     -- depending
with:
        SN, CN -- server and client nonce values
        U -- username
        R -- realm
        P -- password
        <Method> -- entire request header line 0
        <requested-uri> -- uri sans proxy/routing

"WWW-Authenticate: Digest" is digest authentication. it is more complex then basic authentication

Juraj:
"WWW-Authenticate: Digest" is digest authentication. it is more complex then basic authentication

it looks like the arduino would start the authentication process then the server will reply with some specific numbers that have to be sent back to the server in order to authenticate. not that esphttp library looks good but i dont want to use wifi. my plan is to set this arduino outside in my poe junction box and use the arduino and a photo sensor to control the day/night mode of my cameras. i guess my only option is to use that esp8266http library unless someone want to help me construct a program for the ethernet shield

Juraj:
Digest access authentication - Wikipedia

sure but what am i suppose to do about nonce="", and opaque="" and the md5 hashing is a little out of my league

you could try what I do. the device I request accepts the same values based for nonce forever. so you could try to retrieve the values for response from browsers F12 tool and use them as constant string in the request

Juraj:
you could try what I do. the device I request accepts the same values based for nonce forever. so you could try to retrieve the values for response from browsers F12 tool and use them as constant string in the request

okay this might work but just to clarify do you mean the response to the server from the browser. as in capture the response back to the server after the 401. or do you mean the nonce from the 401

Two items which may or may not be interesting:
This thread has a Digest Authentication routine at the bottom of the page
This Github page has an MD5 library for the Arduino.

darrob:
This thread has a Digest Authentication routine at the bottom of the page

it is the server side