Is it possible to pull a compiled binary from GitHub via HTTPUpdate?

I’m trying to setup a project which I can update OTA by automatically downloading a new version of the firmware that I will compile and upload to the project repo on my GitHub, is this possible with HTTPUpdate? I keep running into errors saying HTTP connection refused when I try to point the ESP to either raw format of the bin file on Github or the same on SourceForge and I’m not sure if it’s because they might be using redirects or if there’s something I’m doing wrong.

FYI, I changed the update URL to generic “username” and “projectrepo” because my repo is private, but in the code I’ve uploaded to my board, those point to the proper repo.

I’m using an ESP8266 as my hardware and here’s the relevant code:

#include "globals.h"

void setup() {
  Serial.begin(115200);
  FastLED.addLeds<CHIPSET, DATA_PIN, COLOR_ORDER>(leds, NUM_LEDS);

  setupStripedPalette( CRGB::Red, CRGB::Red, CRGB::White, CRGB::White); //for CANDY CANE
  gPal = HeatColors_p; //for FIRE

  setup_wifi();

  Serial.println("Ready");
  Serial.print("IP Address: ");
  Serial.println(WiFi.localIP());

  client.setServer(mqtt_server, mqtt_port);
  client.setCallback(callback);
}

/********************************** START MAIN LOOP****************************************/
void loop() {
  if (!client.connected()) {
    //reconnect();
  }

  if (WiFi.status() != WL_CONNECTED) {
    delay(1);
    Serial.print("WIFI Disconnected. Attempting reconnection.");
    setup_wifi();
    return;
  }

  if((WiFiMulti.run() == WL_CONNECTED)) {
    t_httpUpdate_return ret = ESPhttpUpdate.update("http://github.com/username/projectrepo/blob/master/firmware.bin?raw=true");
    //t_httpUpdate_return  ret = ESPhttpUpdate.update("https://server/file.bin");
    switch(ret) {
      case HTTP_UPDATE_FAILED:
        USE_SERIAL.printf("HTTP_UPDATE_FAILD Error (%d): %s", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        break;
      case HTTP_UPDATE_NO_UPDATES:
        USE_SERIAL.println("HTTP_UPDATE_NO_UPDATES");
        break;
      case HTTP_UPDATE_OK:
        USE_SERIAL.println("HTTP_UPDATE_OK");
      break;
    }
  }

  client.loop();

  if (effectString == "bpm") {bpmFunc();}

  if (effectString == "candy cane") {candyCane();}

  if (effectString == "confetti" ) {confetti();}

  if (effectString == "cyclone rainbow") {cycloneRainbow();}

  if (effectString == "dots") {dots();}

  if (effectString == "fire") {fire();}

  if (effectString == "glitter") {glitter();}

  if (effectString == "juggle" ) {juggle();}

  if (effectString == "lightning") {lightning();}

  if (effectString == "police all") {policeAll();}

  if (effectString == "police one") {policeOne();}

  if (effectString == "rainbow") {rainbow();}

  if (effectString == "rainbow with glitter") {rainbowGlitter();}

  if (effectString == "sinelon") {sinelon();}

  if (effectString == "twinkle") {twinkle();}

  if (effectString == "christmas alternate") {xmasAlternate();}

  if (effectString == "random stars") {randomStars();}

  if (effectString == "sine hue") {sineHue();}

  EVERY_N_MILLISECONDS(10){
    nblendPaletteTowardPalette(currentPalette, targetPalette, maxChanges);
    {gHue++;}
    if (effectString == "noise") {noise();}
    if (effectString == "ripple") {ripple();}
  }

  EVERY_N_SECONDS(5) {targetPalette = CRGBPalette16(CHSV(random8(), 255, random8(128, 255)), CHSV(random8(), 255, random8(128, 255)), CHSV(random8(), 192, random8(128, 255)), CHSV(random8(), 255, random8(128, 255)));}

  if (flash) {flashFunc();}

  if (startFade && effectString == "solid") {noFade();}

  if (inFade) {fade();}
}

Edit:

Also, here’s the relevant stuff from globals.h (didn’t include the whole project because it’s long and most of it is tested and functional, just this feature has issues:

  #include <ESP8266WiFiMulti.h>
  #include <ESP8266HTTPClient.h>
  #include <ESP8266httpUpdate.h>
  #define USE_SERIAL Serial

  ESP8266WiFiMulti WiFiMulti;

https?

Sorry I'm still a beginner. Are you saying to use HTTPS? If so, I tried that and got the same errors, but I figured HTTPS would require something else and just wanted to get HTTP working as a proof of concept first.

vashp2029:
Sorry I’m still a beginner. Are you saying to use HTTPS? If so, I tried that and got the same errors, but I figured HTTPS would require something else and just wanted to get HTTP working as a proof of concept first.

github redirects http to https.

I tried:

    t_httpUpdate_return ret = ESPhttpUpdate.update("https://rawgit.com/user/repo/master/firmware.bin", "", "D4 8A BC 3F 08 31 5C E5 A4 F0 D0 99 77 A8 85 7B 31 4A 1F 2A");

Now getting a connection refused error...

  1. As Juraj said, if GitHub is configured correctly, it’ll redirect all HTTP requests to use TLS (HTTPS). So your only chance is to use HTTPS.
  2. If the repository is private, you can’t access its files without logging in.
  3. Are you sure that the URL is correct? I’ve never heard of rawgit.com. Maybe you mean raw.githubusercontent.com.
  4. Are you sure that the fingerprint is correct? Try running the following in a terminal (Linux or Mac)
openssl s_client -connect raw.githubusercontent.com:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin | sed 's/:/ /g'

Pieter

PieterP:

  1. As Juraj said, if GitHub is configured correctly, it’ll redirect all HTTP requests to use TLS (HTTPS). So your only chance is to use HTTPS.
  2. If the repository is private, you can’t access its files without logging in.
  3. Are you sure that the URL is correct? I’ve never heard of rawgit.com. Maybe you mean raw.githubusercontent.com.
  4. Are you sure that the fingerprint is correct? Try running the following in a terminal (Linux or Mac)
openssl s_client -connect raw.githubusercontent.com:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin | sed 's/:/ /g'

Pieter

I’m using one of my public repos as the host for now just to test it out and you were correct, rawgit was a third party redirect tool. I used raw.githubusercontent instead and it looks like it’s atleast talking to server now. However, now I get a timeout error:

HTTP_UPDATE_FAILD Error (6): Update error: ERROR[6]: Stream Read Timeout

Any ideas on the next step? This is the relevant line now:

    t_httpUpdate_return ret = ESPhttpUpdate.update("https://raw.githubusercontent.com/myusername/mypublicrepo/master/firmware.bin", "", "CC AA 48 48 66 46 0E 91 53 2C 9C 7C 23 2A B1 74 4D 29 9D 33");

Also, since I don’t have a mac/linux and I’m not sure how to go about getting it in Windows, I obtained the fingerprint for domain from here: GRC | SSL TLS HTTPS Web Server Certificate Fingerprints