Help to optimise for faster processing

Let me preface with, I’m definitely no programmer and I’m learning as I go. This is also my first arduino. So thank you in advance.

Currently working on a project for a dashboard that mirrors the state of a game. The game states are written to a json file as they change. I’m using python on the PC side to monitor if the file gets updated. When it does the python program opens the file and does a json dump. It then verifies the contents have changed from what was previously sent. I did this check to reduce serial sends in case a toggle was quickly pressed on and off or similar. At that point it is sent to the Arduino.

Once the Arduino takes over it parses the json data out, assigns variable and rolls through if statements to turn leds on or off.

For this initial POC I’m only looking at a single file and event type. I have 5 leds hooked up. End sate will be numerous addressable rgb leds, led level indicators, and LCD module or 7 segment etc.

The game writes to around 5 files as it is running. I’m filtering on the “Event” key in the json as that will tell me what data is coming… ie event":“Cargo” / “event”:“Status” / event":“Location” and so on. If I can get this optimized enough and not overload the MEGA I’ll be sending data from all files and most event types at some point.

See this link if curious Player Journel Guide

As of right now I’ve got the led activating within roughly 1.5 to 2 seconds of the game state changing and since I’m learning all this as I go I was hoping for some help in optimizing.

I wasn’t sure if i should break the data down more on the PC side and sending smaller amounts but more often would help?

Arduino declarations for hex keys etc.

#include <ArduinoJson.h>
bool LED;
const int Docked_Pin = 6, GearDown_Pin = 5, Shields_Pin = 4, HardPoints_Pin = 3, Lights_Pin = 2;
int ledArray[] = { Docked_Pin, GearDown_Pin, Shields_Pin, HardPoints_Pin, Lights_Pin };
const int Board_Pin = 13;

//Status File Variables
long previousFlags;
long flags;
int sysPip;
int enginePip;
int wepPip;
int fireGroup;
int guiFocus;
float fuelMain;
float fuelReservoir;
int cargo;
char* legalStatus;

//Status File Flag Keys
#define flagDocked  0x00000001
#define flagLanded  0x00000002
#define flagGearDown 0x00000004
#define flagShieldsUp  0x00000008
#define flagSupercruise  0x00000010
#define flagFlightAssistOff  0x00000020
#define flagHardpointsDeployed  0x00000040
#define flagInWing  0x00000080

#define flagLightsOn  0x00000100
#define flagCargoScoopDeployed  0x00000200
#define flagSilentRunning  0x00000400
#define flagScoopingFuel  0x00000800
#define flagSRVHandbrake  0x00001000
#define flagSRVTurret  0x00002000
#define flagSRVTurretRetracted  0x00004000
#define flagSRVDriveAssist  0x00008000

#define flagFSDMassLocked  0x00010000
#define flagFSDCharging  0x00020000
#define flagFSDCooldown  0x00040000
#define flagLowFuel  0x00080000
#define flagOverHeating  0x00100000
#define flagHasLatLong  0x00200000
#define flagIsInDanger  0x00400000
#define flagBeingInterdicted  0x00800000

#define flagInMainShip  0x01000000
#define flagInFighter  0x02000000
#define flagInSRV  0x04000000
#define flagHudInAnalysisMode  0x08000000
#define flagNightVision  0x10000000

//Status States
bool Docked;
bool Landed;
bool GearDown;
bool ShieldsUp;
bool Supercruise;
bool FlightAssistOff;
bool HardpointsDeployed;
bool InWing;

bool LightsOn;
bool CargoScoopDeployed;
bool SilentRunning;
bool ScoopingFuel;
bool SRVHandbrake;
bool SRVTurret;
bool SRVTurretRetracted;
bool SRVDriveAssist;

bool FSDMassLocked;
bool FSDCharging;
bool FSDCooldown;
bool LowFuel;
bool OverHeating;
bool HasLatLong;
bool IsInDanger;
bool BeingInterdicted;

bool InMainShip;
bool InFighter;
bool InSRV;
bool HudInAnalysisMode;
bool NightVision;


void setup() {
  // put your setup code here, to run once:
//Set pinModes

for (int i = 0; i <= 4; i++){
    pinMode(ledArray[i], OUTPUT);
    }
pinMode(Board_Pin, OUTPUT);

//Open serial port
Serial.begin(9600);



// Wait untilSerial is ready
while (! Serial); 


}

Arduino Code from receiving json on. I feel like the long list of if/else is where I need to improve most.

void loop() {
  
//Recieve and store JSON data from PC
  int     size_ = 0;
  String  payload;
  while ( !Serial.available()  ){}
  if ( Serial.available() )
    payload = Serial.readStringUntil( '\n' );
  StaticJsonDocument<512> doc;

// Deserialize the JSON document
DeserializationError   error = deserializeJson(doc, payload);
  if (error) {
    Serial.println(error.c_str()); 
    return;
  }  

//Set Status Event Values
if (doc["event"] == "Status")
{
  
  flags = doc["Flags"];
  sysPip = (doc["Pips"][0]);
  enginePip = (doc["Pips"][1]);
  wepPip = (doc["Pips"][2]);
  fireGroup = doc["FireGroup"];
  guiFocus = doc["GuiFocus"];
  fuelMain = doc["Fuel"]["FuelMain"];
  fuelReservoir = doc["Fuel"]["FuelReservoir"];
  cargo = doc["Cargo"];
  legalStatus = doc["LegalState"];

//Set Flags and LEDS if Flags Changed
  if (flags != previousFlags)
  {
    previousFlags = flags;
    Docked = flags & flagDocked;
    Landed = flags & flagLanded ;
    GearDown = flags & flagGearDown;
    ShieldsUp = flags & flagShieldsUp;
    Supercruise = flags & flagSupercruise;
    FlightAssistOff = flags & flagFlightAssistOff;
    HardpointsDeployed = flags & flagHardpointsDeployed;
    InWing = flags & flagInWing;

    LightsOn = flags & flagLightsOn;
    CargoScoopDeployed = flags & flagCargoScoopDeployed;
    SilentRunning = flags & flagSilentRunning;
    ScoopingFuel = flags & flagScoopingFuel;
    SRVHandbrake = flags & flagSRVHandbrake;
    SRVTurret = flags & flagSRVTurret;
    SRVTurretRetracted = flags & flagSRVTurretRetracted;
    SRVDriveAssist = flags & flagSRVDriveAssist;

    FSDMassLocked = flags & flagFSDMassLocked;
    FSDCharging = flags & flagFSDCharging;
    FSDCooldown = flags & flagFSDCooldown;
    LowFuel = flags & flagLowFuel;
    OverHeating = flags & flagOverHeating;
    HasLatLong = flags & flagHasLatLong;
    IsInDanger = flags & flagIsInDanger;
    BeingInterdicted = flags & flagBeingInterdicted;

    InMainShip = flags & flagInMainShip;
    InFighter = flags & flagInFighter;
    InSRV = flags & flagInSRV;
    HudInAnalysisMode = flags & flagHudInAnalysisMode;
    NightVision = flags & flagNightVision;


  
//Set LEDS Based on Status
  if(Docked)
  {
    digitalWrite (Docked_Pin, HIGH);
  }
  
  else 
  {
    digitalWrite (Docked_Pin, LOW);
  }

    if(GearDown)
  {
    digitalWrite (GearDown_Pin, HIGH);
    digitalWrite (Board_Pin, HIGH);
  }
  
  else 
  {
    digitalWrite (GearDown_Pin, LOW);
    digitalWrite (Board_Pin, LOW);
  }
  

    if(ShieldsUp)
  {
    digitalWrite (Shields_Pin, HIGH);
  }
  
  else 
  {
    digitalWrite (Shields_Pin, LOW);
  }

    if(HardpointsDeployed)
  {
    digitalWrite (HardPoints_Pin, HIGH);
  }
  
  else 
  {
    digitalWrite (HardPoints_Pin, LOW);
  }

    if(LightsOn)
  {
    digitalWrite (Lights_Pin, HIGH);
  }
  
  else 
  {
    digitalWrite (Lights_Pin, LOW);
  }
  

}
}
}

Here is the final part of my watch and send in Python the json.load is for use elsewhere on the PC side.

    for action, file in results:
        full_filename = os.path.join(path_to_watch, file)
        print(full_filename, ACTIONS.get(action, "Unknown"))

        if "Status" in full_filename:
            print("Status File Updated")

            with open(StatusPath, "r") as StatusFile:
                time.sleep(.1)

                Status = json.load(StatusFile)
                Send = json.dumps(Status)
                TimeStamp = (Status['timestamp'])

                if Send != PreviousStatus:
                    PreviousTimestamp = TimeStamp
                    PreviousStatus = Send
                    print('Sent:', Send)

                    if ser.isOpen():
                        ser.write(Send.encode('ascii'))
                        ser.flush()
                    time.sleep(.015)

I have not studied your code, however if this was my project I would use Python to analyse the data and send a short message to the Arduino perhaps like <0110001> which means turn LEDs 1,2 and 6 on and turn LEDs 0, 3 4 and 5 off.

…R
Serial Input Basics - simple reliable ways to receive data.
Simple Python - Arduino demo

Thanks Robin. I’ll read through those links today. I like your example for doing the on/off though I’d probably need to have a variable indicator in the first position to filter off of.

Something like for status lights

So if dataReceived[0] == “S” would let me send it to the if list for the status leds

Then for example for power distribution. Keeping the data length consistent for the serial read to be static even though in this case I only need 3.

int power = (SysPower, EngPower, WepPower)

if dataReceived[0] == “P”

for (int i = 1; i <= 3; i++){
dataReceived = power[i-1];

  • }*
    The hit the ifs for the power level leds.
    Hopefully I’m on the right track… thanks again for the info.

Another option is to send everything in one message like and then you probably would not need the S or the P

If the Arduino has an array with all the I/O pin numbers - for example

byte pinNumbers[] = {7,9,11,5,4,3};

Then if the message “011000” is in the char array recvdChars you can update everything with a few lines like this

for (byte n = 0; n < 6; n++) {
   byte ledStatus = recvdChars[n] - 48;
   digitalWrite(pinNumbers[n], ledStatus);
}

…R

Thanks again for the feedback Robin. Forgive me if this goes off the rails a bit…

I think I need to take a step back and break down every function I want the arduino to perform and see how simple I can make it.

For the LED control I’m fairly certain I can get that encapsulated into one message. I’ll be doing more than on off for some and planned on using addressable RGB.

The LED load could be something like

If I stick to a limited set of ledStatus (0-9) this should work. Mostly because I’m not sure how to do classes/structs very well yet… assuming that’s what I would need. Or ledStatus wouldn’t be an array and just assigned to the pass char with an if ledStatus = perhaps?

Also is it even possible to assign a function to ledStatus like this? Where say led_BlinkRed would = a loop function?

Also what is the “- 48” for in the below loop?

led_off [] = {0, 0, 0}

ledAddress[] = { Docked, GearDown, Shields, HardPoints, Lights, SysPower1, SysPower2, SysPower3, SysPower4, SysPower5} 
 ledStatus[] = {led_off, led_Green, led_Red, led_Yellow, led_BlnkRed, led_BlnkYellow}


for (byte n = 0; n < 6; n++) {
   byte ledStatus = recvdChars[n] - 48;
   digitalWrite(ledAddress[n], ledStatus);
}

I still need to learn how to format for the addressable RBGs once I get them. Was going to use the FastLed library but think I have the general idea now.

What I need to account for next is passing over text for the screen.

If anything this is really making me think deeper about how to process things efficiently. Thank you

Zarklin:
If I stick to a limited set of ledStatus (0-9) this should work. Mostly because I’m not sure how to do classes/structs very well yet… assuming that’s what I would need. Or ledStatus wouldn’t be an array and just assigned to the pass char with an if ledStatus = perhaps?

You will need to provide more information about what you have in mind.

Also what is the “- 48” for in the below loop?

It converts the ascii value of the character to a number. For example the ascii value for ‘1’ is 49 so if you subtract 48 you get the number 1

…R

Robin2:
You will need to provide more information about what you have in mind.

It was more a thought of if I were to define more than 10 Led states. Say off, white, red, green, blue, yellow... blink for each etc.

Robin2:
It converts the ascii value of the character to a number. For example the ascii value for '1' is 49 so if you subtract 48 you get the number 1

Ah OK, I wasn't thinking about the 1 being sent as a char not an int. My brain hurt when I had to do % for conversion TO ASCII on an exercise. Accounting for upper lower while excluding symbols... I really need to start that class over... and actually finish it...

if (isupper(plaintext[i]))
            {    
                ciphertext[i] = (((((plaintext[i] + key) - 65)) % 26)) + 65;
            }
            else if (islower(plaintext[i]))
            {
                ciphertext[i] = (((((plaintext[i] + key) - 97)) % 26)) + 97;
            }

Zarklin:
It was more a thought of if I were to define more than 10 Led states. Say off, white, red, green, blue, yellow... blink for each etc.

You could use upper-case letters of the alphabet to represent 26 variants. 'A' is ascii 65 so subtract 65 from it and treat it as 0.

Ah OK, I wasn't thinking about the 1 being sent as a char not an int.

I am also assuming it is sent as a char - that's why the 48 is being subtracted. (I think I missed your double negative, but I will leave my comment)

...R

Hooray for functions and pointers to pull this off. Reading and re-reading how they work made my head hurt.

Making some good progress I think.

I can only have one light blinking at a time for now. I’m sure it’s in part from the blink timer being global variables. I tried moving them inside the function but I would get one blink and stop.

So that will be tomorrows project.

Thank you again for all the guidance thus far Robin. It is truly appreciated. Also I just re-read a couple of my posts and I apologize for the poor structure. They were typed in haste between meetings :slight_smile:

// will store last time LED was updated
unsigned long previousMillis = 0;        

// interval at which to blink (milliseconds)
const long interval = 1000;           

// ledState used to set the LED
int ledState = LOW;             

int ledPin[] = {2,3,4,5,6,13};  // the number of the LED pins

char* recvdChars = "121111";

//LED Status Functions
void LED_Off (int);
void LED_On (int);
void LED_Blink(int);

//LED Status Function Pointer
void (*ledStatus[])(int) = {LED_Off, LED_On, LED_Blink};


void setup() {

// set the digital pins as output:
 for (byte i = 0; i < 5; i++){
    pinMode(ledPin[i], OUTPUT);
  }


}


void loop() {

//Set recvdChars to ledStatus and n to pin number
for (int n = 0; n < 5; n++) {
  int s = recvdChars[n] - 48;
  ledStatus[s](n);

}
 
}

//Functions defined

void LED_Off (int p){
  digitalWrite(ledPin[p], LOW);
}

void LED_On (int p){
  digitalWrite(ledPin[p], HIGH);
}

void LED_Blink (int p){

unsigned long currentMillis = millis();

  if (currentMillis - previousMillis >= interval) {
    // save the last time you blinked the LED
    previousMillis = currentMillis;

    // if the LED is off turn it on and vice-versa:
    if (ledState == LOW) {
      ledState = HIGH;
    } else {
      ledState = LOW;
    }

    // set the LED with the ledState of the variable:
    digitalWrite(ledPin[p], ledState);
  }
  }

your pc is much more powerfull than the arduino if you have a pc in your project, then the pc should do all the work!!! you should be only sending the minim to your arduino. which would be what leds are to be lit.