Help with correct syntax for json parser

Dear all :slight_smile:
Problem is solved. Thanks for dropping by my question :smiley:

Dear all.
For some most likely simple reason I keep getting the syntax and/or notation wrong.

The Arduino software is just a small piece. The purpose is to send a json formatted command to a parser running in a Python script on a Windows PC. This Python script converts the json command to a command that a lighting control software understands. This is also running on the same PC.

However I keep getting it wrong.

Code:

/*
NIH FitnessCenter Kontaktbox

Formål med boxen er at modtage knaptryk signal fra knapper på væg, konvertere dem til en netværkskommando og sende dette til LightJockey Server sådan, at lys kan styres fra knapper.
Derudover skal LED ring omkring knapper lyse og anvendes til at indikere tilstande
PIR sensor tilsluttes også disse bokse, og skal anvendes til at sikre, at lys bliver slukket når der ikke er nogen i rummet, samt til at tænde gangareals lys/finde vej lys.



*/

#include <SPI.h>
#include <Ethernet.h>
#include <ArduinoJson.h>
#define MIN_INPUT_PIN 22                              //Laveste input pin nummer
#define MAX_INPUT_PIN 43                              //Højeste input pin nummer
unsigned int cnt=0;                                   //Fri tællevariabel. Bruges til alt muligt
bool printServerData = true;                        // Sættes til sand hvis data fra Server skal printes ud på seriel port. OBS det tager noget CPU power.

StaticJsonDocument<200> command;

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };  // Kontaktbox MAC Adresse. HUSK - skal være unik! Derfor skal der laves ligeså mange .ino filer som der er boxe - en til hver box.
char server[] = "192.168.70.20";                      // IP Adresse for LJ Server 
IPAddress ip(192, 168, 70, 201);                      // IP Adresse for denne kontaktbox.
IPAddress myDns(192, 168, 70, 1);                     // IP Adresse for DNS - bruges egentlig ikke da DNS ikke bruges.

EthernetClient client;                                // Init Ethernet fobindelse som client.

void(* resetFunc) (void) = 0;                         // Reset funktion. Nulstil program pointer til adresse 0


void setup() 
{
  for (cnt=MIN_INPUT_PIN; cnt<=MAX_INPUT_PIN; cnt++)   // Initialize input ports
  {
    pinMode(cnt,INPUT_PULLUP);
  }

                                                       //Initialize output ports for PWM LEDs
  pinMode(44,OUTPUT);
  pinMode(45,OUTPUT);
  pinMode(46,OUTPUT);
  pinMode(2,OUTPUT);
  pinMode(3,OUTPUT);
  // Pin 4 is used for ethernet shield
  pinMode(5,OUTPUT);
  pinMode(6,OUTPUT);
  pinMode(7,OUTPUT);
  pinMode(8,OUTPUT);
  pinMode(9,OUTPUT);
  // Pin 10 is used for Ethernet shield
  pinMode(11,OUTPUT);
  pinMode(12,OUTPUT);
  pinMode(13,OUTPUT);

  Ethernet.begin(mac,ip);
  Ethernet.init(10);                                  // Pin 10 bruges til SS til Ethernet shield
  Serial.begin(9600);                                 // Init seriel port. Bruges til debugging
  while (!Serial);                                    // Vent på Serial port er oppe. Kun nødvendig på Arduino med Native USB port
                                                      // Kommandoliste, bruges til...

                                                      //  command["action"] = "cue";  (Test)
                                                      //  command["id"] = "405";      (Test)
                                                      //  command["pin"] = "31";      (Test)
  if (Ethernet.hardwareStatus() == EthernetNoHardware)
  {
    Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware.");
    while (true)
    {
     for (cnt=0;cnt<5;cnt++)
      {
        digitalWrite(13,HIGH);
        delay(200);
        digitalWrite(13,LOW);
        delay(500);
      }
      resetFunc();
    }
  }
  
  if (Ethernet.linkStatus() == LinkOFF)               // Virker kun med W5200 og W5500
  {
    Serial.println("Ethernet cable is not connected.");
  }
  else
  {
    Ethernet.begin(mac, ip, myDns);
    Serial.print("  Fast IP: ");
    Serial.println(Ethernet.localIP());
    delay(1000);                                       // give the Ethernet shield a second to initialize
    Serial.print("connecting to ");
    Serial.print(server);
    Serial.println("...");
  }
}

uint8_t get_pin_changed()
{
  return 33;
}

void send_command(uint8_t pin)
{
  command["pins"] = 31;
  if (client.connect(server, 5888))
  {
    Serial.print("connected to ");
    Serial.println(client.remoteIP());
    Serial.print("Local port ");
    Serial.println(client.localPort());
    // Make a HTTP request:
    //client.println("POST /action HTTP/1.1");
    client.println("POST /controllers HTTP/1.1");
    client.println("Content-Type: application/json");
    //client.println("Host: 192.168.70.20");
    client.println("Connection: close");
    client.print("Content-Length: ");
    client.println(measureJson(command));
    client.println();
    //serializeJson(command, client);
    Serial.print("JSON:  ");
    serializeJson(command, Serial);
    serializeJson(command, client);
    Serial.println();
  }
  else
  {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
      for (cnt=0;cnt<5;cnt++)
      {
        digitalWrite(12,HIGH);
        delay(200);
        digitalWrite(13,HIGH);
        delay(200);
        digitalWrite(13,LOW);
        digitalWrite(12,LOW);
        delay(200);
      }
  }
}

void loop()
{
  /*uint8_t len = client.available();                       // Hvis der kommer data fra server så print dem ud på seriel porten. Max 80 tegn ad gangen.
  if (len > 0)
  {
    byte buffer[80];
    if (len > 80) len = 80;
    client.read(buffer, len);
    if (printServerData) 
    {
     Serial.write(buffer, len);                           // show in the serial monitor (slows some boards)
    }
  }
  if (!client.connected())                                // if the server's disconnected, stop the client:
  {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    while (true) 
    {
      for (cnt=0;cnt<5;cnt++)
      {
        digitalWrite(13,HIGH);
        delay(500);
        digitalWrite(12,HIGH);
        delay(500);
        digitalWrite(13,LOW);
        digitalWrite(12,LOW);
        delay(500);
      }
      resetFunc();
    }
  }
*/
    //if knap er trykket på send kommando
    //switch (fdsfgdsg)
    //case (gdsagads)
  send_command(get_pin_changed());
  delay(1000);
  digitalWrite(13,HIGH);
  while(digitalRead(31));
  digitalWrite(13,LOW);
}

Important note: As you can see I am on purpose returning the number 33 - this is for test purposes. When the command parser gets the number 33 it shall turn off the light in the "Teknik rum"

The config file on the PC looks like this:

{
    "commands": {
        "cardio_on": {
            "action": "cue",
            "id": "1",
            "parameter": "restart"
        },
        "teknik_on": {
            "action": "cue",
            "id": "405"
        },
        "teknik_off": {
            "action": "cue",
            "id": "406"
        }
    },
    "controllers": {
        ".115": {
            "name": "Teknik",
            "pins": [
                {"id": "31", "cmd": "teknik_on"},
                {"id": "33", "cmd": "teknik_off"}
            ]
        }
    }
}

And the Python command parser on the PC is this:

import json


class Parser:
    def __init__(self, filename: str):
        with open(filename) as fh:
            self._config = json.load(fh)

    def find_ip(self, ip: str):
        for e in self._config['controllers']:
            if ip.endswith(e):
                return self._config['controllers'][e]

    def get_command(self, ip: str, pin: str):
        o = self.find_ip(ip)
        for p in o['pins']:
            if str(p['id']) == str(pin):
                return self.get_named_command(p['cmd'])
        return None

    def get_named_command(self, command: str) -> json:
        # for c in 
        print("Search for: ", command)
        for c in self._config['commands']:
            print(c)
            if c == command:
                return self._config['commands'][c]


# p = Parser('config.json')
# o = p.find_ip('.50.209')

# print(o['name'])

Finally the Python script for converting the json command to a message the lighting software understands is this:

from flask import Flask, request
import json

from ljcommand import LJCommand
from command_parser import Parser


app = Flask(__name__)


@app.before_first_request
def init():
    global parser, lj
    parser = Parser('config.json')
    lj = LJCommand()


def handle_action(data: json):
    action = str(data['action']).lower()
    aid = int(data['id'])
    parameter = str(data['parameter']) if 'parameter' in data else None

    if action == 'cue':
        return lj.cue(aid, parameter)


@app.route('/controller', methods=['POST'])
def controller():
    if request.method == 'POST':
        data = request.json
        pin = int(data['pin'])

        cmd = parser.get_command(request.remote_addr, pin)
        if handle_action(cmd):
            return 'success', 200
        else:
            return 'failure', 501


@app.route('/action', methods=['POST'])
def receive_action():
    if request.method == 'POST':
        if handle_action(request.json):
            return 'success', 200
        else:
            return 'failure', 501


@app.route('/command', methods=['POST'])
def receive_command():
    if request.method == 'POST':
        data = request.json
        command = parser.get_named_command(data.command)

        if handle_action(command):
            return 'success', 200
        else:
            return 'failure', 501



@app.route('/')
def home():
    # ljcommand.send(1002, -1, 1)
    # print(json.dumps(parser._config['.50.101']))


I know for sure that the Python parts works as planned because I can send commands using Bitfocus Companion to control the lighting software. This works flawlessly.

So I "just" getting some syntax wrong or I am sending the wrong command.

The error looks like this:


192.168.70.30 - - [11/Dec/2022 18:08:53] "POST /action HTTP/1.1" 200 -
192.168.70.30 - - [11/Dec/2022 18:08:53] "POST /action HTTP/1.1" 200 -
192.168.70.201 - - [11/Dec/2022 18:08:55] "POST /controller HTTP/1.1" 500 -
Traceback (most recent call last):
  File "C:\Users\NIH Fitness Lys PC\AppData\Local\Programs\Python\Python38-32\Li
b\site-packages\flask\app.py", line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File "C:\Users\NIH Fitness Lys PC\AppData\Local\Programs\Python\Python38-32\Li
b\site-packages\flask\app.py", line 2076, in wsgi_app
    response = self.handle_exception(e)
  File "C:\Users\NIH Fitness Lys PC\AppData\Local\Programs\Python\Python38-32\Li
b\site-packages\flask\app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Users\NIH Fitness Lys PC\AppData\Local\Programs\Python\Python38-32\Li
b\site-packages\flask\app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Users\NIH Fitness Lys PC\AppData\Local\Programs\Python\Python38-32\Li
b\site-packages\flask\app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Users\NIH Fitness Lys PC\AppData\Local\Programs\Python\Python38-32\Li
b\site-packages\flask\app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)

  File "C:\Temp\LJ Webserver\LJ Webserver\app.py", line 33, in controller
    cmd = parser.get_command(request.remote_addr, pin)
  File "C:\Temp\LJ Webserver\LJ Webserver\command_parser.py", line 16, in get_co
mmand
    for p in o['pins']:
TypeError: 'NoneType' object is not subscriptable

As you can see, commands from 192.168.70.30 works just fine - this is the companion software.

The Arduino has the 192.168.70.201 IP address - and commands from this one is not interpreted correctly. I am very new to json - so if any one could help me out that would really be awesome :smiley:
Thanks in advance.

PS: I know that this question is somewhat outside the normal Arduino scope - but I am hoping that someone here can look through it as say ahhhh! You just need to add "" or "pin" instead of "pins" or something simple similar to that. I think it is this kind of problem I have - but I am unsure.

What do you see in the Arduino Serial monitor?

The post requests from 192.168.70.30 are going to a different URL (/action Vs /controller) check what happens when they both post to the same place.

Usually JSON data is sent as the body of the post request so I'm not sure if this line is correct:
Serial.print("JSON: ");
I don't know if that will be interpreted as another header and create a header named "JSON" in the request or just add the text "JSON:" to the beginning of the request body - either way I don't think it's correct.

It’s sent to Serial, not to the client so not an issue

I would test with a canned json print as text and not use json at all just to qualify you got everything right - ie duplicate a request you know works and send it from arduino manually

1 Like

This appears to be the error that is causing the crash. Looks like self.find_ip(ip) is not returning data that can be indexed.

1 Like

Hi all

Thanks a lot for your suggestions and help and time.

It turned out it was a stupid detail I forgot!

In the config.json file the code is as follows:

{
    "commands": {
        "cardio_on": {
            "action": "cue",
            "id": "1",
            "parameter": "restart"
        },
        "teknik_on": {
            "action": "cue",
            "id": "405"
        },
        "teknik_off": {
            "action": "cue",
            "id": "406"
        }
    },
    "controllers": {
        ".115": {
            "name": "Teknik",
            "pins": [
                {"id": "31", "cmd": "teknik_on"},
                {"id": "33", "cmd": "teknik_off"}
            ]
        }

Note the .115 - this refers to the last segment of the IP address of the client ie. the arduino device. And my arduino device is programmed with .201 as the last segment of the IP address (192.168.70.201) And since there was no error handeling in the python script it became quite hard to figure out that this was the root cause!

So it works now! Thanks a lot to all of you for spending time and helping out :slight_smile:

Hi John - This was actually very close to the root cause! Points for your help :smiley:

Hi Touch1337 - this line was only for debugging purposes. I had to jump in my car to go to dinner with family so I did not do code-clean-up before posting here. thanks a lot for helping out :slight_smile:

Hi Jackson - I forgot to post this - but I found the problem - I was sending from wrong IP. The config.json files specifies that IP address .116 must be used, but I used .201! This was the rootcause of the problem. Add to this that I did not have any error handeling in the python code so it became very hard to figure out what caused the problem. It works now - and thanks for spending time on this problem :slight_smile:

great, happy for you.

thanks for coming back to us

1 Like

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.