Client.read to String, data not being received correctly.

Using Arduino Programming Questions
#1

Hi guys, complete beginner here. I am working on a project that it turns out is a bit too much for my very little programming knowledge. Please be gentle.
The total project will be a device to control my digital audio mixer over it's network. Yes, I am a musician, not a programmer :slight_smile:
This device connects to the same router where the mixer is also connected. I normally control it from a browser on my Android tablet or phone. But, I decided i wanted to control some of its functions from my guitar effects Pedalboard, that can output MIDI.
Also, this device will hopefully have some rotary encoders and buttons that I could assign to some of the mixer's functions like to control the music player for example . The mixer receives and sends "Text" controls over the network. I need to pick up this information, pass it to variables, to keep the device in Sync with the mixer.
Anyway, now that you know what I want to do, let's try to explain the problem. the best I can.
The code below is some testing I am doing to read the network, compare the lines of text being CONTINUOUSLY received (client.read), compare the text to check which parameter was received, and then extract the value. here is a little dump of the controls the mixer Accepts/sends.

SETS^a.3.afs.eq.1^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.3.afs.eq.2^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.3.eq.prname^
SETD^i.2.aux.1.post^0
SETD^i.2.aux.2.pan^0.5
SETD^i.2.aux.2.value^0
SETD^i.2.aux.2.mute^0
SETD^i.2.aux.0.post^0
SETD^i.2.aux.1.pan^0.5
SETD^i.2.aux.1.value^0
SETD^i.2.aux.1.mute^0
SETD^i.2.gate.depth^0
SETD^i.2.aux.0.pan^0.5
SETD^i.2.aux.0.value^0
SETD^i.2.aux.0.mute^0
SETD^i.2.gate.thresh^0
SETD^i.2.gate.attack^0
SETD^i.2.gate.release^0.3150737134
SETD^i.2.gate.hold^0.3042441765
#2

as you can see for example on this example (SETD^i.2.gate.hold^0.3042441765) the first part of the text dictates the parameter being controlled, and then the numeric value, is the position of a potentiometer or fader. It ranges from 0 for 0% to 1 for 100% control. Now here is the code related to this problem. I still don’t have the complete code together. I am still testing all different bist individually.

/*
  Web client



  =========================================================


  <---- Pinout ---->
  W5x00 <--> STM32F401CC or  STM32F103CB
  SS    <-->  PA4 <-->  BOARD_SPI1_NSS_PIN
  SCK   <-->  PA5 <-->  BOARD_SPI1_SCK_PIN
  MISO  <-->  PA6 <-->  BOARD_SPI1_MISO_PIN
  MOSI  <-->  PA7 <-->  BOARD_SPI1_MOSI_PIN
  =========================================================

*/

#include <SPI.h>
#include <Ethernet.h>



#include "HystFilter.h"
HystFilter potA( 4096, 128, 14 ) ;  // Input 12 bit ADC = 4096, 128 discrete output values required, 
// margin = 8 units (of 4096)




// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
IPAddress server(10, 10, 2, 1);  // numeric IP for  Ui24  (no DNS)
//char server[] = "Ui.IO";    // name address for Google (using DNS)

// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(10, 10, 2, 7);

// Initialize the Ethernet client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

//long hits; // the number of google hits

//unsigned long beginMicros, endMicros;
unsigned long byteCount = 0;
bool printWebData = true;  // set to false for better speed measurement



//midi in variables
byte state = 0;
byte statusByte;
byte dataByte1;
byte dataByte2;
byte channel;
byte incomingByte;
byte MidiChannel = 15;

bool Play = 0;

int potPin = PA0;    // select the input pin for the potentiometer
int potval = 0; //initial potvalue
int oldpotval = 0; // old pot value
float potsendval = 0; // we need a variable to convert a value for the Ui fader
float PotValue = 0;
byte number;
byte value;
byte note;
byte velocity;
int ledPin = PC13;

void setup() {

  analogReadResolution(12);

  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);



  // Open serial communications and wait for port to open:

  Serial1.begin(115200);



  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial1.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial1.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial1.println("connected");
    // Make a HTTP request:
    client.println("GET /raw HTTP1.1\n\n");
    //client.println("Host: www.google.com");
    client.println("Connection: close");
    client.println();
  }
  else {
    // kf you didn't get a connection to the server:
    Serial1.println("connection failed");
  }

  //Serial2.println("Serial2 Alive");

}
float masterVol;
float playerVol;
String numData;

String data;
int Detect = 0;
int Detect1 = 0;
int Detect2 = 0;

void loop()
{
  // if there are incoming bytes available
  // from the server, read them and print them:

  int len = client.available();
  client.println("ALIVE\n");
  if (len > 0) {
    byte buffer[512];
    if (len > 512) len = 512;
    client.read(buffer, len); // Read Data sent by the audio mixer
    if (buffer > 0) {
      data += (char*)buffer; // Convert the data to String


      // Master Volume

      if (data.length() >= 0) {  // If the String has data
        Detect = data.indexOf("SETD^m.mix^");
        if (Detect >= 0) // First character's Index is 169. Normally should be 0
          // and I don't understand why it is 169.
        {

          numData = data.substring(180, 186); // This takes the numeric value
          // from "SETD^m.mix^0.868486" and places it on another string.

          masterVol = numData.toFloat(); // Puts the value extracted in a "float" type variable
          Serial.print("Master Volume = ");
          Serial.println(masterVol * 100); // Converts to Percentage and makes
          // it easier to use
        }

        numData = ""; //resets this variable

        // Line In Mix Fader

        Detect1 = data.indexOf("SETD^l.0.mix^");
        if (Detect1 >= 0) // First character Index is 169
        {

          numData = data.substring(182, 188);

          playerVol = numData.toFloat();
          Serial.print("Line In Volume = ");
          Serial.println(playerVol * 100);
        }

        numData = "";

        // Channel 8 Mix Fader

        Detect2 = data.indexOf("SETD^i.7.mix^");
        if (Detect2 >= 0) // First character Index is 169
        {
          numData = data.substring(182, 188);

          Serial1.println(numData);
          playerVol = numData.toFloat();
          Serial1.print("Main Mix Ch8 = ");
          Serial.println(playerVol * 100);
        }

        numData = "";

      }
      data = ""; // Reset the main String

    }
    byteCount = byteCount + len;
  }


  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial1.println();
    Serial1.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while (true);
  }
// A test Potentiometer controlling the Mixer's Music Player volume via the network.

  potval = potA.getOutputLevel( analogRead(potPin));    // read the value from the pot
  if (oldpotval != potval) {          //check if the value is the same, we don't want multiple sends
    // Serial.println(potval);
    oldpotval = potval;                 // Make the old value same as the new value
    potsendval = potval; //    Convert to Float


    //  client.print("SETD^i.0.mix^");  // Send SETD value with No NEW LINE
    //  client.println((potsendval / 255), 4); // Send the value with NEW LINE

    SETD();
    client.print("p.0");
    Print_mix();
    client.println((potsendval / 127), 4);

    SETD();
    client.print("p.1");
    Print_mix();
    client.println((potsendval / 127), 4);

    oldpotval = potval;

  }

}
// Repetitive text is placed in separate functions to same RAM.

void SETD()
{
  client.print("SETD^");
}
void Print_mute()
{
  client.print(".mute^");
}
void Print_solo()
{
  client.print(".solo^");
}
void Print_post()
{
  client.print(".post^");
}
void Print_fx()
{
  client.print(".fx.");
}
void Print_Mute_Gr()
{
  client.print("SETD^mgmask^");
}
void Print_MEDIA()
{
  client.print("MEDIA_");
}
void digitech()
{
  client.print(".digitech.");
}
void enabled()
{
  client.print("enabled^");
}
void Print_gain()
{
  client.print(".gain^");
}
void Print_pan()
{
  client.print(".pan^");
}
void Print_mix()
{
  client.print(".mix^");
}
void Print_value()
{
  client.print(".value^");
}
void Print_aux()
{
  client.print(".aux.");
}

So, what this code does, is I move the mixer faders on the browser interface, and I extract the values from the commands and assign them to variables. Now the problems.

  1. When a new device connects to the mixer, it sends all settings at one, so all devices can sync with it.
    My code is not reading this correctly at startup as you can see. on this serial output. I am receiving 2 parameters only here.
connecting...
connected
Main Mix Ch8 = TD^i.1
Player Volume = .35049
  1. This code
data.indexOf("SETD^i.7.mix^");

returns the indexOf the text being compared with the data placed on the data String. The index is the position of the first character, in this case “S”. It should return 0, but it returns 169. Related with problem 1 above?
SO, to extract the values I need to add the number of the charaters to the number 169. Remember this should in my opinion report 0. So, the values in this case are placed between position 182 and 188.

numData = data.substring(182, 188);
#3

Normally it should be something like:

numData = data.substring(13, 19);

It looks like something is placing lots of characters before the actual text starts? :o

  1. Problem number 3 is really making me frustrated as I tried everything I could possibly think :confused:

When I move the faders of the mixer on their own, the values received, and placed in "numData" are correct. The problem is when I move two faders at the same time. If i move them very slowly, the values are fine, but if I move them a bit faster, the variable that reads the values from the "Line In" parameter, get the values from the "Master Volume", and as the complete text of the Line In parameter is 2 or 3 characters longer than the one from the Master volume, the first numbers disappear including the decimal point. So, the data is placed in the wrong variable, actually in both variables, but one is also wrong. I marked this below on the serial output below.

// Player Volume on its own

Player Volume = 0.8577
Player Volume = 0.8577
Player Volume = 0.8577
Player Volume = 0.8636
Player Volume = 0.8696
Player Volume = 0.8756
Player Volume = 0.8654
Player Volume = 0.8654
//Master Volume on its own

Master Volume = 0.0059
Master Volume = 0.0119
Master Volume = 0.0119
Master Volume = 0.0178
Master Volume = 0.0178
Master Volume = 0.0238
Master Volume = 0.0238
Master Volume = 0.0297
Master Volume = 0.0297
Master Volume = 0.0357
Master Volume = 0.0357
Master Volume = 0.0416
Master Volume = 0.0416

// Both

Player Volume = 0.8595
Master Volume = 0.0476
Player Volume = 047652 // Same value as the line above without dedcimal. It should be different anyway
Player Volume = 0.8476
Master Volume = 0.0536
Player Volume = 053608
Master Volume = 0.0595
Player Volume = 059565
Master Volume = 0.0655
Player Volume = 065522
Master Volume = 0.0714
Player Volume = 071478
Master Volume = 0.0774
Master Volume = 0.0833
Player Volume = 083391
Master Volume = 0.0953
Player Volume = 095304
Master Volume = 0.1012
Player Volume = 101261
Master Volume = 0.1072
Player Volume = 107217
Master Volume = 0.1072
Player Volume = 107217
Master Volume = 0.1131
Player Volume = 113174
Master Volume = 0.1131
Player Volume = 113174
Player Volume = 0.7523
Master Volume = 0.1250
Master Volume = 0.1310
Player Volume = 131044
Master Volume = 0.1310
Player Volume = 131044
Player Volume = 0.7403
Master Volume = 0.1370
Player Volume = 137000
Master Volume = 0.1429
Player Volume = 142957
Player Volume = 0.7225
Master Volume = 0.1489
Player Volume = 148913
Master Volume = 0.1489
Player Volume = 148913
Master Volume = 0.1548
Master Volume = 0.1608
Player Volume = 160826

So, it looks like the code is placing the same value on both variables, but shifted because the texts have different amount of characters.

I even tried different variables for every parameter, tried make a client.read for each of them, but always the same result. The value form one is also inserted on the second one when moved at the same time :frowning:
I am at loss with these 3 annoying problems, hence my cry for help here.. Yes, I have done a lot of reading, otherwise I would never got this far.. I am quite surprised I did actually.
Also, please take in consideration that English is not my main language, and i might not understand some of the programming terms.
Thank you in advance for your help and patience.

João

#4

here is a little dump of the controls the mixer Accepts/sends.

Do you know if each of these lines is terminated with a cr/lf?

What is the longest message received?

Your message reading routine is not correct. It is better to read the entire message and then parse.

Try this, and report what you see. You can work on finding what you want if this is reading correctly.

if(client.available() > 0) //If there is stuff in the receive buffer
  {
    char textMessage[50] = {};
    byte numChars = client.readBytes(textMessage, 50);//.readBytes returns number read
    textMessage[numChars] = '\0';//null Terminator
    Serial.println(textMessage);
  }
#5

Hi thank you for replying.

cattledog:
Do you know if each of these lines is terminated with a cr/lf?

The mixer sends/receives a command per line

What is the longest message received?

It is the RTA (real Time Analyser). Some 400 or more characters. Actually I would just like to ignore this if possible somehow.

Your message reading routine is not correct. It is better to read the entire message and then parse.

Try this, and report what you see. You can work on finding what you want if this is reading correctly.

Had to cut the output a lot because only 9000 chars allowed. But you will have an idea of the output.

connecting...
connected
HTTP/1.1 200 OK
Server: rkdmixer
Content-Type: t
ext/plain

UPDATE_PLAYLIST
RTA^AAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAA=
SETS^firmware^1.0.7548-ui12
SETS^model^ui12

SETS^a.3.afs.eq.11^1000.0000000000,116.0000000000,
0.0000000000,0
SETS^a.3.eq.prname^
SETS^a.3.afs.eq
.5^1000.0000000000,116.0000000000,0.0000000000,0
S
ETS^a.3.afs.eq.6^1000.0000000000,116.0000000000,0.
0000000000,0
SETS^a.3.afs.eq.3^1000.0000000000,116
.0000000000,0.0000000000,0
SETS^a.3.afs.eq.4^1000.
0000000000,116.0000000000,0.0000000000,0
SETS^a.3.
afs.eq.9^1000.0000000000,116.0000000000,0.00000000
00,0
SETS^a.3.afs.eq.10^1000.0000000000,116.000000
0000,0.0000000000,0
SETS^a.3.afs.eq.7^1000.0000000
000,116.0000000000,0.0000000000,0
SETS^a.3.afs.eq.
8^1000.0000000000,116.0000000000,0.0000000000,0
SE
TS^a.3.afs.eq.1^1000.0000000000,116.0000000000,0.0
000000000,0
SETS^a.3.afs.eq.2^1000.0000000000,116.
0000000000,0.0000000000,0
SETS^a.3.afs.eq.0^1000.0
000000000,116.0000000000,0.0000000000,0
SETS^a.3.g
ate.prname^
SETS^a.3.dyn.prname^
SETS^a.2.afs.eq.0
^1000.0000000000,116.0000000000,0.0000000000,0
SET
S^a.2.gate.prname^
SETS^a.2.dyn.prname^
SETS^a.3.n
ame^
SETS^a.2.afs.eq.11^1000.0000000000,116.000000
0000,0.0000000000,0
SETS^a.2.afs.eq.9^1000.0000000
000,116.0000000000,0.0000000000,0
SETS^a.2.afs.eq.
10^1000.0000000000,116.0000000000,0.0000000000,0
S
ETS^a.2.afs.eq.3^1000.0000000000,116.0000000000,0.
0000000000,0
SETS^a.2.afs.eq.4^1000.0000000000,116
.0000000000,0.0000000000,0
SETS^a.2.afs.eq.1^1000.
0000000000,116.0000000000,0.0000000000,0
SETS^a.2.
afs.eq.2^1000.0000000000,116.0000000000,0.00000000
00,0
SETS^a.2.afs.eq.7^1000.0000000000,116.0000000
000,0.0000000000,0
SETS^a.2.afs.eq.8^1000.00000000
00,116.0000000000,0.0000000000,0
SETS^a.2.afs.eq.5
^1000.0000000000,116.0000000000,0.0000000000,0
SET
S^a.2.afs.eq.6^1000.0000000000,116.0000000000,0.00
00000000,0
SETS^a.1.afs.eq.9^1000.0000000000,116.0
000000000,0.0000000000,0
SETS^a.1.afs.eq.10^1000.0
000000000,116.0000000000,0.0000000000,0
SETS^a.1.a
fs.eq.7^1000.0000000000,116.0000000000,0.000000000
0,0
SETS^a.1.afs.eq.8^1000.0000000000,116.00000000
00,0.0000000000,0
SETS^a.1.afs.eq.11^1000.00000000
00,116.0000000000,0.0000000000,0
SETS^a.1.afs.eq.1
^1000.0000000000,116.0000000000,0.0000000000,0
SET
S^a.1.afs.eq.2^1000.0000000000,116.0000000000,0.00
00000000,0
SETS^a.1.afs.eq.0^1000.0000000000,116.0
000000000,0.0000000000,0
SETS^a.1.afs.eq.5^1000.00
00000000,116.0000000000,0.0000000000,0
SETS^a.1.af
s.eq.6^1000.0000000000,116.0000000000,0.0000000000
,0
SETS^a.1.afs.eq.3^1000.0000000000,116.000000000
0,0.0000000000,0
SETS^a.1.afs.eq.4^1000.0000000000
,116.0000000000,0.0000000000,0
SETS^a.1.gate.prnam
e^
SETS^a.2.eq.prname^
SETS^a.2.name^
SETS^m.dyn.p
rname^
SETS^m.eq.prname^
SETS^m.name^MASTER
SETS^s
ettings.udpcoms^[]
SETS^var.afsdata^fstack;;;fstac
k/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/
fstack;;;fstack/fstack;;;fstack
SETS^settings.bloc
k.pass^
SETS^var.currentPlaylist^
SETS^var.current
Track^
SETS^var.currentSnapshot^Ethernet Test
SETS
^schema^6
SETS^flavour^1
SETS^type^8ch
SETS^var.cu
rrentShow^Joao Alves
SETS^var.rta^
SETS^var.testco
de^
SETS^i.1.instrument^
SETS^i.1.iosyscmd^
SETS^i
.1.iosysname^
SETS^i.1.name^ACOUSTICGTR
SETS^i.1.d
yn.prname^
SETS^i.1.eq.prname^
SETS^i.0.dyn.prname
^My Guitar 7 Feb 18
SETS^i.0.eq.prname^Guitar Esse
n New
SETS^i.0.gate.prname^
SETS^i.0.iosysname^
SE
TS^i.0.iosyscmd^
SETS^i.0.name^ELECTRICGTR
SETS^i.
0.instrument^
SETS^m.gate.prname^
SETS^m.afs.eq.10
^1000.0000000000,116.0000000000,0.0000000000,0
SET
S^m.afs.eq.11^1000.0000000000,116.0000000000,0.000
0000000,0
SETS^m.afs.eq.8^1000.0000000000,116.0000
000000,0.0000000000,0
SETS^m.afs.eq.9^1000.0000000
000,116.0000000000,0.0000000000,0
SETS^m.afs.eq.2^
1000.0000000000,116.0000000000,0.0000000000,0
SETS
^m.afs.eq.3^1000.0000000000,116.0000000000,0.00000
00000,0
SETS^m.afs.eq.1^1000.0000000000,116.000000
0000,0.0000000000,0
SETS^m.afs.eq.6^1000.000000000
0,116.0000000000,0.0000000000,0
SETS^m.afs.eq.7^10
00.0000000000,116.0000000000,0.0000000000,0
SETS^m
.afs.eq.4^1000.0000000000,116.0000000000,0.0000000
000,0
SETS^m.afs.eq.5^1000.0000000000,116.00000000
00,0.0000000000,0
SETS^m.afs.eq.0^1000.0000000000,
116.0000000000,0.0000000000,0
SETS^i.4.iosyscmd^
S
ETS^i.4.iosysname^

S
ETS^a.0.afs.eq.7^1000.0000000000,116.0000000000,0.
0000000000,0
SETS^a.0.afs.eq.8^1000.0000000000,116
.0000000000,0.0000000000,0
SETS^a.0.afs.eq.5^1000.
0000000000,116.0000000000,0.0000000000,0
SETS^a.0.
afs.eq.6^1000.0000000000,116.0000000000,0.00000000
00,0
SETS^a.0.afs.eq.11^1000.0000000000,116.000000
0000,0.0000000000,0
SETS^a.0.afs.eq.9^1000.0000000
000,116.0000000000,0.0000000000,0
SETS^a.0.afs.eq.
10^1000.0000000000,116.0000000000,0.0000000000,0
S
ETS^a.0.afs.eq.0^1000.0000000000,116.0000000000,0.
0000000000,0
SETS^a.0.afs.eq.3^1000.0000000000,116
.0000000000,0.0000000000,0
SETS^a.0.afs.eq.4^1000.
0000000000,116.0000000000,0.0000000000,0
SETS^a.0.
afs.eq.1^1000.0000000000,116.0000000000,0.00000000
00,0
SETS^a.0.afs.eq.2^1000.0000000000,116.0000000
000,0.0000000000,0
SETS^a.0.gate.prname^
SETS^a.0.
dyn.prname^
SETS^s.3.dyn.prname^
SETS^s.3.eq.prnam
e^
SETS^a.0.eq.prname^Test
SETS^a.0.name^
SETS^s.3
.gate.prname^
SETS^s.2.eq.prname^
SETS^s.2.name^
S
ETS^s.3.name^
SETS^s.2.gate.prname^
SETS^s.2.dyn.p
rname^
SETD^a.0.link2master^0
SETD^a.2.link2master
^0
SETD^a.3.link2master^0
SETD^a.1.link2master^0
S
ETD^a.3.afs.livelift^0
SETD^a.3.afs.sensitivity^0.
5
SETD^a.3.afs.clearlive^0
SETD^a.3.afs.clearfixed
^0
SETD^var.mtk.present^0
SETD^afs.enabled^1
SETD^
a.3.afs.fmode^0
SETD^a.3.afs.cmode^0
SETD^a.3.afs.
enabled^0
SETD^a.3.afs.logic^1
SETD^a.3.afs.cleara
ll^0
SETD^a.3.afs.numtotal^12
SETD^a.3.afs.numfixe
d^6

SETD^a.3.eq.peak.2^0.5
S
ETD^a.3.gate.release^0.3150737134
SETD^a.3.gate.ho
ld^0.3042441765
SETD^a.3.gate.thresh^0
SETD^a.3.ga
te.attack^0
SETD^a.3.gate.depth^0
SETD^a.3.dyn.rel
ease^0.48828125
SETD^a.3.dyn.outgain^0.3334960938

SETD^a.3.dyn.softknee^0
SETD^a.3.dyn.attack^0.3437
5
SETD^a.3.gate.prmod^0
SETD^a.3.gate.bypass^0
SET
D^a.3.gate.enabled^1
SETD^a.3.dyn.prmod^1

BAQCAgAAAAAAAHcACgAADPcAAAAAAPcAAAAAAHcAAAAAAHcAAA
AAAHcAAAAAAHcAAAAAAHcAAAAAAPcAAAAAAPcAAAAAAAD3AAAA
AAAA9wAAAAAAAPcAAAAAAAD3AAAAAAAA9wAAAAAAAPcAAAAAAA
D3AAAAAAAA9wAABQL3AAAAAPcAAAYE9wAADQv3AAAAAPcAAAAA
9wAAAAAA9wAAAAAA9w==
RTA^AAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^var.s
pioa^8151

More than 50 for sure. :slight_smile:

#6

And you can see that there are errors on the output… Some lines were split…
Also, This data is continuously arriving and being read. This is real time.
There is the possibility of more devices connected to the mixer, and when one of them makes changes to the settings, all the others must sync.

João

#7

The mixer sends/receives a command per line

Ok, there's a fixed terminator.

Thy this reading routine

if(client.available() > 0) //If there is stuff in the receive buffer
  {
    char textMessage[400] = {};
    byte numChars = client.readBytesUntil('\n',textMessage,400);//.readBytesUntil returns number read
    textMessage[numChars] = '\0';//null Terminator
    Serial.println(textMessage);
  }
#8

This one doesn't read anything.. It just prints Connected..

#9

What if you go back to the readByte() version with 400 instead of 50 for the length.
I think there may be new line and carriage returns at the start of the message or embedded in the message.

#10

I just realize that for these big message, the length variable holding the return value from readBytes() or readBytesUntil() needs to be an int and not a byte.

if(client.available() > 0) //If there is stuff in the receive buffer
  {
    char textMessage[400] = {};//size for max message
    int numChars = client.readBytes(textMessage, 400);//.readBytes returns number read
    //int  numChars = client.readBytesUntil('\n',textMessage,400);
    textMessage[numChars] = '\0';//null Terminator
    Serial.println(textMessage);
  }
#11

Hi, sorry, had to go to sleep..

Some lines are being splitted and printed on the next line.. See the output here:

SETS^a.2.afs.eq.1^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.2.afs.eq.2^1000.0000000
000,116.0000000000,0.0000000000,0  // belongs to the line above
SETS^a.2.afs.eq.3^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.2.afs.eq.4^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.3.name^
SETS^a.2.dyn.prname^
SETS^a.2.afs.eq.0^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.2.gate.prname^
SETS^a.2.eq.prname^
SETS^a.2.name^
SETS^a.1.afs.eq.3^1000.0000000000,116.0000000000,0.0000000000,0
SETS^a.1.afs.eq.4
^1000.0000000000,116.0000000000,0.0000000000,0  // belongs to the line above

Output is also far too slow.. About 6 seconds since I move a control on the mixer, and I see it on the serial monitor.
Thanks

João

#12

if i change it to

int  numChars = client.readBytesUntil('\n',textMessage,400);

it seems to print the output correctly… No split lines.

João

#13

wow, you were absolutely right.. detection wasn't being made properly :slight_smile: I applied your idea to the code I was running before, and now it seems to be working correctly. All the 3 problems seem to have been solved, the "indexOf" is returning the correct value, and the icing on top of the cake is that it runs fast.. Real time.. Thank you so much for your help.. Please keep following this topic. I will be posting updates.. here is the relevant code.:

void loop()
{
  // if there are incoming bytes available
  // from the server, read them and print them:

  int len = client.available();
  client.println("ALIVE\n");

  
  if (len > 0) {
    byte buffer[512];
    if (len > 512) len = 512;
    client.readBytesUntil('\n', buffer, len); // Read Data sent by the audio mixer
  
    
    if (buffer > 0) {
      data += (char*)buffer; // Convert the data to String
     // Serial1.print(data);
 
      // Master Volume

      if (data.length() > 0) {  // If the String has data
       // Detect = data.indexOf("SETD^m.mix^");
        if (data.indexOf("SETD^m.mix^") >= 0) // First character's Index is 169. Normally should be 0
         
          
        {
//Serial1.print(data.indexOf("SETD^m.mix^"));
          numData = data.substring(11, 17); // This takes the numeric value
          // from "SETD^m.mix^0.868486" and places it on another string.

          masterVol = numData.toFloat(); // Puts the value extracted in a "float" type variable
          Serial1.print("Master Volume = ");
          Serial1.println(numData);
        // Serial1.println(masterVol * 100); // Converts to Percentage and makes
          // it easier to use
          numData = ""; //resets this variable
        }

        

        // Line In Mix Fader

       // Detect1 = data.indexOf("SETD^l.0.mix^");
        if (data.indexOf("SETD^l.0.mix^") >= 0) // First character Index is 169
        {

          numData = data.substring(13, 19);

          playerVol = numData.toFloat();
          Serial1.print("Line In Volume = ");
          Serial1.println(numData);
        //  Serial1.println(playerVol * 100);
          numData = "";
        }

        

        // Channel 8 Mix Fader

       // Detect2 = data.indexOf("SETD^i.7.mix^");
        if (data.indexOf("SETD^i.7.mix^") >= 0) // First character Index is 169
        {

          numData = data.substring(13, 19);

         // Serial1.println(numData);
          playerVol = numData.toFloat();
          Serial1.print("Main Mix Ch8 = ");
          Serial1.println(numData);
         // Serial1.println(playerVol * 100);
          numData = "";
        }
        
      }
      
    }
    data = ""; // Reset the main String
    byteCount = byteCount + len;
  }


  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial1.println();
    Serial1.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while (true);
  }

There is still a little problem. when the value extracted from the text message is 0 or 1 (no decimals), I get something like this.

Master Volume = 0.0595
Master Volume = 0.0416
Master Volume = 0AAAAA
.....

Master Volume = 0.8875
Master Volume = 0.9887
Master Volume = 1AAAAA

But I think all will be ok, after converting to float.

João

#14

Still have problems. :frowning: Never ending story. It looks like the code is writing the new line without erasing what was there before. Not clearing the bytes if that makes sense? For example, this serial output from the String "data"

SETD^f.1.gate.enabled^1333496093845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.2.mix^0.75417719623496093845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.2.pan^0.55417719623496093845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.2.mute^055417719623496093845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.3.pan^0.59623496093845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.3.value^0.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.3.mute^00.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.3.post^00.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.2.pan^0.5.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.2.value^0.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.2.mute^00.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
SETD^f.1.aux.2.post^00.758536602845 18k;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstack/fstack;;;fstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

It looks like it is writing the new data on top of the old one without deleting it. Makes sense?

This here:

SETD^f.1.aux.3.value^0.758536602845 
SETD^f.1.aux.3.mute^00.758536602845 18k;;

The mute value looks identical to the one above. because it was writen over it. It should be:

SETD^f.1.aux.3.mute^0

Thanks

#15

Well, it looks like I should have paid more attention to what you wrote :frowning:

Thanks a lot for your help. Really appreciated.

if (client.available() > 0) 
  {

    client.println("ALIVE\n");

    byte buffer[512];
    int len = client.readBytesUntil('\n', buffer, 512); // Read Data sent by the audio mixer
    buffer[len] = '\0';//null Terminator

    if (buffer > 0) {
      data += (char*)buffer; // Convert the data to String
      // Serial1.println(data);


     
      // Master Volume

      if (data.length() > 0) {  // If the String has data

        // Detect = data.indexOf("SETD^m.mix^");
        if (data.indexOf("SETD^m.mix^") >= 0) // First character's Index is 0. 

        {

          numData = data.substring(11, 17); // This takes the numeric value
          // from "SETD^m.mix^0.868486" and places it on another string.

          masterVol = numData.toFloat(); // Puts the value extracted in a "float" type variable
          Serial1.print("Master Volume = ");
          Serial1.println(numData);
          // Serial1.println(masterVol * 100); // Converts to Percentage and makes
          // it easier to use
          numData = ""; //resets this variable
        }



        // Line In Mix Fader

        // Detect1 = data.indexOf("SETD^l.0.mix^");
        if (data.indexOf("SETD^l.0.mix^") >= 0) // First character Index is 0
        {

          numData = data.substring(13, 19);

          playerVol = numData.toFloat();
          Serial1.print("Line In Volume = ");
          Serial1.println(numData);
          //  Serial1.println(playerVol * 100);
          numData = "";
        }



        // Channel 8 Mix Fader

        // Detect2 = data.indexOf("SETD^i.7.mix^");
        if (data.indexOf("SETD^i.7.mix^") >= 0) // First character Index is  0

          numData = data.substring(13, 19);

          // Serial1.println(numData);
          playerVol = numData.toFloat();
          Serial1.print("Main Mix Ch8 = ");
          Serial1.println(numData);
          // Serial1.println(playerVol * 100);
          numData = "";

        }
      }
    }
    data = ""; // Reset the main String
  }


  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial1.println();
    Serial1.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while (true);
  }
#16

Well, it looks like I should have paid more attention to what you wrote :frowning:

Yes, the null terminator is important.

I don't think your conversion to the String object is quite correct.

    byte buffer[512];
    int len = client.readBytesUntil('\n', buffer, 512); // Read Data sent by the audio mixer
    buffer[len] = '\0';//null Terminator

    if (buffer > 0) {
      data += (char*)buffer; // Convert the data to String
      // Serial1.println(data);

buffer is a pointer to a memory location of a declared array buffer[512] and will always have a value >0.

Since you have a null terminated character array you can simple use

data = buffer;

I recommend avoiding the use of Strings, as they can cause memory fragmentation and reliability issues. You are fairly close to being able to use character arrays everwhere instead of Strings, but it will take some additional work.

I would see if better memory management with the Strings can keep the program working reliably. Check out the use of the .reserve() function.
https://www.arduino.cc/reference/en/language/variables/data-types/string/functions/reserve/

Let's see how the program runs over time to see if you have an issue with the Strings.

#17

cattledog:
Since you have a null terminated character array you can simple use

data = buffer;

Buffer is byte, and data is String. As far as I know we can't just equal them? we must do a conversion using "data += (char*)buffer; "??

I recommend avoiding the use of Strings, as they can cause memory fragmentation and reliability issues. You are fairly close to being able to use character arrays everwhere instead of Strings, but it will take some additional work.

The problem is that I am using stuff like indexOf, SubString, etc. This is my first attempt at extracting data from some sort of serial.. I didn't even know that these existed :slight_smile: They seem to be working..

I would see if better memory management with the Strings can keep the program working reliably. Check out the use of the .reserve() function.

Thanks. Will have a read for sure..

Let's see how the program runs over time to see if you have an issue with the Strings.

I use two Strings only (data and numData), Their data is used for all parameters. Two should be ok? Specially when running on a STM32F401CC chip? :slight_smile: It's 84Mhz, 256KB flash, and 64KB RAM. It should be Ok, right? :wink: :slight_smile:
But I still have to add lots of stuff to this code. MIDI, Buttons, and 12 rotary encoders using 2 MCP23017 pin expanders. I know, I am crazy.. :slight_smile:
Let's hope the encoders play well with this code..
At this time I am using only 7% of the RAM and the serial detection code is nearly complete. This is the only code that uses Strings.. There are a bunch of "if" statements though :slight_smile:

João

#18

Buffer is byte

No, buffer[512] is a byte array and with the way you add the terminator to it when read, it can be assigned to a String.

You might need data = (char*)buffer but I don't think so from my testing. The String object treats the bytes like chars without the cast.

Did you try data = buffer?

#19

Not yet, but I will..
I still didn't because the way it is now seems to be working reliably, and I moved on with some other code, just that i have the feeling that i am moving forward.. Also my PC is giving me the Blue Screen of Death.. Windows, right?
Thanks a lot for your help.. You really sent me in the right direction. thank you.
Le't also see how it goes after I add the 12 rotary encoders. :o

João

#20

Depending on how things go, you may need to change the reading technique from readBytesUntil() to one which is "non blocking" and follow the methods of Robin2's excellent tutorial.Serial Input Basics

If the entire program is non blocking and loops fast, the read with end marker example can replace the readBytesUntil().

I'm not familiar with Ethernet and all the protocols, and I have never been clear whether your message is available in bits and pieces, or whether it is fully received and assembled in some buffer, before becoming available to read.