I have a system that has some 60+ sensors in it. I needed to store the data on a CF then latr send it to another system.
I am using a routine like this.
const size_t capacity = JSON_ARRAY_SIZE(60) + JSON_OBJECT_SIZE(3);
DynamicJsonDocument doc(capacity);
char outbuffer[512];
File ReadpackdataFile = SD.open("packedJsonLog.txt");
if (ReadpackdataFile) {
// read from the file until there's nothing else in it:
while (ReadpackdataFile.available()) {
char buff[4];
sprintf(buff, "%02X ", ReadpackdataFile.read());
strcat(outbuffer, buff);
//Serial.print(buff);
}
// close the file:
ReadpackdataFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening packedJsonLog.txt");
}
deserializeMsgPack(doc, outbuffer);
Serial.print("Output Buffer");
Serial.println(outbuffer);
I seem to be however getting some garbage at the start that is causing the deserialization not to work.
Output Buffer⸮⸮?H83 A4 6E 6F 64 65 A5 65 62 2D 30 31 A2 74 73 CE 5D 32 9A D2 A4 64 61 74 61 DC 00 3C CA 42 92 FB 5F CA 42 6E 97 27 CA 40 B9 27 83 CA 42 D0 37 C2 CA 42 A7 BE F6 CA 42 CE F7 5F CA 42 D0 DE 17 CA 42 38 A8 26 CA 41 A7 58 0F CA 42 21 40 39 CA 42 EE 6D 51 CA 40 3B E0 CA CA 42 03 F3 7C CA 42 C4 BE E4 CA 42 06 E3 56 CA 42 D8 DE 25 CA 42 B0 4C C9 CA 3F CC 8A 36 CA C1 11 F6 F2 CA 42 8E 9B 54 CA 42 C9 24 73 CA 42 36 BC BB CA 42 54 24 FF CA 42 17 4A 88 CA 41 B2 D1 0D CA 41 CB 29 DF CA 42 AF 57 7F CA 42 5F 9C 3E CA 42 1F 66 FB CA C0 0F 13 70 CA 42 61 AA D5 CA C1 90 62 6F CA 41 E0 6B F7 CA 42 D9 91 72 CA 42 47 FD 88 CA 42 DD 60 EF CA 40 FD 8C 1A CA BD 0F 6F E0 CA 42 3F 47 43 CA 42 84 4F CC CA 42 7C EF 35 CA 42 34 8A 8D CA 42 8F F8 FC CA 42 B2 A7 DD CA 42 AF F5 A6 CA BE 5D 3C 5D CA 42 65 82 D8 CA 42 B3 EF 9A CA 42 78 CC 92 CA 42 10 27 05 CA 41 E3 3E 32 CA 41 93 2C 4A CA 41 F5 67 5F CA 42 06 AC EA CA 41 F2 4F 8B CA 41 8E F3 F6 CA 42 D7 39 2B CA 42 A1 EB 68 CA 42 A7 2E B4 CA 42 C2 49 0A
If I print the output using the "Serial.print(buff);" part i do not see the same garbage at the start.
Im not sure where it is coming from and im not sure if the way I have chosen to read the line from the CF is quite the right way to do things so would appreciate any thoughts on this or advice as to doing it another way.
The reason I have opted for deserializeMsgPack() is that I have a fair amount of data and it needs to be transmitted over the network so trying to reduce the bytes as much as possible so as to reduce transmission times and the amount of information that will eventually be sent over the main link back to a database.
Is "outbuffer" a local variable? If so, it isn't initialized. When you use 'strcat()' to add a string to the end of the buffer, it finds the end by searching for the first null character. Try making the first character a null:
johnwasser:
Is "outbuffer" a local variable? If so, it isn't initialized. When you use 'strcat()' to add a string to the end of the buffer, it finds the end by searching for the first null character. Try making the first character a null:
char outbuffer[512] = {0};
or
char outbuffer[512];
outbuffer[0] = 0;
Thanks @johnwasser that worked a treat!
and the information stored in outbuffer is excatly what I would have expected.
83 A4 6E 6F 64 65 A5 65 62 2D 30 31 A2 74 73 CE 5D 34 7D 96 A4 64 61 74 61 DC 00 3C CA C0 F6 07 DC CA C1 90 DB 3D CA 42 52 EB 8A CA 42 20 28 2C CA 42 AC 47 8C CA 42 B2 32 71 CA 42 04 21 5D CA 42 75 7D C4 CA C1 8D 49 9C CA 42 AC EC 74 CA 42 3D 69 74 CA 41 3C 01 04 CA 42 87 32 1E CA 42 60 99 AA CA 41 98 AF B8 CA C1 97 AF F3 CA 42 BE F6 A3 CA C0 77 22 0C CA C1 6E 6C 2B CA 40 88 82 0D CA 3F 70 B0 BA CA 42 A2 BE CC CA 42 16 EA 6E CA C0 2E 53 78 CA 42 49 90 0F CA 42 69 3D 96 CA 42 CF 13 9F CA 42 85 2E 5D CA 42 75 C3 24 CA 42 4A 50 39 CA 42 DC E5 5A CA 41 78 DC 84 CA 42 7A 9D 64 CA 41 EF BA EF CA 42 C7 05 E5 CA 42 1C 60 FB CA 41 A7 65 6E CA 40 97 9B 2C CA 42 E1 E1 DF CA C1 05 D7 89 CA 41 BB 15 61 CA 41 18 D0 AA CA 42 DD 7E FB CA C1 34 6F BA CA 42 DA 0E 78 CA 40 DE 73 8B CA 42 21 57 EE CA 42 BE A8 E2 CA 42 A2 12 C4 CA 41 A9 06 63 CA C0 CE C7 A9 CA 42 DD 25 00 CA 42 EF 80 A8 CA C0 AE 8D 14 CA 42 98 6B 47 CA 42 B6 42 1D CA 41 EA 70 4A CA 40 73 BF 55 CA 42 D4 49 AC CA 40 F2 95 2D
If i take the output and paste it in to msgpack-lite demo it decodes correctly however for some reason its if i try to deserialise it using the following its not showing up with any data.
const size_t capacity = JSON_ARRAY_SIZE(60) + JSON_OBJECT_SIZE(3);
DynamicJsonDocument doc(capacity);
DeserializationError error = deserializeMsgPack(doc, outbuffer);
String nodename = doc["node"];
long timestamp = doc["ts"];
float ts1 = doc["data"][0]; //just get the first temperature for testing
Serial.println("Stored Data");
Serial.print("Node: ");
Serial.print(nodename);
Serial.print(" Timestamp: ");
Serial.println(timestamp);
Serial.print("Sensor Data: ");
Serial.println(ts1);
The Jsondoc should contain something like the following (truncated as there are so many sensors)
Don't you think that testing the value of error might be important?
Yes however there were no errors coming back.
I found the issue. now why I thought I had to convert the data being pulled in from the CF to hex then feed that to deserializeMsgPack is anyone's guess. Maybe it was the product of a 24 hour stint staring at the screen and trying to make things more difficult than they needed to be. how knows.
But Taking the data directly from the CF in to the buffer and feeding that in to deserializeMsgPack() worked perfectly. lol.
Thank you guys for all your help. Now to clean up the mess and add a checksum so it can be sent out to the mesh network and across a lora gateway.
