Shift Register Chain stops sending correct sensor data after 32 registers

shnlmn · December 12, 2023, 4:03am

I have created a series of PCBs to collect sensor data from an array of hall effect sensors. The project description is here LED Pegboard with 2,000 sensors

The problem... after hundreds of hours of preparing, designing and testing I'm now installing and I am hitting an issue. No matter what configuration I use, I cannot read from more than 2 of my custom built PCB boards...

For the code, I'm just running the CS and SCLK manually with a loop over all the bits. Its very slow, each clock pulse is 1000 microseconds, which is fine for the use case here. The hall effect sensors are pulled high. Every sensor when triggered returns low, as expected, up to the sensors on the third board. At that point, triggering one of the sensors on the first register (sometimes its the first, sometimes its the 7th.. ) returns every bit after it as low. I have a total of 9 boards in groups of three, so I've moved around the boards and tried many in the third position and the result is always the same, so I don't think it is an issue with the PCB.

I have one hint, in that I was having similar issues with noise on just one board which I solved by shortening my ground line to the ESP32 board as much as possible, or using two wires... I'm not sure why that solved the noise problem, but perhaps it is linked to this problem.

My fix at this moment is to add another ESP32 for the third board. It seems to work fine. But I'd like to understand why this issue is happening.

Below is a photo of the custom PCBs I made. It is only partially connected in this photo, but my question stands for fully connected boards. Its a mess, I know, but bear with me:

The ethernet cables are connected to hall effect sensors, each cable carrying 8 sensor signals. The boards direct the signals to shift registers which are then controlled by signals from the ESP32 board. The boards also trace the power, clock and chip select signals to all the shift registers. The data line daisy-chains from one board to the next.
And here is the code:

// Script to read 74HC165 shift register and send over Websockets

#include <Arduino.h>
#include <WiFi.h>
#include <WiFiMulti.h>
#include <WiFiClientSecure.h>
#include <WebSocketsClient.h>


// Setup Websocket
WiFiMulti WiFiMulti;
WebSocketsClient webSocket;

// WIFI Creds
const char *ssid = "SSID"; // <======== EDIT THIS
const char *password = "password"; // <==== EDIT THIS

// Device ID
const char *DEVICE_ID = "/?id=1"; // <========= EDIT THIS

// frequency
const int freq = 1000;

// Serial Communication Pins
const int csPin = 5;
const int clkPin = 3;
const int dataPin = 16;
const int register_count = 16 + 16 + 16; // 16 + 16 + 12 for panels 1 & 3, 16 * 3 for panel 2
const int bits = 8 * register_count;

// DEVICE PIN
const int LED_PIN = 15;

// Array to store shift register data
int dataArray[bits];

// Read 74HC165 
void read74HC165(int csPin, int clkPin, int dataPin, int *dataArray, int bits)
{
  // Load parallel data into shift register
  digitalWrite(csPin, LOW);
  delayMicroseconds(freq);
  digitalWrite(csPin, HIGH);

  // Read serial data from shift register
  for (int i = 0; i < bits; i++)
  {
    dataArray[i] = digitalRead(dataPin);
    digitalWrite(clkPin, HIGH);
    delayMicroseconds(freq);
    digitalWrite(clkPin, LOW);
    delayMicroseconds(freq);
  }
}


void webSocketEvent(WStype_t type, uint8_t * payload, size_t length) {

	switch(type) {
		case WStype_DISCONNECTED:
			Serial.printf("[WSc] Disconnected!\n");
			break;
		case WStype_CONNECTED:
			Serial.printf("[WSc] Connected to url: %s\n", payload);

			// send message to server when Connected
			webSocket.sendTXT("Connected");
			break;
		case WStype_TEXT:
			Serial.printf("[WSc] get text: %s\n", payload);

			// send message to server
			// webSocket.sendTXT("message here");
			break;
		case WStype_ERROR:			
		case WStype_FRAGMENT_TEXT_START:
		case WStype_FRAGMENT_BIN_START:
		case WStype_FRAGMENT:
		case WStype_FRAGMENT_FIN:
			break;
	}

}

void setup() {
	// Serial.begin(921600);
	Serial.begin(115200);

	// Set pin modes
	pinMode(csPin, OUTPUT);
	pinMode(clkPin, OUTPUT);
	pinMode(dataPin, INPUT);
	pinMode(LED_PIN, OUTPUT);

	WiFiMulti.addAP(ssid, password);

	//WiFi.disconnect();
	while(WiFiMulti.run() != WL_CONNECTED) {
		digitalWrite(LED_PIN, LOW);
		delay(300);
		digitalWrite(LED_PIN, HIGH);
		delay(300);
	}

	Serial.printf("[SETUP] WiFi connected\n", WiFi.localIP());

	// server address, port and URL
	webSocket.begin("192.168.254.137", 5000, "/?id=1");

	// event handler
	webSocket.onEvent(webSocketEvent);

	// try ever 5000 again if connection has failed
	webSocket.setReconnectInterval(5000);

}

void loop() {
	webSocket.loop();
	
	// // Read 74HC165
	read74HC165(csPin, clkPin, dataPin, dataArray, bits);

	// Send over Websockets
	String message = "";
	for (int i = 0; i < bits; i++)
	{
		message += String(dataArray[i]);
	}
	webSocket.sendTXT(message);

}

An example output on the server side, with the faulty sensor reading, looks like this:

| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | <----- BIT INDEX
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | <----- FIRST BOARD STARTS HERE
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | <----- SECOND BOARD STARTS HERE
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | <--- THIRD BOARD STARTS HERE
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

And, for the sake of interest, here is the front of the project with the LEDS lit up to random values:

And a closeup of the actual sensor mounts:

kolaha · December 12, 2023, 4:12am

wow!

kolaha · December 12, 2023, 4:19am

you send every bit as byte ? is sending 8 bit as one byte prohibited? if yes then:

// Script to read 74HC165 shift register and send over Websockets

#include <Arduino.h>
#include <WiFi.h>
#include <WiFiMulti.h>
#include <WiFiClientSecure.h>
#include <WebSocketsClient.h>


// Setup Websocket
WiFiMulti WiFiMulti;
WebSocketsClient webSocket;

// WIFI Creds
const char *ssid = "SSID"; // <======== EDIT THIS
const char *password = "password"; // <==== EDIT THIS

// Device ID
const char *DEVICE_ID = "/?id=1"; // <========= EDIT THIS

// frequency
const int freq = 1000;

// Serial Communication Pins
const int csPin = 5;
const int clkPin = 3;
const int dataPin = 16;
const int register_count = 16 + 16 + 16; // 16 + 16 + 12 for panels 1 & 3, 16 * 3 for panel 2
const int bits = 8 * register_count;

// DEVICE PIN
const int LED_PIN = 15;

// Array to store shift register data
char dataArray[bits + 1] = {0};

// Read 74HC165
void read74HC165() {
  // Load parallel data into shift register
  digitalWrite(csPin, LOW);
  delayMicroseconds(freq);
  digitalWrite(csPin, HIGH);
  
    // Read serial data from shift register
    for (int i = 0; i < bits  i++)  {
      dataArray[i] = digitalRead(dataPin) + '0';
      digitalWrite(clkPin, HIGH);
      delayMicroseconds(freq);
      digitalWrite(clkPin, LOW);
      delayMicroseconds(freq);
    }
}


void webSocketEvent(WStype_t type, uint8_t * payload, size_t length) {

  switch (type) {
    case WStype_DISCONNECTED:
      Serial.printf("[WSc] Disconnected!\n");
      break;
    case WStype_CONNECTED:
      Serial.printf("[WSc] Connected to url: %s\n", payload);

      // send message to server when Connected
      webSocket.sendTXT("Connected");
      break;
    case WStype_TEXT:
      Serial.printf("[WSc] get text: %s\n", payload);

      // send message to server
      // webSocket.sendTXT("message here");
      break;
    case WStype_ERROR:
    case WStype_FRAGMENT_TEXT_START:
    case WStype_FRAGMENT_BIN_START:
    case WStype_FRAGMENT:
    case WStype_FRAGMENT_FIN:
      break;
  }
}

void setup() {
  // Serial.begin(921600);
  Serial.begin(115200);

  // Set pin modes
  pinMode(csPin, OUTPUT);
  pinMode(clkPin, OUTPUT);
  pinMode(dataPin, INPUT);
  pinMode(LED_PIN, OUTPUT);

  WiFiMulti.addAP(ssid, password);

  //WiFi.disconnect();
  while (WiFiMulti.run() != WL_CONNECTED) {
    digitalWrite(LED_PIN, LOW);
    delay(300);
    digitalWrite(LED_PIN, HIGH);
    delay(300);
  }

  Serial.printf("[SETUP] WiFi connected\n", WiFi.localIP());

  // server address, port and URL
  webSocket.begin("192.168.254.137", 5000, "/?id=1");

  // event handler
  webSocket.onEvent(webSocketEvent);

  // try ever 5000 again if connection has failed
  webSocket.setReconnectInterval(5000);
}

void loop() {
  webSocket.loop();

  // // Read 74HC165
  read74HC165();

  // Send over Websockets
  webSocket.sendTXT(dataArray);
}

dougp · December 12, 2023, 5:43am

Is there any buffering of CS/MOSI/SCLK? Possibly a fanout issue?

If you have the tools (scope, logic analyzer) it would be useful to check the waveforms as they get farther away from the source.

Wawa · December 12, 2023, 6:32am

I see you power the shift registers with 5volt, and drive them with 3.3volt logic from a weak ESP pin, or do you have a level translators and buffers between ESP and shift registers. Do you have any termination at the end of those lines. Love to see a diagram of this.
Leo..

shnlmn · December 12, 2023, 7:12am

I don't have any buffers per se, and while I'm level-shifting the incoming data down to 3.3v, I am not up-leveling the clock and chip select... I could see how that might be an issue. That would be a trivial fix with the bi-directional level shifter.

shnlmn · December 12, 2023, 7:15am

Hmm, no buffering... i'm not sure how to implement that. I have a hand-held oscilliscope/multimeter, I should sit down and check the waves at each board... I am not sure my scope is particularly the right tool for this, though, it seems quite limited. I'm still learning how to use it.

shnlmn · December 12, 2023, 7:25am

I'm not entirely sure what you are proposing here. Are you asking why I'm sending the array of bits as a string over websockets? If so, it is solely to make testing and debugging the sensors easier both from the serial terminal of the ESP32 and on the server side.

Wawa · December 12, 2023, 7:32am

As long as you're not using an I2C level shifter for that.
See this page.
Leo..

kolaha · December 12, 2023, 7:58am

no, i asking why you send only 0 and 1 while it is possible to send from 0 to 255. it answers not why only 2 PCB working properly, just curious.

wvmarle · December 12, 2023, 10:47am

The fact that it is exactly at the start of the third board suggests a poor wire connection to me, specifically the data connection. Are you very sure this is not the case, e.g. by swapping the PCB order?

dataPin should be MISO, not MOSI. You're reading data.
MISO needs level shifting down to ESP levels.
The CLK connects to all chips so needs sufficient drive. A buffer is needed, also to bring this signal up to 5V.

shnlmn · December 12, 2023, 5:37pm

Sorry, did the drawing late for me, it is MISO. The level shifter is brining the data down to 3.3V.

I've tried several different configuration of boards, since I have so many. It was all the same,but to be honest I was using the same wire itself. Maybe that wire was bad? It would match the pattern of 'the thing I didn't think of becoming the issue' that this project has become.

Also... I mean, I started this project with practically no real knowledge, so I jumped into the deep end with it. I'm afraid I'm not familiar with buffering in this context, can you provide me with a resource or link I can learn more?

shnlmn · December 12, 2023, 5:38pm

Yeah, it is purely for being able to read the output in the serial display.

shnlmn · December 12, 2023, 5:42pm

I see... the I2C doesn't just work with any signal? I'm using this one for the data in. I have some of the 74AHCT125 that I used on the LED controller boards, I'll try those instead... I was concerned looking at the data sheet that they only go 3.3v to 5v, but your link shows they go down as well.

kolaha · December 12, 2023, 6:28pm

did you tried my sketch?

shnlmn · December 12, 2023, 6:37pm

The wall is installed, so I need to go to the site to properly test any solutions, but I will be able to this weekend.

dougp · December 12, 2023, 6:40pm

Wawa · December 12, 2023, 6:46pm

It's a voltage-only level shifter, with a weak HIGH and a LOW with the same current as the ESP pin.
A TTL level shifter chip has proper "power" outputs.

Did you see that I also mentioned "termination".
You might have to use a resistor to ground at the far end of the clock/data wires, to minimise ringing. Try ~470 ohm. Or, better, 1k to ground and 1k to 5volt.
Leo..

shnlmn · December 12, 2023, 7:50pm

I see, the SN74ACHT125 is a buffer, really, but will also bring the 3.3v up to 5v, or vice versa. Should i be adding one in between each board then?

Also, for terminating, here is the PCBs shown with the clock and select traces isolated. Should the termination be where I'm indicating with the red arrows, or can i do it at the top of the board? At this point, I have to jump the pins with a wire.

Also, here are some screenshots of the schematics. My original plan was to put an ESP on each board to manage the data and push it to the next, but that seemed overkill, so I jumped the Serial Input terminal to the serial input to the last register and the serial out to the Serial Ouput terminal. I changed the ESP32 board to the S2mini in the schematics, so they don't match the PCB 1 to 1, but are functionally the same:

Overall Schematic

Closeup

Shift Register Sheet "INPUT MODULE"

Wawa · December 13, 2023, 12:24am

No, just try buffering the data/clock outputs of the ESP first. And chip select.
Level shifting of the input of the ESP can be done with two resistors (1k:2k).

As said, only at one point, and at the end of the line.
Furthest away from the buffer of the ESP.
But leave that for last, so you can see what fixes your problem.
I expect you need both.
Leo..

Topic		Replies	Views
AH1751 Hall sensors + 74HC595 shift register grid (chessboard) LEDs and Multiplexing	3	1751	May 6, 2021
will an interrupt mess up my arduino serial out Interfacing	2	983	May 6, 2021
Daisy chained shift registers ESP32 General Guidance	16	201	June 24, 2025
Daisy Chained Shift Registers failing when >6 connected (decoupling issue?) LEDs and Multiplexing	7	3543	May 6, 2021
Too many shift registers? General Electronics	16	3815	May 6, 2021

Shift Register Chain stops sending correct sensor data after 32 registers

Related topics