First Arduino project - Home automation.

Projects General Guidance
1

Go gentle - this is my first foray into the world of Arduino.

Situation: I am looking to make my house an entire "Smart House", I will be re-wiring the entire property back to a distribution panel, controlled by relays, hooked up to MQTT. I will run separate cables for each socket, each light, data cables for all switches and a bunch of sensors for each room.
I am at the design stage of this project at the moment - eager to try and make each individual element work before committing it to the whole house

Question: I am looking to have all the switches connected to one Arduino. I am looking for ~51 illuminated momentary push buttons throughout the house. The LED on each button must be controlled separately to the button itself. So essentially I have 51 push buttons and 51 LEDs to control all independently.
I am not at the point of coding, this is a design question.
I have done lots of reading on shift registers etc, I think I can achieve what I want using daisy chained 74HC165s for the buttons and daisy chained 74HC595s and ULN2803s for the LEDs.
The buttons may be pressed simultaneously (think different people in different rooms - may happen to use at the same time). The LEDs will illuminate if the intended load is on, but with additional time based rules applied. It is possible that all 51 LEDs may be required to be on at the same time, although unlikely.
I have tried to do my own research, the attached Eagle file is a culmination of it. It is my first work with eagle - please point out where I can improve/learn.
The LEDs are all attached via 220ohm resistors, the capacitors are all 100uf, the inputs of the 74HC165s are all pulled low via 10k resistors.
The Arduino Mega Rev3 will be powered via 12V on the VIN pin, this will also power the LEDs. Everything else is powered via the 5V VCC pin.
Is this achievable?
Have I followed best practices?
How can I improve the design?
Does the 5V Vcc pin have capability to power all the chips?/will the 12v be able to power 51 LEDs? I used the transistor arrays so I wouldn't exceed the current draw on the I/O pin for the board.

When I get round to looking at the code - will the Mega be able to handle dealing with all the inputs, reporting their state to an MQTT broker and listening to the broker for the state of the LEDs and actioning accordingly.
Or is that asking too much?

Cheers

G

LEDs.zip (67 KB)

2

Read some of the many home automation projects, before you start re-inventing well known errors. Separate cables instead of a bus system is one such misfeature.
Also learn the Arduino basics before starting a bigger project.

3

I have read many home automation projects - I am wanting to do sockets etc as well as just lighting. Hence why I've ended up planning separate cables rather than a bus system.

In terms of the Arduino side of it - it doesn't seem overly complex, what I am looking to do is use shift registers etc to reduce the number of Arduinos I require. I am happy to learn as I go, hence asking for advice before I have committed to it. This is also not my first home automation project.

4

The existing home automation protocols don't require re-wiring your house. And, it's a LOT easier to upgrade or add something if you use one of the standards. And, I'm pretty sure it's a violation of building codes if you mix mix high & low voltage in an electrical box.

I recommend you choose one of the existing protocols and BUY controlled switches & outlets even if you want to build your master controller. It's not easy to build a switch/dimmer that fits into a wall box, works locally and remotely, and looks good. And I doubt you can buy it for the price you can buy it. Remote dimmers go for between $25 & $50 USD.

I am wanting to do sockets etc

You can buy switched outlets to replace existing outlets or there are little plug-in switches & dimmers.

I've had an X-10 system for many years that "evolved" into a combination X-10/Insteon system, Some of my X10 stuff has become unreliable and Insteon is no longer compatible with X-10 so I'm in the process of updating to Z-Wave.
I chose Z-Wave because it seems to be the most popular, with the largest selection of components. Zigbee is popular too. Apple, Amazon, and Google have formed a home automation consortium and they've chosen Zigbee so that's probably "the future".

5

I got a celling fan in the bathroom I want to control, add an ESP32, a relay, a humidity sensor, do some programming on the RPi and through MQTT Broker the whole thing connects to my web site; no wires. I thought of doing the one controller for the entire house but the wiring scheme became a monster. Wireless might be a consideration.

6

DVDdoug:
The existing home automation protocols don't require re-wiring your house. And, it's a LOT easier to upgrade or add something if you use one of the standards. And, I'm pretty sure it's a violation of building codes if you mix mix high & low voltage in an electrical box.

I recommend you choose one of the existing protocols and BUY controlled switches & outlets even if you want to build your master controller. It's not easy to build a switch/dimmer that fits into a wall box, works locally and remotely, and looks good. And I doubt you can buy it for the price you can buy it. Remote dimmers go for between $25 & $50 USD.
You can buy switched outlets to replace existing outlets or there are little plug-in switches & dimmers.

I've had an X-10 system for many years that "evolved" into a combination X-10/Insteon system, Some of my X10 stuff has become unreliable and Insteon is no longer compatible with X-10 so I'm in the process of updating to Z-Wave.
I chose Z-Wave because it seems to be the most popular, with the largest selection of components. Zigbee is popular too. Apple, Amazon, and Google have formed a home automation consortium and they've chosen Zigbee so that's probably "the future".

I currently use Zigbee for all my lights, my central heating, I've done my own alarm system based on Tasmota over MQTT etc, I've looked at the regs with regards to SELV and LV in the same distribution box, I have a friend who is a sparky and is going to commission the electrical side. There are a whole host of reasons why I want to move back to a wired system. One of the reasons I want to pull individual cables and not use a X10/Insteon system is exactly that, I don't want to be tied to any particular system. So it can't become outdated.
I have the ability to go back to stud work throughout the house, the wiring scheme does become a mess, however I have time and the desire to plan carefully, label properly etc.
I am content with the design of my light switches and sockets with push buttons, they look significantly better than my current Zigbee smart plugs and it will integrate better into how I would like a smart house (i.e it doesn't matter how you control it voice/mechanical/app etc) But, I feel this has come away from my OP.

My question is less about the design of the home automation, more to do with the design of controlling 51 LEDs and 51 push buttons using the minimum I/O pins on the arduino, and whether that design is sound.

Cheers

G

7

Your code has to poll all sensors all the time. If a change is detected the new output state has to be shifted out. That's all so far.

8

DrDiettrich:
Your code has to poll all sensors all the time. If a change is detected the new output state has to be shifted out. That's all so far.

Okay - so that seems positive. I can look at the code for it if it is going too be achievable.
Is the wiring okay? I wasn't sure on things such as the sizing/placing of the decoupling capacitors. Or whether the chips were daisy chained together correctly?

9

one of my questions goes to distance.
it sounds like you want to have 2 control lines to each swtich. one for the LED to control independent as a indicator and the other as a switch to accept user inputs.

based on that, it sounds like you will remove all power wiring to all the switches and replace with low voltage wiring.

as for each device, it sounds like you want to run a direct line to each light fixture and also to each outlet. possibly 2 lines to each duplex outlet.

based on that, you would have a panel next to your breaker panel and inside of that, you would have all your relays to control all the lights.

lots and lots of hard decisions to make.
run a power line to the first receptacle, then add in a relay box, and run a wire from there to each of the other receptacles that follow or run multiple new wires from the panel.

I would start with sparky. since he will be doing the work.
pulling out wires and drilling new holes to accomidate bundles of wires will be time consuing, costly in materials and invasive to the walls.

10

dave-in-nj:
it sounds like you want to have 2 control lines to each swtich. one for the LED to control independent as a indicator and the other as a switch to accept user inputs.

Correct

dave-in-nj:
based on that, it sounds like you will remove all power wiring to all the switches and replace with low voltage wiring.

Yes that's the plan, all light switches become low voltage wiring.

dave-in-nj:
as for each device, it sounds like you want to run a direct line to each light fixture and also to each outlet. possibly 2 lines to each duplex outlet.

based on that, you would have a panel next to your breaker panel and inside of that, you would have all your relays to control all the lights.

Also spot on, obviously some rooms will have multiple light fittings on a single cable.
Yep - well actually I will be putting the breakers inside the panel, so each line will have a breaker, a relay then a separate wire all within a large distribution panel. There will be no ring mains etc.

dave-in-nj:
lots and lots of hard decisions to make.
run a power line to the first receptacle, then add in a relay box, and run a wire from there to each of the other receptacles that follow or run multiple new wires from the panel.

Yes - that is the decision I've been facing. I have come to the conclusion I wanted all the relays in the same place (possibly split between two places - for example a panel upstairs, a panel downstairs)

dave-in-nj:
I would start with sparky. since he will be doing the work.
pulling out wires and drilling new holes to accomidate bundles of wires will be time consuing, costly in materials and invasive to the walls.

You're not wrong, I have spoken to him already - fully aware this is not a small project in terms of labour.
The basis of this post was for the schematic of the Arduino Mega, I am confident with the rest of the plan. But I just want to know if 51 LEDs and 51 switches on only 7 I/O pins of the Mega is achievable, from what I read on other posts/web searches, there are no issues daisy chaining the 74HC165s and the 74HC595s. But most I saw only had two shift registers daisy chained and spoke of the potential to do more.

This is where my knowledge runs out and I am looking for guidance/advice/opinions.

11

SPI is the buss that contorls the shift registers.

you have some basic lines, then a single channel for operation.

so, for the first group of shift registers, you use the basic lines
for the second group, you use the same basic lines, then one additonal pin to control the second group.

a few hundred is within the abilities of a mega.

you could have one channel as only outputs and a second channel as only inputs.

12

dave-in-nj:
SPI is the buss that contorls the shift registers.

you have some basic lines, then a single channel for operation.

so, for the first group of shift registers, you use the basic lines
for the second group, you use the same basic lines, then one additonal pin to control the second group.

a few hundred is within the abilities of a mega.

you could have one channel as only outputs and a second channel as only inputs.

Thank you - continuing to show how naïve I am to shift registers, is there any noticeable delay on reading inputs, especially if the input is on the last shift register in the daisy chain, which in my case would be the 7th? Likewise do they handle simultaneous button presses well? Even if the simultaneous inputs are across multiple shift registers?
In terms of the outputs, the LEDs, using the transistor array and wiring the 51 LEDs straight to 12v VIN line - is that going to be enough to power them all? Potentially all being on at the same time? Clearly I couldn't do this from the Arduino chip directly due to the current draw.
Cheers

13

when you say 'noticeable' what is your definition ?

the SPI bus only does like a million times a second so, say each of 64 registers might read/write a few thousand times a second.

14

Hi,
Welcome to the forum.

Please read the post at the start of any forum , entitled "How to use this Forum".
OR
http://forum.arduino.cc/index.php/topic,148850.0.html.
Then look down to item #7 about how to post your code.
It will be formatted in a scrolling window that makes it easier to read.

Can you please post a copy of your circuit, in CAD or a picture of a hand drawn circuit in jpg, png?

Please do not attach .zip files, in your case we need a jpg EXPORTED image from the CAD you are using.
By posting the CAD file you assume everyone has your CAD package.

Thanks.. Tom.. :slight_smile:
PS, Do not use screen capture as the resolution will be poor, your CAD should have the facility to EXPORT a jpg file.

15

dave-in-nj:
when you say 'noticeable' what is your definition ?

the SPI bus only does like a million times a second so, say each of 64 registers might read/write a few thousand times a second.

That qualifies as not noticeable for my purposes! Ha.

TomGeorge:
Hi,
Welcome to the forum.

Please read the post at the start of any forum , entitled "How to use this Forum".
OR
http://forum.arduino.cc/index.php/topic,148850.0.html.
Then look down to item #7 about how to post your code.
It will be formatted in a scrolling window that makes it easier to read.

Thank you, I've got no code as of yet - this was a design question, code to come.

TomGeorge:
Please do not attach .zip files, in your case we need a jpg EXPORTED image from the CAD you are using.
By posting the CAD file you assume everyone has your CAD package.

Thanks.. Tom.. :slight_smile:
PS, Do not use screen capture as the resolution will be poor, your CAD should have the facility to EXPORT a jpg file.

Apologies, I've attached an exported .png of the circuit. Let me know if that isn't adequate.
Cheers

Circuit.png
Circuit.png5413×4711 252 KB

Circuit.png
Circuit.png5413×4711 252 KB

16

You need to consider failure modes too - it maybe better to have several Arduinos controlling lights etc so that in the event of a failure the whole house doesn’t shutdown .

Also there is a lot of very good commercial gear out there - you have to consider maintenance , reliability, keeping it updated, what happens when you sell up, power failure, etc - commercial gear maybe the better option.

17

The Mega has a linear regulator so I would power it from a supply of less than 12v for thermal reasons.

Why do your switch indicator LEDs need a 12v supply?

Long cables and high impedance circuits are susceptible to picking up radio frequency interference. For your switch inputs I would add 20Hz RC low pass filter circuits to protect the shift registers and Arduino inputs from voltage spikes.

Don't forgert, software rolling average debounce algorithms for mechanical switches will reduce your effective scan speed by a factor of 2 or 3.

There is nothing more annoying than a user interface that fails to register inputs or is slow to indicate status changes, so I would try to keep the Mega as dumb as possible so that the switch scanning and LED driving doesn't get interrupted by blocking code that lurks in poorly designed peripheral or network libraries.

MQTT makes sense if you have a distributed system, however it sounds like you have a centralized system? I'm not sure any MQTT based system is going to give you consistent sub second response times. Layers upon layers of software are the enemy of speed.

18

I liked the X-10 it was fun to push the button, then guess when the light would come on.
In my humble opinion. you could have the buttons control the relays directly
the arduino is not needed for that.

and, you would not be the first house to have low voltage switches controlling line voltage devices.

the automation is that you could ALSO, know when things were on and you could ALSO use your phone if you chose.

19

hammy:
You need to consider failure modes too - it maybe better to have several Arduinos controlling lights etc so that in the event of a failure the whole house doesn’t shutdown .

Also there is a lot of very good commercial gear out there - you have to consider maintenance , reliability, keeping it updated, what happens when you sell up, power failure, etc - commercial gear maybe the better option.

My plan was to have spares of everything ready to go, but point is taken on failure modes.
I prefer learning myself and DIY, two reasons - one I understand what is happening, two I know how to fix it/make it exactly how I like.

mikb55:
The Mega has a linear regulator so I would power it from a supply of less than 12v for thermal reasons.

Why do your switch indicator LEDs need a 12v supply?

Okay - the LEDs don't could easily power them from a separate 5V power supply, what do you recommend for the Mega? The docs say the recommend range is 7-12V? But can operate 6-20?

mikb55:
Long cables and high impedance circuits are susceptible to picking up radio frequency interference. For your switch inputs I would add 20Hz RC low pass filter circuits to protect the shift registers and Arduino inputs from voltage spikes.

Don't forgert, software rolling average debounce algorithms for mechanical switches will reduce your effective scan speed by a factor of 2 or 3.

Now that is very useful, thank you - I will start reading up on RC low pass filters. Like you said I would rather solve it with hardware than software and reduce what I am doing with the Mega.

mikb55:
MQTT makes sense if you have a distributed system, however it sound like you have a centralized system? I'm not sure any MQTT based system is going to give you consistent sub second response times. Layers upon layers of software are the enemy of speed.

I am aiming for a centralised system in terms of the distribution panel. But I will use other programs etc to interface with the MQTT broker too, MQTT keeps it in a format that lots of software can understand and so far my experience with MQTT has been very good.

dave-in-nj:
I liked the X-10 it was fun to push the button, then guess when the light would come on.
In my humble opinion. you could have the buttons control the relays directly
the arduino is not needed for that.

and, you would not be the first house to have low voltage switches controlling line voltage devices.

the automation is that you could ALSO, know when things were on and you could ALSO use your phone if you chose.

I'd like to avoid that particular experience!
I could bypass the arduino and connect buttons directly to the relays, but that removes one layer of automation for me. I would like to be able to change the affect a button has on a software layer. For example if the particular setup I start with then doesn't become the setup I have to live with, if the better half would rather the top button did blinds rather than lights etc etc.

20

one suggestion I would highly recomend is that you terminate all wires on patch panels.
check the DMX controllers and their hard wire scheme. Cat-X connects the 8 wires is a specifc method.
using some industry standard method would make a plug and play much easier in the future. and those patch panels can be had on e-bay or any number of suppliers.

Another suggestion is it get a system working with LEDs and relays. you can do that without much effort and expand on that.

get a box of Cat-x wire,
run one wire across the basement and back to your lab for LED's
run a second for relays

once you are satisfied with that do a full room and figure out what you will want too add when you do the rest of the house.