Go Down

Topic: Multi-node CAN bus: a progress report (Read 8 times) previous topic - next topic

Graynomad

iyiyi this thread must get the gold medal for long posts :), here's another one.

Quote
Do you think that my pre-use check is not enough,

I like to catch problems as close to the source as possible.

Quote
that I should actually range check every analog read?

I'm not up on what they all do but that's quite possible.

Quote
Or even break out of the averaging as soon as one unreasonable value is read?

Not one bad reading, but don't add the bad one to the average, maybe count the bad ones and abort if you get more than 3, if it comes good assume it was noise and start averaging again but the bad readings did not get through. You've got Damping set to 20, I'm not sure what values you normally get but imagine if 10 was the current average and you get 1023 because a wire broke, that will seriously skew the average, get 5 in a row and you are in trouble.

As you say you do test later but if you were happily cruising at 500 and a wire breaks you will ramp up to 4500 in no time and the current code will think that is OK, it will reset to 2500 but that's still 5x faster than the user expected. By the time your daughter responds she could be down the stairs.

Quote
I wasn't aware of this convention.Among PerfectScript writers, preferred form is to write vVariableName

That format is Hungarian notation, disappeared years ago AFAIK and not popular with C programs I think.

Quote
Notation    Example
Standard C / GNU notation    mouse_weight
Hungarian notation    iMouseWeight
Camel case (or lower Camel case)    mouseWeight
Pascal case (or upper Camel case)    MouseWeight
All capitals case    MOUSE_WEIGHT


My preference is for C notation for class member names and local objects, camel case for most other things and all caps for defines and consts. It's not a hard rule of course and people have their own conventions.

Quote
Which (for the new building) has been in the news of late.  Was it your stuff that got hacked?

No, my stuff was put in the first building when that was new. Hacking wasn't a problem then because there was no internet and the comms link around the building was optical fibre, pretty hard to tap into.

Quote
If I discover that I can't explain it, then I know that I'm probably not doing it right.

That's a very good technique, and quite true.

Quote
It combines on one board everything that I have on 3 (MCU + CAN + DC-DC converter),

That's a good thing for reliability, connectors are always a weak point.

Quote
Does nobody know of an existing simple, well-tested protocol for those applications?

Not me and I've looked a lot. There seems to be nothing between "ascii with a checksum" and full-blown Modbus or whatever.

This is a pet subject of mine and I've spent the last couple of years (on and off) working on designs, starting from a very complex master/slave protocol to a simple multi-master version. For most of that time the topology has been async serial over a redundant ring because with an RR you can cut the cable and it has almost no affect on the network traffic.

See here for a brief description of the idea

http://www.ardweenet.com/

After all that just a few weeks ago I decided to use CAN as the base level so most of my previous work is out the window :) I still prefer an RR over a bus but truth is CAN must be very reliable given it's used in vehicles so I'll get over that.

So having decided on CAN, and given that the SAM3X has two CAN controllers, it was a natural thing to design a Due-like board to act as the "master" and that's where I am at now, distracted yet again from my network design :(

Quote
My messages contain at most 4 bytes of data and the other space is used for the bit-wise complements of these.

The CAN frame has a CRC and it is very robust, I think you are wasting half your payload here. With the full 8 bytes you can represent all numeric data types up to float.

I'll have a better read through your protocol design doc.

______
Rob

Rob Gray aka the GRAYnomad www.robgray.com

LROBBINS

#16
Jun 13, 2013, 10:33 am Last Edit: Jun 13, 2013, 10:58 am by LROBBINS Reason: 1
Quote
The board is 1.950 x 1.950 inch², just under 5x5cm².

Yes, that would fit.  How are you handling power for the (multi-node) system?

Yes, as Rob points out, connectors are a major headache and I'll describe some of my experience (and more or less crude fixes) when I reply to his post.  As far as the Phoenix goes, I just generally dislike screw-clamp connectors in a high-vibration environment - I prefer to solder and de-solder if need be.  If I had to use screw connections, I think I'd follow aircraft bus practice - a barrier strip with studs, extra-secure external star lock washers, and anti-vibration or jam nuts.
Ciao, Lenny

LROBBINS

Hi again Bernt,

Quote
- The need of a higher level CAN protocol: we all conclude that none of the well-known protocols fits our needs. ... I have difficulties to imagine that no-such protocol exists out there.
Perhaps the lack of same is understandable in the original context for which CAN was introduced: first heavy industrial vehicles and equipment like cranes, then for motor vehicles in general.  Here, proprietary interests take precedence over inter-operability.
Quote
- Robs clear example of Arduino analogRead() tells me, that, for a finalised product with serious security constraints, we need to get rid of Arduino software, and do a more secure reimplementation of a subset. But for prototypes and demonstrators, the Arduino approach stays still very attractive. If it was for driving my-self in a wheel-chair, I would probably stay forever with Arduino libraries, in an approach close to the one of John Williamson "wheelchairdriver.com" (and rely just on some amount of testing).
And as much of what Roy calls "sanity testing" in your program as can reasonably be added even if one avoids Arduino entirely.  It's like having message-testing over and above what CAN already does.  Can you really be sure that there's NO unusual circumstance in which your "perfect" code doesn't turn out to be less than perfect?  (And a completely independent Mayday shut-down is probably in order too if you don't test to the tightest of compliance standards.)
Quote
- The appealing idea of a low cost mini "inertial navigation unit" is probably good for outdoor use with sometimes slipping wheels. But inside, you can not rely on the compass sensor used in conjuction with the gyros. Anyway, more different sensors gives better chance to detect abnormal, potentially dangerous situations (as once again slipping wheels).
If you have slipping wheels indoors, the chair itself is badly engineered.  Actually, for indoors (i.e. short term corrections), the gyros alone are adequate - the compass would only be used to correct drift over long tracts.
Quote
- The programming through the CAN-bus based on Yellowsky's modification of Fabian Greifs work is coming soon. ..: Yellowsky transformed parts of Fabian's CAN debugger firmware in a simple Arduino sketch that allows us to use  as interface between the host PC and the CAN bus : standard Arduinos with CAN shield, your boards or the CANinterfacer. This does not give the full-speed CAN debugging as Fabian's CAN debugger, but it's sufficient for programming devices on the CAN bus.
If that were a meal, my mouth would be watering (don't know if French has an expression like "mouth-watering" in English, or "acqualina in bocca" in italiano).
Ciao,
Lenny

bernt


Dear Rob Dear Rob (and everybody else),


http://www.ardweenet.com/

After all that just a few weeks ago I decided to use CAN as the base level so most of my previous work is out the window :) I still prefer an RR over a bus but truth is CAN must be very reliable given it's used in vehicles so I'll get over that.

So having decided on CAN, and given that the SAM3X has two CAN controllers, it was a natural thing to design a Due-like board to act as the "master" and that's where I am at now, distracted yet again from my network design :(


I just finished reading your "http://www.ardweenet.com/". Very interesting!
It makes me question our perhaps too simple network topology:
- How do today's commercial wheelchairs do it? I think they use CAN, don't they? Do they use double CAN cabling? Do they even use ring topology?
(How to get two separate cables to the wheelchair armrest?) Can somebody answer the same questions for cars?

I like your board. You built on Arduino compatibility, you chose the same TSR_1-2450 6.5 - 36 V switching power supply, and the size is to 1/20 of an inch the same than the CANinterfacer. RS-485 and RS-422 was a serious option for us too. And as you, we chose finally more modern CAN, designed for cars, having close requirements.

Quote from your web:
Quote
NOTE: One particularly important function required right now is to cast a critical eye over the network software design. I have a fairly large document that describes this that I can send to any experienced programmers with an interest, even better if you have this sort of networking experience, IE RS-485 based control networks.NOTE: One particularly important function required right now is to cast a critical eye over the network software design. I have a fairly large document that describes this that I can send to any experienced programmers with an interest, even better if you have this sort of networking experience, IE RS-485 based control networks.

I'm interested. Looks like there is a lot to learn from in there. (From your forum I got the following dead link: http://busnet.robgray.com/documentation/docs/BUSnet-dg-protocol-v0-2.pdf )

Thank you very much for the nicely done documentation on your website,
   Bernt


bernt


Hi Lenny,

How are you handling power for the (multi-node) system?

There is that TSR 1-2450 switching DC/DC converter on board (6.5 - 36 V input, same as used by Rob on his board). There is no fuse on board, the converter can handle shorts.

Power for digital parts comes typically from the bus cabling, but you can connect elsewhere if needed. If any power electronics is associated to a node, it gets separate fat supply cabling and over-current protection.

On the small single handed sail boats (now preparing for Miniji and Access dinghies), we will have a CAN concentrator box with star cabling. A small sealed lead-acid battery goes in the same waterproof container, with an extra valve to the outside.

Have a nice day,
   Bernt

craigcurtin


:), hmmm good point.

While it's true I retired 14 years ago at the tender age of 45 I have since learned to appreciate what people mean when they say "I don't know how I found the time to work".

I have 25 acres of land to clear of grass that in places is as tall as me. I'm cutting a series of walking tracks around my land which involves a lot of digging and track hardening with logs, I reckon I have 3-4ks of track to make and I've done maybe 400m. Yesterday I chain-sawed 3 fallen trees and moved the logs to my log pile.

I'm designing a house made of shipping containers plus landscaping around our existing containers, installing an eco loo in one container and building an office/workshop in the other plus a roof over the two and installing solar panels on top of one.

I'm doing some contract circuit and PCB design work for a Brazilian company and likewise for a Russian company, plus designing a Due-like CAN controller board and a monitoring-and-control network for personal use but also hopefully for release as an open source project. I also do a lot of nature photography and until recently wrote photography articles for magazines.

Then of course I have to see to my home brew (where it's entirely my responsibility to control production, distribution and consumption), maintain my 6x6 army truck/home and occasionally drive said truck to a new location when we get bored with the view in our current location.

After all that I need time to chill out, wind down, watch the wallabies feeding outside my window, listen to music and generally contemplate my life and/or navel.

I tell you, retirement ain't what it's cracked up to be.

______
Rob



You Crack Me UP !!!! :)

LROBBINS

Hi Rob,
Quote
iyiyi this thread must get the gold medal for long posts, here's another one.
Obviously, I don't mind that at all.
Quote
Not one bad reading, but don't add the bad one to the average, maybe count the bad ones and abort if you get more than 3, if it comes good assume it was noise and start averaging again but the bad readings did not get through.
Perhaps abort when bad instances >= Damping/10.  Damping will vary from user to user by an order of magnitude or more, depending on the chair and the user's fine motor characteristics.
Quote
As you say you do test later but if you were happily cruising at 500 and a wire breaks you will ramp up to 4500 in no time and the current code will think that is OK, it will reset to 2500 but that's still 5x faster than the user expected. By the time your daughter responds she could be down the stairs.
Yes, in part, but let me play devil's advocate for a moment.  Months ago I did some timing tests.  Outermost worst-case loop time was 5 msec, of which 4 msec was used for analog reads and averaging.  I should re-run that, as my analog reading algorithm then was from hunger, but assuming that timing remains roughly that,  what would be the effect of not issuing a safety stop if the average was badly off but not quite out of bounds.  If a real fault, it would not trigger the SafetyStop until the next loop, so at most 8 msec later.  Given the electrical and mechanical inertia of a chair this might not even be noticeable.  If it were a "glitch", if we don't abort the averaging the reading would recover in the next cycle so there would only be a 4 msec speed command change.  Now, what would happen if a Safety Stop were triggered because of that "glitch"?  We would send a Safety Stop at 4 msec or less.  What does that do?  In the Roboteq, and in all commercial WC controllers for that matter, it immediately, and very sharply, stops the chair AND requires a system re-set to recover.  Imagine if that happened while crossing an intersection.  Perhaps the best compromise would be to discard extreme reads during the averaging (but increment the counter anyway, while reducing the denominator for the division so that it can't get into an infinite loop), but NOT abort.  In any case, this suggests building a bit of testing into the code: calculate both a "noisy" average without discards and a "clean" average with discards and compare them.  Might want to actually use the "clean" one, but the comparison would give me a kind of figure-of-merit for the whole installation, electrical, electronic as well as the program.  A first pass at this could be made with my mockup - the trimmers and open wiring used to mimic a joystick are very noisy - but this seems to me like a good quality-control check to do for a real system.
Quote
That's a good thing for reliability, connectors are always a weak point.
Yes, connectors are a real headache.  I'll describe some personal experience, but commercial communications device makers have told me that they think connectors are almost always the reason for the dismal failure rates of communications found in a formal U. of Toronto study done a few years ago.  Rachi's first two communications computers had a portable on the back of her chair with the screen removed and remoted in front of her.  Over a period of years we went through quite a few connectors; as small as mini-SCSI, as large as Centronics, and never found one where contact pressure was maintained after a not many weeks of on-chair vibration, even with external pieces added to lock plug and jack together.  I ended up soldering lots of 32 ga. wires (but never tried a MIL spec Centronics).  Computer no. 3 was a Compaq laptop for which I made a hinge and HP supplied a cable so that I could turn the screen around, as one does now on convertible tablets.  No. 4 was in fact a HP-Compaq convertible (T4200, changed to 4400 after a MB failure), and her latest is a Fujitsu T5010.  USB connectors, which don't rely on bent strip contacts seem a lot more resistant to this, but the normal ones stick out and get banged into things, and the shell still has strip fingers that are easily bent.  First thing I do with a board that has a mini-B is to put some Goop (actually the industrial version called "E6000") around the shell.  (An alternative that I've sometimes used for vibration resistance is to put both male and female on flying leads and let the cable take the shake.) Dynamic has a nice compromise when using headers to sandwich boards - two of the pins are soldered through rather than being male-female connections; not too much work if you have to take them apart, but still much more secure than just plugging things together.  They use typical pin-header connectors or molex-type connectors for harnesses, but I add a drop of Goop to those too - easily removed if necessary, but can not come loose otherwise, and probably damps some vibration too.  BTW - do you know of any female headers that will accept square pins but are made with machined contacts like better SIP connectors?
Quote
For most of that time the topology has been async serial over a redundant ring because with an RR you can cut the cable and it has almost no affect on the network traffic.
I don't want a WC to do that!  If a mode goes down, for whatever reason, I want the chair to STOP.
Quote
The CAN frame has a CRC and it is very robust, I think you are wasting half your payload here. With the full 8 bytes you can represent all numeric data types up to float.
Unless I've read things badly, (1) the CRC field is used only to decide whether to send an ack and is not passed into the message buffer, (2) when the transmitting node gets an ack, it only knows that SOME node has gotten an intact message, not that the INTENDED node has gotten an intact message.  Hence, the addition of the complement check.  I think that the CANopen medical "flavor" also does this.  Yes, I might have to send more messages, but 16 bits are already more precise than an analog read, and certainly more precise than anyone can position a joystick, and so far I've needed just one message per action.

Bye for now,
Lenny


LROBBINS

Quote
There is that TSR 1-2450 switching DC/DC converter on board (6.5 - 36 V input, same as used by Rob on his board). There is no fuse on board, the converter can handle shorts.

My question was too vague.  I am using a Recom switching converter because it, or similar, let me use a single push button, few extra components, simple programming and no extra wiring to control power and/or sleep state for the entire network.  How are you controlling power across the net?  (Aside from this remaining question, I'm handling power in the same way you do.)
Quote
It makes me question our perhaps too simple network topology:
- How do today's commercial wheelchairs do it? I think they use CAN, don't they? Do they use double CAN cabling? Do they even use ring topology?

Yes, wheelchair systems use proprietary CAN protocols and (to some degree) hardware.  They are single cable, star or daisy chain configured (not restricted by controller mfr, but a matter of choice by the chair OEM - no ring)  with at most a single termination resistor and slow speeds.  Nodes that are not critical usually have just one connector so that they are necessarily at the end of a chain and their failure doesn't stop the net.  Nodes that are critical have two connectors, and if put at the end of an arm must rely on software to shut down the net if there's a fault; all functioning nodes repeatedly send a "here I am" message, and if one doesn't arrive the chair won't start or shuts down.
Ciao,
Lenny

bernt


My question was too vague.  I am using a Recom switching converter because it, or similar, let me use a single push button, few extra components, simple programming and no extra wiring to control power and/or sleep state for the entire network.  How are you controlling power across the net?  (Aside from this remaining question, I'm handling power in the same way you do.)

I still do not understand. What do you mean by "controlling"? ...cutting the main battery at distance? ...setting motor current? ...counting Amp-hours to get battery level? ...something else?

Quote

Nodes that are critical have two connectors, and if put at the end of an arm must rely on software to shut down the net if there's a fault; all functioning nodes repeatedly send a "here I am" message, and if one doesn't arrive the chair won't start or shuts down.

What is the second connector for ? ...a separate CAN-bus? ...or just an electrically parallel connector to easy daisy-chained cabling of the single bus ?

Thank you,
   Bernt

LROBBINS

Quote
What do you mean by "controlling"?

http://forum.arduino.cc/index.php?topic=168849.0
Quote
What is the second connector for ? ...a separate CAN-bus? ...or just an electrically parallel connector to easy daisy-chained cabling of the single bus ?

The last of these - easy cabling of the bus (whether star or daisy chain).  Dynamic also sells 4-receptacle jacks for when 2 aren't enough.  Rachi's chair actually has two 4-way blocks on the back of the seat frame, and they form the "center" of the star.  Most of the CAN modules move up and down with the seat, so there's less relative motion of cables connecting them there.  Only the bus cable to the power module (actually 2 in parallel to reduce voltage drop as they are the highest-current cables - Dynamic passes up to 12A on the bus cable) and to the Lighting/actuator module go between parts of the chair that move with respect to each other.
Ciao,
Lenny

bernt

Hi Lenny,

http://forum.arduino.cc/index.php?topic=168849.0

I didn't think a lot about that problem, but sometimes I think about doing it like with Raymarine Course Computers, that have provisions for a distant power-switch, requiring  extra cabling. They do not use a momentary switch/signal, the state of the power system follows the entry's state. This allows only for a single simple switch and is ok for yacht installations.


I wanted the pushbutton to do three things:

(1) turn power ON if it is OFF
(2) turn power OFF, with an orderly shutdown of the CAN network,  if it is ON and the system is AWAKE
(3) interrupt SLEEP if the system is sleeping


For the wheelchair CAN bus, I understand that you should be able to wake up other nodes over the CAN network too. For the moment, I do not understand how you achieve this. Isn't there a need for an extra wire in parallel with the CAN bus, a wake up wire? And if we need that wire anyway, why not having just one central power switch, close to the battery?

Bernt

LROBBINS

Hi Bernt,
Quote
For the wheelchair CAN bus, I understand that you should be able to wake up other nodes over the CAN network too. For the moment, I do not understand how you achieve this. Isn't there a need for an extra wire in parallel with the CAN bus, a wake up wire? And if we need that wire anyway, why not having just one central power switch, close to the battery?

No need for an extra wire.  For the nodes that just go to sleep rather than a real power off, it works like this. 
(1) A sleeping 2551 still receives and will awaken if a message arrives.  Activity on RxD is then used to awaken the MCU which then resets the 2151. 
(2) The Roboteq output (but not its MCU) is put to a "sort of" sleep by opening the high current contactor, and the Roboteq MCU is turned OFF (when the chair is turned OFF rather than sleeping) via a reed relay attached to AUX. 
(3) MASTER can go to sleep after a period of inactivity, but first zeros Roboteq output, sets the other nodes to sleep, and puts its 2151 and 2551 to sleep. 
(4) The push button interrupt awakens MASTER and the whole process is reversed. 
(5) To turn the system off rather than just letting it go to sleep, the push button going low tells MASTER to (1) zero Roboteq output, (2) tell AUX to open the relay that powers the Roboteq MCU, (3) put non-MASTER nodes to sleep, (4) pulls down the pin connected to the DC-DC converter control pin. 

No extra wires are needed, and the one-button control is within reach of the driver, not down near the battery.

Actual implementation is a bit more complex: everything has to be done in the correct order, some messages have to be sent one-shot to prevent blocking a 2151 because of absence of a returned ack, the 2151 has to be reinitialized at various points, after interrupt 0 goes low we have to wait for the pin to go high again to avoid immediately going back to sleep, messages sitting in buffers have to be dumped etc.  Actually, unread messages won't prevent the 2151 from going to sleep, but I have LED's on the Rx0BF and Rx1BF pins as a diagnostic help, and one or both might stay lit wasting power if the buffers aren't cleared - in the 2151 the only way to do that is to read the buffers until they're empty.

Ciao,
Lenny

bernt


(5) To turn the system off rather than just letting it go to sleep, the push button going low tells MASTER to (1) zero Roboteq output, (2) tell AUX to open the relay that powers the Roboteq MCU, (3) put non-MASTER nodes to sleep, (4) pulls down the pin connected to the DC-DC converter control pin. 

... and how can the driver switch on again his wheelchair?

   Bernt

LROBBINS

Quote
... and how can the driver switch on again his wheelchair?
Ha, ha - I did leave that part out didn't I.
Pressing the button pulls the converter's control pin high and charges a capacitor.  Output from the converter powers MASTER and the RC keeps it powered long enough for Setup () to pull the pin connected to the converter's control pin HIGH.  If you look at the circuit, you'll see a diode or two, needed to OR some things and protect from screwed-up wiring, and a transistor to invert button HIGH to interrupt pin LOW for waking from sleep.
Ciao,
Lenny

Graynomad

Quote
How do today's commercial wheelchairs do it? I think they use CAN, don't they? Do they use double CAN cabling? Do they even use ring topology?

I can't imagine they do. If CAN is reliable enough for cars it's reliable enough for wheel chairs and I've never seen a dual CAN topology.

Quote
I'm interested. Looks like there is a lot to learn from in there.

BUSnet is old stuff, I forgot about existing pages with links. I'll email you a file based on the latest version.

Quote
Perhaps abort when bad instances >= Damping/10.

My feeling is that such error detection is absolute, not relative to any other parameter. 10 errors is bad regardless of the Dampening setting.

You would never stop because of a single glitch. I can't quite follow the noisy and clean averages logic but I think it's best to split the problem into two parts, error detection and response.

To my mind the error detection is pretty simple, X number of bad readings in a row is wrong. You have to decide what X is but apart from that there are no grey areas.

It's the response that's the hard part...

Quote
Imagine if that happened while crossing an intersection.

Quote
If a mode goes down, for whatever reason, I want the chair to STOP.


The response is context sensitive, I have no answer to that I'm afraid.

I'm not suggesting you use a redundant ring here, but just because a network can continue after a cable fault doesn't mean it should, the event is detectable and the response is up to the programmers. But a cut cable need not be fatal, you flag the error and call the maintenance man, meanwhile the factory carries on making money. That's the beauty of a RR, a bus will lose some or maybe all the nodes depending on the location of the fault.

Quote
(1) the CRC field is used only to decide whether to send an ack and is not passed into the message buffer,

I would assume so.

Quote
(2) when the transmitting node gets an ack, it only knows that SOME node has gotten an intact message, not that the INTENDED node has gotten an intact message.

True.

CAN does not handle (2) as has been stated above. That's where a higher-level protocol comes in. As for (1), if the frame passes the CRC test you don't need the CRC value, you trust that the controller knows it's stuff and you get the data. I'm not sure what the statistical chances of an undetected error are in a CAN frame but they must be extremely low.

That said if you only need 4 bytes or less the inverted image will add another level of protection.

_____
Rob










Rob Gray aka the GRAYnomad www.robgray.com

Go Up