output pin mapping to shift register or similar

Good day... I am in a classic "two pins short" scenario with my current project. I am used to using shift registers for various things, and do not mind them, however this particular project is highly dependant on various libraries like OneWire and LEDControl. Is there a way to use a shift register or similar device in conjunction with a software library layer to effectively present more "Arduino pins" such that other libs can use them?

For instance, OneWire and LEDControl can all use any digital output pin. I already have an 74HC595 in this project, so imagine for a second that I am only using 4 bits of it, leaving me with 4 extras which i would rather use and free up some of my GPIOs for other things. I'd like to be able to do something like pseudo code:

// !!!!!!!! this is pseudo/discussion code!!! dont attempt to use since the library doesnt actually exist!!!

#include shifterToRegularPins.h

/* use BCD shift register on pins 5, 6, and 7 to 
   map to software layered pins, eight wide, 
   starting at 20 (winds up as 20-27);
*/
#define shifterLatch 5
#define shifterClock 6
#define shifterData 7
#define shifterWidth 8
#define shifterFirstSoftPin

PinShifter shifter1=PinShifter(shifterLatch, shifterClock ,shifterData, shifterWidth); 

// Then later on down in code
pinMode(20, OUTPUT); // naturally this fakepin can only be output
digitalWrite(20, HIGH); //turn on this pin

// use some libraries, which as long as they only use output modes, dont care that the pins are fake/software emulated/whatever
OneWire ds(21);
ds.begin;
LedControl lc=LedControl(22,23,24,25);

// use above stuff as normal

Naturally, this would be somewhat nasty on use of stack, may not work with some parts that are unforgiving on timing and so on, But for some applications, "speed to market" and freedom to choose parts is more important. In my case, it would be a time saver, since I don't get to work on electronics as much as i would like. My fallback is to change the 7-seg driver over to an i2c part, and use i2c parts for my two input buttons, but above idea sure did sound nice in my head. Perhaps my day job on general purpose computers taints me... you can always "solve" a problem by acquiring more software layers. Good ones really will save you from having to re-do the system, until you get so many that you are crushed under their weight of course!

Anyone?

http://www.arduino.cc/en/Tutorial/ShiftOut

That has nothing to do with the original poster's question.

To address the actual question, there doesn't seem to be a standard way to address bits on a Shift Register. You might look at this project: http://nootropicdesign.com/ez-expander/#library However, you probably have to go and re-write the libraries you are using to make it work. All that effort is probably higher than just changing hardware like you suggested.

Huh? People do this every day.

The original poster was asking for a way to use libraries like OneWire with a shift register. Please tell show where in the ShiftOut page this is explained.

the key phrase is

more “Arduino pins” such that other libs can use them

I will attempt to clarify, I am already using ShiftOut for some easy stuff like indicator LEDs and relays. The post is all about being able to use some sort of IO expansion, such that libraries written to use a standard pin number might do so transparently thru a software layer, which in turn manipulates the IO expander rather than a regular pin.

Its interesting enough to me that I may see if I can overload digitalWrite and maybe a few other methods. I’ll have to look in the libraries to see what functions they actually use. Naturally anything that does straight instructions will not benefit (i.e. PORTX << whatever;)

CMiYC’s link to EZ-expander is right on, except for the additional concept that libraries would be able to use it transparently.

Yes. I want to make it work with libraries, without modifying said libraries.

Ok, lets work by example here. I currently have the following code:

#include <OneWire.h>
#define oneWirePin 8

OneWire ds(8);  // oneWire bus is on arduino pin 8

// later... 
void doSomething() {
  ds.reset();  // drags my bus low for a while
  ds.select(addr);
  ds.write(0x44,1); // bit-bangs the bus a little bit
}

So i instantiate the OneWire library, and all it needs to know is the pin number which my 1-wire devices are attached to (more complete example at http://www.arduino.cc/playground/Learning/OneWire).

Later, when i want to access that bus, i call my own function doSomething() which in turn calls things inside the OneWire library (i.e. ds.reset). The library then manipulates the pin (8 in this case) to do its work (talk to a temp sensor).

Now imagine i hook my onewire bus to a shift register or similar… I will be unable to use it that way without modifying (perhaps heavily) the oneWire library.

I just realized that Onewire is a bad example here because it also uses the pin for input. But pretend it doesnt for the sake of argument. LEDControl works similarly (see my original post) but definitely only does output.

An arduino with more pins eventually has the same problem. GPIOs or shift registers are cheap, and work great for this sort of thing IF you aren't depending on other people's components (libraries).

The thing i am looking for is not magical at all... an ordinary shift register or GPIO chip, simply with software that is beyond my current skill to produce. I could get the same effect by rewriting (rather than overloading) digitalWrite() i suppose. at least that would be preferable to touching every library.

pseudo-code:

void digitalWrite(uint8_t pin, bool state) {
    if (pin > actualPins) {
        expander.digitalWrite(pin, state);  // based on ShiftOut or maybe ez-expander
    } else {
        original_digitalWrite(pin, state);
    }
}

With LedControl, looks from the source that it uses SPI functions, but uses digitalWrite() for the chipselect. so that works in this case to move just that one pin.

OneWire uses some macros, again this is where my skill level is lacking and was hoping to find help or someone that had already done it....

#define DIRECT_READ(base, mask)         (((*(base)) & (mask)) ? 1 : 0)
#define DIRECT_MODE_INPUT(base, mask)   ((*(base+1)) &= ~(mask))
#define DIRECT_MODE_OUTPUT(base, mask)  ((*(base+1)) |= (mask))
#define DIRECT_WRITE_LOW(base, mask)    ((*(base+2)) &= ~(mask))
#define DIRECT_WRITE_HIGH(base, mask)   ((*(base+2)) |= (mask))

I guess I'll work on it if time permits, and in the mean time work around and hope that someone a little more open minded will read the thread.

The thing i am looking for is not magical at all... an ordinary shift register or GPIO chip, simply with software that is beyond my current skill to produce

Or indeed given the constraints:-

I want to make it work with libraries, without modifying said libraries.

beyond anybodies current skills. Or indeed future skills with out the evocation of magic (which you have also ruled out). In short no you can't redirect pins from a library to your own driver without changing the library or rewriting the whole of the foundations of the system.

I agree with the OP that being able to address pins on a shift register as if they were standard Arduino pins would be helpful for a lot of folks.

If the function in the library uses only calls to Digital.write or .read, I imagine an abstraction could be made but it would be pretty slow. Most of the better libraries use direct port I/O and have a way to map physical pins to their respective ports, so this idea becomes useless again.

If you’re two pins short but those pins need to do OneWire or LCD control or etc. I think you need to dedicate hardware to them on the chip. Perhaps you can re-arrange some of the other hardware to free up the pins you need?

I have a project here with 16 A/D inputs and 16 relay outputs, not to mention programmable gain using 48 analog switches and Ethernet, using just an ATMEGA328. Heck, I’m not even using the analog input pins :slight_smile:

Trying to approach limited hardware like the Arduino with the software programmer’s mindset can be a bad idea - the software developer I work with has many many more tools and capabilities that I do with my Arduino, but I can work within its limitations because I understand more or less where everything goes in the hardware and firmware. Rather than pile on more software I can usually find a hardware solution to fix the problem the right way.

Good luck!

I guess you guys misinterpretted me a little bit. I can solve my "two pins short" problem a number of ways, and this is one that i thought about. Not knowing if it had been done before, dumb, or so on, the post was more of an intellectual exercise than anything.

Most of the better libraries use direct port I/O

Looked, and yes they do. I'm a little disappointed in LiquidCrystal in this regard... i think I'll add to it and do this, because i sure am learning a heck of a lot right now.

Trying to approach limited hardware like the Arduino with the software programmer's mindset can be a bad idea

Point very much taken.

I suspect he doesn't really understand what is going on beneath the UI.

Quite the contrary... I just don't enjoy dealing with it too much. The arduino core source code I can read, just not bright enough to modify or come up with much on my own. I guess thats why i wasn't a good developer. Pointer arithmetic, assembly, bit math and all that I can do, and didn't even get fired for doing it professionally. I just don't have a natural knack for it that I suspect alot of you folks do. So i took middle of the road and did lots of sysadmin and solving business problems, and tried to forget assembly. Years on, thats getting old too and I needed a hobby other than booze. Recently I found Arduino/AVR, and have been having a ball because i can do nice projects at low cost and without lots of brain damage of reinventing the wheel all the time just to make it work. Thats why i wanted my sacred cow of "works with libraries", and you guys bring up some good points of why it isn't practical.

End of ramble, thanks for the lively discussion... Rich, lighten up a little man! Its a hobby, right?

So for anyone who comes across this thinking the same things i did... I tried to work this out in software with the shift register. The crummy thing is that because it is shifting, even if you shift the bits as fast as you can, you still wind up having a pin that is supposed to be HIGH that during the shift goes LOW for long enough to screw up other parts. In my case, a MAX7221 chipselect line goes low, and he loads whatever random bits he had seen go by earlier. Anything i tried to compensate for this would screw up rise/fall times when i actually do use the pin or add unacceptable delays that screw up other parts of my code.

I did make LiquidCrystal workon the shift register this way for its address lines, but the clock has to be left on a regular pin, and it was painfully slow.

So now i have a GPIO part on order that interfaces over i2c. That should be a fun experiment as well, same sort of "can i do this". It has been very educational for me, but if you are actually trying to get something done, rearrange the hardware, or optimize without the use of libraries. Trying to have my cake and eat it too would have been very painful if there had been any deadlines involved here.

Happy hacking!

you still wind up having a pin that is supposed to be HIGH that during the shift goes LOW for long enough to screw up other parts.

Why? If you're using SRs with latched outputs (such as the 74xx595) you won't get any rippling as the shift occurs. Apart from speed of updates there should be no practical difference between using SRs and parallel IO.


Rob

If you're using SRs with latched outputs (such as the 74xx595) you won't get any rippling as the shift occurs.

Quite right... not sure why I didn't think of that. I recreated my test rig (previous tests on breadboards) and was unable to duplicate my previous result of ~60uS dip on an output pin of the 74HC595. Perhaps i had SH_CP and ST_CP shorted together or something last time. Backward would not have worked, so I do not know how i came up with that result.

So I'll try again for posterity with LiquidCrystal.

I was able to rewrite my own functions of LEDControl to use spitransfer() instead of the bit bang stuff, which lets me put the LED driver part on the same SPI bus as an SD card. So that effectively solves the original problem that got me started on this thread in the first place.

Perhaps you don't want to know this, but of course I2C is serial (just like ShiftOut)

Yes I do know that... I guess the bar on novices is lower than i thought around here. Serial or parallelness wasn't my problem, it was the (now hard to explain since i cannot duplicate it) output dip when using shiftOut. The point of using an i2c GPIO is that I know it will play nicely with other i2c parts that i have nearby on my prototype. I really don't want anything hanging off my SPI pins other than the SD card if I can help it, so the GPIO on i2c will wind up driving the LED driver, LCD, and probably a set of dip switches for configuration.

The posterity test was successful… this does work, and can suit some needs, yes of course its slow. I will likely use it occasionally… its certainly cheaper than GPIO parts for occasions that its needed. I can either stuff the new function in a given library, and copy that lib and change calls to digitalWrite, or i can use an #ifdef in wiring.h to only grab my new function when its needed.

Just for the heck of it, i made this simple program that anyone can use, and tested just exactly how slow it is to “fake” pins in this manner. So the loop() just writes the pin high and low, and loops again. Uncomment the test you are interested in to try it.

In short, its wicked slow, here are the timings for an entire cycle of loop() (measured with my scope, not software or anything) with consistent units.

digitalWriteShifted on a “fake” pin = 245.0us
digitalWriteShifted on a regular pin = 9.0us
digitalWrite normal = 7.8us
direct port assignment = 0.812us
direct port assignment inside while() instead of loop() = 0.250us

heres the code:

boolean foo=1;  //wtf why cant this compiler deal with a #define being first statement?

// measure timing of "fake" pins example using a shift register
// note this was done on an atmega328 @16Mhz, standard arduino clone "BBB" from http://www.moderndevice.com

//ST_CP of 74HC595
#define latchPin 8
//SH_CP of 74HC595
#define clockPin 7
//DS of 74HC595
#define dataPin 9

byte shifterState = 0;


void setupShifter() {
  shifterState = 0;
  //set pins to output so you can control the shift register
  pinMode(latchPin, OUTPUT);
  pinMode(clockPin, OUTPUT);
  pinMode(dataPin, OUTPUT);
  //set em all low to begin with  
  for (int i=20; i<28; i++) {
        digitalWriteShifted(i, LOW);
  }
  delay(100);
  //Serial.println("shifter setup done");
}



void shiftIt(byte x) {
    digitalWrite(latchPin, LOW);
    shiftOut(dataPin, clockPin, MSBFIRST, x);  
    digitalWrite(latchPin, HIGH);
}



void digitalWriteShifted(uint8_t pin, uint8_t pinstate) {
  /* 
    Serial.print("changing pin  ");
    Serial.print(pin, DEC);
    Serial.print(" to state ");
    Serial.println(pinstate, HEX);
  */ 
    
  if ( pin < 20 ) {
    digitalWrite(pin, pinstate);
  } else {
    // Q0 on shift register = fake pin 20,
    // so subtract 20 to get register bit position
    pin = pin - 20;
    
    if (pinstate) { //HIGH
      shifterState |= (1 << pin);
    } else { //LOW
      shifterState &= ~(1 << pin);
    }
    shiftIt(shifterState);
  } 
}






void setup() {
  Serial.begin(57600);
  setupShifter();
}


void loop() {
  /**********************
    // ~245us, 50% duty cycle
    digitalWriteShifted(20,HIGH);
    digitalWriteShifted(20,LOW); 
  */
  
  /*
    // ~9us, 47% duty cycle
    digitalWriteShifted(13,HIGH);
    digitalWriteShifted(13,LOW); 
  */
  
  /**********************
    // ~7.8us, 46% duty cycle
    digitalWrite(13, HIGH);
    digitalWrite(13, LOW);
  */
  
  /**********************
    //812ns total, 8% duty cycle
    PORTB = B00100000;
    // ~62ns spent high
    // then...
    PORTB = B00000000;
    // ~750ns spent low reexecuting loop()
  */
  
  /**********************/
    //250ns total, 25% duty cycle
    while(1) {
      
      PORTB = B00100000;
      // ~62ns spent high
      
      //then
      PORTB = B00000000;
      //~188ns spent low, evaluating the while()
    }
/*   */
}

Hmm, pretty slow eh? But presumably not much worse than using shiftout from the main prog and does provide a level of abstraction.

What if you do your own version of shiftout within digitalWriteShifted (ie don't call shiftit(), just move the bits) using direct port manipulation?


Rob

Hmm… tried, but i think my bitmath-fu is too weak. With the following code i get my output pulses on the output of the shift register every 6000us. I must have screwed up something with the bits, but i cannot find it… start at highlighted part.

int i;


void digitalWriteShifted(uint8_t pin, uint8_t pinstate) {
  /* 
  Serial.print("changing pin  ");
  Serial.print(pin, DEC);
  Serial.print(" to state ");
  Serial.println(pinstate, HEX);
  */ 
    
  if ( pin < 20 ) {
    digitalWrite(pin, pinstate);
  } else {
    // Q0 on shift register = fake pin 20,
    // so subtract 20 to get register bit position
    pin = pin - 20;
    
    if (pinstate) { //HIGH
      shifterState |= (1 << pin);
    } else { //LOW
      shifterState &= ~(1 << pin);
    }
    //shiftIt(shifterState);

[glow]    // replace shiftIt with direct port manipulation to get timings.
    // assumptions:  
    //     clock is D7 (PORTD bit 7)
    //     latch is D8 (PORTB bit 0)
    //      data is D9 (PORTB bit 1)
    //
    // from here on out, statements commented out are replaced by the following statement(s)    [/glow]
    
    //digitalWrite(latchPin, LOW); //instead...
    PORTB &= ~(1 << 0);
    
    //shiftOut(dataPin, clockPin, MSBFIRST, x);  
    
        for (i = 0; i < 8; i++)  {
                //if (bitOrder == LSBFIRST)
                //        digitalWrite(dataPin, !!(val & (1 << i)));
                //else
                //  digitalWrite(dataPin, !!(shifterState & (1 << (7 - i))));
                
                // assume always MSBFIRST for speed sake)
                if (!!(shifterState & (1 << (7 - i))))
                   PORTB |= (1 << 1);
                else
                   PORTB &= ~(1 << 1);

                //digitalWrite(clockPin, HIGH);
                PORTD |= (1 << 7);

                delayMicroseconds(1);
                //digitalWrite(clockPin, LOW);
                PORTD &= ~(1 << 7);

        }

    //digitalWrite(latchPin, HIGH);
    PORTB |= (1 << 0);

  } 
}

I did try giving the register more time with delayMicroseconds of 20, that still did not work. In particular, my latch pin has a saw-tooth wave shape (with flat bottoms that are quite wide), and there seems to be lots of noise on clock and data pins. I know it has to do with code, since last nights code everything is still nice square waves.

boggled, time to sleep.

amazing what sleep will do for you… wierd waveform on my latch pin was caused by a small cap between it and ground, put there after reading shiftOut() tutorial. in my deranged state, i believed that it was not happening with “old” code. New timings, with clearly sub-par hard coded pin assignments above. Not amazing, but servicable depending upon the workload:

digitalWriteShifted on a “fake” pin = 245.0us
digitalWriteShifted on a “fake” pin = 77.0us
digitalWriteShifted on a regular pin = 9.0us
digitalWrite normal = 7.8us
direct port assignment = 0.812us
direct port assignment inside while() instead of loop() = 0.250us

Yet more posterity. Code below results in new metric:

digitalWriteGPIO on a “fake” pin on a MAX7312 i2c GPIO chip = 665.0us

digitalWriteGPIO on a regular pin = 9.0us (same as Shifted on regular pin)
digitalWriteShifted on a “fake” pin using ShiftOut = 245.0us
digitalWriteShifted on a “fake” pin using direct port manipulation = 77.0us
digitalWriteShifted on a regular pin = 9.0us
digitalWrite normal = 7.8us
direct port assignment = 0.812us
direct port assignment inside while() instead of loop() = 0.250us

// test how fast we can toggle an output pin on a MAX7312 GPIO chip

#include <Wire.h>

#define MAX7312_ADDR 0x20
byte shifterState = 0;

void digitalWriteGPIO(uint8_t pin, uint8_t pinstate) {
  /* 
  Serial.print("changing pin  ");
  Serial.print(pin, DEC);
  Serial.print(" to state ");
  Serial.println(pinstate, HEX);
  */ 
    
  if ( pin < 20 ) {
    digitalWrite(pin, pinstate);
  } else {
    // Q0 on shift register = fake pin 20,
    // so subtract 20 to get register bit position
    pin = pin - 20;
    
    if (pinstate) { //HIGH
      shifterState |= (1 << pin);
    } else { //LOW
      shifterState &= ~(1 << pin);
    }

    Wire.beginTransmission(MAX7312_ADDR);
    Wire.send(0x02);
    Wire.send(shifterState);
    Wire.endTransmission();

  }
}



void setup () {
  //Serial.begin(57600);

  Wire.begin();
  Wire.beginTransmission(MAX7312_ADDR);
  Wire.send(0x06);
  Wire.send(0x00); // set port1 (0-7) to all outputs
  //Wire.send(0xFF); // set port2 (8-15) to all inputs 
  Wire.endTransmission();
  
}

void loop () {
  
  // takes ~660us for a full cycle, 50% duty cycle
  digitalWriteGPIO(20,HIGH);
  digitalWriteGPIO(20,LOW); 

}