MD_Midi activating Neopixels

Hi, first of all, thank you @marco_c for this amazing library. It has helped me immensly.

I have successfully got the example files working, and playing midi files from and SD card through my midi shield (on an Uno) to my keyboard. I am now trying to activate neopixels corrosponding to the keys on the keyboard to make a 'piano tutor' kind of thing. As you can see in the code below, I am storing the 3 values from the array in 3 variables - eventID, pitch and volume. I think I have understood that correctly - when I put the code in the debug state, these 3 data variables are spouted to the serial monitor. I have also read your blog post on this.

From your blog, I see that EventID 0x80 is note on and 0x90 is note off, but there are lots of other control commands coming through from each midi file I play.

The code is (sort of) working. It lights the neopixels for each note, but they only stay on momentarily. I guess thats where the other control commands come in, but I'm not sure how to use them in code. There is also lots of 'noise', with pixels lighting that aren't a midi note being sent - again, I guess control commands and whatnot. Obviously, I don't want to use delays and even millis gives a noticable difference in the music.

Anyway, here's the part of the code that I modified from MD_Midifile_play in your examples. Be gentle, I'm fairly new to Arduino coding.

#if USE_MIDI

  if ((pev->data[0] >= 0x80) && (pev->data[0] <= 0xe0))

  {

uint8_t EventID =  pev->data[0];  //store 3 values from data array into seperate variables
uint8_t pitch =    pev->data[1];
uint8_t velocity = pev->data[2];

if (EventID = 0x90) //0x90 is note on?
{  

 pixels.setPixelColor(127 - pitch, pixels.Color(0, 255, 0));  //set pixel that corrosponds to pitch to red
 pixels.show(); 
}

if (EventID = 0x80) //0x80 is note off?
 
{  
 
 pixels.setPixelColor(127 - pitch, pixels.Color(0, 0, 0)); //set pixel that corrosponds to pitch to off
 pixels.show(); 
 
}

    Serial.write(pev->data[0] | pev->channel);
    Serial.write(&pev->data[1], pev->size-1);
  }
  else
    Serial.write(pev->data, pev->size);

#endif
  DEBUG("\n", millis());
  DEBUG("\tM T", pev->track);
  DEBUG(":  Ch ", pev->channel+1);
  DEBUGS(" Data");
  for (uint8_t i=0; i<pev->size; i++)
    DEBUGX(" ", pev->data[i]);
}

Thank you again!

Thank me after you try using == where you now use =.

if (EventID == 0x90) //0x90 is note on

== is a test for equality, = is for assignment.

a7

Thank you!!!!!!

1 Like

I also note that you are not handling the case of NOTE ON on with a velocity of 0 (which is actually interpreted as a a NOTE OFF command).

If you look at the midiCallback function in the library's PlayIO example there is a case statement with all the possible things you will get from the MIDI event being fired. You can just copy it and write your own playNote() function and it will work.

Thank you. I'll did look at using that example.

One more question - do you think it will be possible to have, say, 5 pixels 'cascading' down to a key, reaching the last one when the note is played? The Arduino might be handling 8 midi events, and 40 pixel animations at once? Maybe I could have the pixel animations handled by a separate Uno, controlled by the master that is reading the midi file from the SD card? Still, not sure how to delay the midi event until the pixels have completed their cascade. Maybe read the midi event, start the animation, move the midi events from the SD card into a buffer, and play the note when the animation has finished? Thank you again.

Running eight five pixel animations at once is within the capabilities of the microprocessor.

You'll have to get a bit more specific or I need to read your description again.

You'll need to think about what happens when two or more of the eight are trying to control a single pixel.

If you mean the key initiates something, but the note even doesn't get sent along until it's animation finishes I rj k that is easy enough but it just doesn't sound like much fun to play.

I'll read it again when I am not moving. And at a bigger window.

a7

Thank you. I actually got an error saying my Uno doesn't have enough storage when adding another row of pixels. I've plugged in my R4 and uploaded the code. The pixels are still animating, but no sound from the midi keyboard?

Now it will be necessary to post your code, use the IDE Autoformat tool if it isn't already pretty, and the <CODE/> button in the message composition window.

Say more if need be about what it is supposed to do and isn't doing, or is doing that it should not.

a7

Will do first thing tomorrow. I've read that the minima and R4 have issues with the midi libraries, so it seems it's not going to be as easy as unplugging the midi shield and putting it on top of another Arduino board.

In the meantime, do you have any throughts on how I can get the pixels cascading towards the keys? I think my main problem is going to be timing. Currently the pixel comes on when a midi key press is sent, and turns off when the note is released. I'll have to light the string of pixels one after the other before the note is played.

The big problem I've been having is timing. Sometimes, upto 8 notes are being played at once, and the code (above) takes the 3 variables from the array as per the midi instructions. I can't add any delays, as even a 50ms delay really messes up the music.

You don't need to play each note as soon as you get the event. If everything is delayed by the same amount, then there is no apparent delay to the listener if they are all played back in the same order with the same (delayed) timing. This should allow you time to animate the LEDs.

Hi,

@alto777 @marco_c I've made some progress today. I can't post code, as it's not on this computer - i'll email it to myself tomorrow and post it if required.

I was getting a lot of memory messages using the neopixel and fastled libraries, so I moved 3 neopixel strips to a seperate Uno and set up i2c between the midi shield/SD card shield uno. My aim is to light a note on each strip, one after the other with a short delay towards where the note should be played.

I've got the 3 bits of data from the midi/SD Uno transferring to the neopixel uno successfully, and I have copied the switch statement from your IO example to the pixel-uno to handle the lights. This is working, and I am lighting the notes on the 3 strips in time with the music.

However, I am still struggling to animate the LEDs. When I put a delay between the 3 strips, the music quickly goes out of sync, especially in a fast tune. I understand I can use each event to start the animation, and 'play' the actual midi sound after the animation has finished, but I'm struggling to get my head around how to do this. Maybe you can help without seeing the code? Perhaps give me apseudo explanation that I can implement in the meantime?

Thank you again.

Without seeing what your animation looks like, and maybe how you have coded it, any advices will be extremely general.

Anything like this can be broken down into a sequence of steps. Animators might call these frames, and aim to have the code publish a new frame every 20 or 30 milliseconds.

The code to do the animation should be designed so that calling it makes ann animation move one step along its storyline.

The loop() is arranged such that when an animation is playing, the animation function gets called regularly, in this vague description meaning at the frame rate.

The animation function keeps track of where it is so it can know what to do next. Next time. Next frame.

If you have multiple animations playing out at once, each animation will need its own set of variables that describe what it is up to and how far along it is.

If the loop function is non-blocking, you can be doing several things "at once", so by appearances the animations are smooth, and other stuff might be going on as well, although here it sounds like you just waiting on an animation to finish so you can send the note.

It will help us help you if you post what you got so far.

a7

@marco_c @alto777

Here's the code. I've divided it into the function from the 'sender' (which has the midi shield and SD card), and the 'receiver' that is connected to the 3 Neopixel strips. I'll tidy up the 'sender' code when I have it working. I understand I only need the switch cases on the receiver Uno. All the sender needs to do is take the midi file from the SD card, play it, and send the events through i2c to the receiver.

My questions are:

  • How can I activate the 3 neopixel 'notes' with a short delay between them before the note is played (physically heard) on the midi output? You can see I commented out the 50ms delay, as even this was messing up the tunes.
  • You can see on the receiver code that I have added an if statement to the 'note on' case to deal with a note on with 0 velocity. Is there a better way of doing this?
  • This is a general coding theory question regarding calling functions - I modified the receiver code from an i2c tutorial I found online. In the loop, I can't see the receiveEvent function being called. I only see it called in the setup. How is the code constantly receiving information from the i2c bus when it isn't continually being called in the main loop? (I told you I wans't a strong coder!)

OK, that's it. Thank you again for your continued support.


void playNote(uint8_t note, bool state)
{
  if (note > 128) return;
 
}

void midiCallback(midi_event *pev){

  Wire.beginTransmission(4); // transmit to device #4
  Wire.write(pev->data[0]);  //sending 0x12 to Slave
  Wire.write(pev->data[1]); //sending 0x45 to Slave
  Wire.write( pev->data[2]);  //sending 0x12 to Slave
  Wire.endTransmission();

  if ((pev->data[0] >= 0x80) && (pev->data[0] <= 0xe0))

  {
   Serial.write(pev->data[0] | pev->channel);
   Serial.write(&pev->data[1], pev->size-1);
 }
  else
    Serial.write(pev->data, pev->size);

uint8_t EventID =  pev->data[0];
uint8_t pitch =    pev->data[1];
uint8_t velocity = pev->data[2];

switch (pev->data[0])
  {
  case NOTE_OFF:    // [1]=note no, [2]=velocity
    DEBUGS(" NOTE_OFF");
    playNote(pev->data[1], SILENT);
    break;

  case NOTE_ON:     // [1]=note_no, [2]=velocity
    DEBUGS(" NOTE_ON");
    // Note ON with velocity 0 is the same as off
    playNote(pev->data[1], (pev->data[2] == 0) ? SILENT : ACTIVE);
    break;

  case POLY_KEY:    // [1]=key no, [2]=pressure
    DEBUGS(" POLY_KEY");
    break;

  case PROG_CHANGE: // [1]=program no
    DEBUGS(" PROG_CHANGE");
    break;

  case CHAN_PRESS:  // [1]=pressure value
    DEBUGS(" CHAN_PRESS");
    break;

  case PITCH_BEND:  // [1]=MSB, [2]=LSB
    DEBUGS(" PITCH_BLEND");
    break;

  case CTL_CHANGE:  // [1]=controller no, [2]=controller value
  {
    DEBUGS(" CTL_CHANGE");
    switch (pev->data[1])
    {
    default:              // non reserved controller
      break;

    case CH_RESET_ALL:    // no data
      DEBUGS(" CH_RESET_ALL");
      break;

    case CH_LOCAL_CTL:    // data[2]=0 off, data[1]=127 on
      DEBUGS(" CH_LOCAL_CTL");
      break;

    case CH_ALL_NOTE_OFF: // no data
      DEBUGS(" CH_ALL_NOTE_OFF");
    //  for (uint8_t i = 0; i < ARRAY_SIZE(pinIO); i++)
     // {
    //    uint8_t pin = pgm_read_byte(pinIO + i);
    //    digitalWrite(pin, SILENT);
    //  }
      break;

    case CH_OMNI_OFF:     // no data
      DEBUGS(" CH_OMNI_OFF");
      break;

    case CH_OMNI_ON:      // no data
      DEBUGS(" CH_OMNI_ON");
      break;

    case CH_MONO_ON:      // data[2]=0 for all, otherwise actual qty
      DEBUGS(" CH_MONO_ON");
      break;

    case CH_POLY_ON:      // no data
      DEBUGS(" CH_POLY_ON");
      break;
    }
  }
  break;
  }
}

and the reveiver...

#include <Adafruit_NeoPixel.h>
#include <Wire.h>

volatile byte myData[3];
volatile bool flag = false;

Adafruit_NeoPixel pixel1(101, 5, NEO_GRB + NEO_KHZ800);
Adafruit_NeoPixel pixel2(101, 4, NEO_GRB + NEO_KHZ800);
Adafruit_NeoPixel pixel3(101, 3, NEO_GRB + NEO_KHZ800);

// Define constants for MIDI channel voice message IDs
const uint8_t NOTE_OFF = 0x80;    // note on
const uint8_t NOTE_ON = 0x90;     // note off. NOTE_ON with velocity 0 is same as NOTE_OFF
const uint8_t POLY_KEY = 0xa0;    // polyphonic key press
const uint8_t CTL_CHANGE = 0xb0;  // control change
const uint8_t PROG_CHANGE = 0xc0; // program change
const uint8_t CHAN_PRESS = 0xd0;  // channel pressure
const uint8_t PITCH_BEND = 0xe0;  // pitch bend

// Define constants for MIDI channel control special channel numbers
const uint8_t CH_RESET_ALL = 0x79;    // reset all controllers
const uint8_t CH_LOCAL_CTL = 0x7a;    // local control
const uint8_t CH_ALL_NOTE_OFF = 0x7b; // all notes off
const uint8_t CH_OMNI_OFF = 0x7c;     // omni mode off
const uint8_t CH_OMNI_ON = 0x7d;      // omni mode on 
const uint8_t CH_MONO_ON = 0x7e;      // mono mode on (Poly off)
const uint8_t CH_POLY_ON = 0x7f;      // poly mode on (Omni off)
const char fileName[] = "AIR.MID";
const uint8_t ACTIVE = HIGH;
const uint8_t SILENT = LOW;

#define DELAYVAL 500 // Time (in milliseconds) to pause between pixels

void setup() {
 
Wire.begin(4);                // join i2c bus with address #4
Wire.onReceive(receiveEvent); // register event

pixel1.begin(); // INITIALIZE NeoPixel strip object (REQUIRED)
pixel2.begin(); // INITIALIZE NeoPixel strip object (REQUIRED)
pixel3.begin(); // INITIALIZE NeoPixel strip object (REQUIRED)

pixel1.clear(); // Set all pixel colors to 'off'
pixel2.clear(); // Set all pixel colors to 'off'
pixel3.clear(); // Set all pixel colors to 'off'

pixel1.setBrightness(255); 
pixel2.setBrightness(255);  
pixel3.setBrightness(255); 


pixel1.show();   // Send the updated pixel colors to the hardware.
pixel2.show();   // Send the updated pixel colors to the hardware.
pixel3.show();   // Send the updated pixel colors to the hardware.
}

void loop() {

   if (flag == true)
  {
    Serial.println(myData[0], HEX); //shows: 12
    Serial.println(myData[1], HEX); //shows: 45
    Serial.println(myData[2], HEX); //shows: 45
    flag = false;
  }

uint8_t EventID =  myData[0];
uint8_t pitch =    myData[1];
uint8_t velocity = myData[2];



switch (myData[0])
  {
  case NOTE_OFF:    // [1]=note no, [2]=velocity
    pixel1.setPixelColor(127-pitch, pixel1.Color(0, 0, 0));
    pixel2.setPixelColor(127-pitch, pixel2.Color(0, 0, 0));
    pixel3.setPixelColor(127-pitch, pixel3.Color(0, 0, 0));

    pixel1.show();   // Send the updated pixel colors to the hardware.
    pixel2.show();   // Send the updated pixel colors to the hardware.
    pixel3.show();   // Send the updated pixel colors to the hardware.
    break;

  case NOTE_ON:     // [1]=note_no, [2]=velocity
    pixel1.setPixelColor(127-pitch, pixel1.Color(150,0 , 0));
    pixel2.setPixelColor(127-pitch, pixel2.Color(0, 0, 150));
    pixel3.setPixelColor(127-pitch, pixel3.Color(0, 150, 0));

    pixel1.show();   // Send the updated pixel colors to the hardware.
    //delay(50);
    pixel2.show();   // Send the updated pixel colors to the hardware.
    //delay(50);
    pixel3.show();   // Send the updated pixel colors to the hardware.

if (velocity == 0) //0x90 is note on
{  
    pixel1.setPixelColor(127-pitch, pixel1.Color(0, 0, 0));
    pixel2.setPixelColor(127-pitch, pixel2.Color(0, 0, 0));
    pixel3.setPixelColor(127-pitch, pixel3.Color(0, 0, 0));

    pixel1.show();   // Send the updated pixel colors to the hardware.
    pixel2.show();   // Send the updated pixel colors to the hardware.
    pixel3.show();   // Send the updated pixel colors to the hardware.
}

    break;

  case POLY_KEY:    // [1]=key no, [2]=pressure
    
    break;

  case PROG_CHANGE: // [1]=program no
    
    break;

  case CHAN_PRESS:  // [1]=pressure value
   
    break;

  case PITCH_BEND:  // [1]=MSB, [2]=LSB
    
    break;

  case CTL_CHANGE:  // [1]=controller no, [2]=controller value
  {
    
    
    {
    default:              // non reserved controller
      break;

    case CH_RESET_ALL:    // no data
     
      break;

    case CH_LOCAL_CTL:    // data[2]=0 off, data[1]=127 on
      
      break;

    case CH_ALL_NOTE_OFF: // no data
    
      break;

    case CH_OMNI_OFF:     // no data
    
      break;

    case CH_OMNI_ON:      // no data
      
      break;

    case CH_MONO_ON:      // data[2]=0 for all, otherwise actual qty
    
      break;

    case CH_POLY_ON:      // no data
     
      break;
    }
  }
  break;
  }

}

void receiveEvent(int howMany) //howMany = data bytes received from Amster = 2 here
{
  for (int i = 0; i < howMany; i++)
  {
    myData[i] = Wire.read();
  }
  flag = true;
}```

THX. It will be easier when you post complete sketches for all parts of this. I'm in the transport cafe, not conducive to reading code, neither is this stupid tablet. Later for that.

No, you can't. It looks like receiveEvent is registered with something external to your code (in a library or like that) as a callback function.

After which registration the i2c thing works in the background, and when it thinks it is a good idea, calls the function you registered. Sometimes there are more than one callback for different kinds of events. Seen in some button libraries for example, onPress, longPress and so forth.

So here, when the i2c subsystem recognizes whatever an event is, it uses the function you provided to handle it.

Typically your higher level code would notice some side effect that your callback produces after whatever work it can do - perhaps raising a flag that lets you know something is arrived and should maybe be considered.

It's a nice mechanism, but it can make just reading the code hard as one has to keep in mind that other things are happening than the line of code you may be squinting at just now.

a7

Right now your animation consists in lighting a pixel on each of three strips when you press the note, and unlighting them when you release.

Three strips of LEDs that each correspond to the entire keyboard let's say horizontally, and arranged on top of each other so the note-on-off effect plays out vertically.

And now to animate you use delay, which sucks. Or no delay, which also is unsatisfcatory.

If we assume a rhythm to this key press A, AB, ABC ... key release ABC AB A, where in those steps should the note command(s) be passed along to the instrument?

Anyway, this is reasonably straightforward to code, certainly for one note at a time and that would inform a generalized solution for N notes.

If you lock in your ideas for the animation, the code can be special-purposed and some shortcuts taken. Or you could craft a general "keystrokes make things happen on LEDs" and be able to try different such things.

This link is to a post on this I am too lazy to cut and paste. It will be hard until it is easy, then you will wonder about how you ever did without.

HTH

a7

Not quite... The midi file is read from the SD card on the Uno with the midi shield. It transmits these events to the Uno with the neo pixels, and lights the individual lights on the strands according to the notes. Meanwhile, the Uno with the midi shield plays the note. No input from the user on the keyboard at this point. I want the individual LEDs to cascade toward the corresponding key on the keyboard for the user to then play. Like this... https://youtu.be/TSoXBkF832I?si=UY2iMDVw0laUZD0C

I see. So you have a mini cascade of just three rows?

If I got the A, AB, ABC and reverse right, when does the note from the SD card get played? I guess as soon as it gets read, and the cascade A .. ABC is launched at the same moment.

And same for note off events which also come from the SD card file?

a7

At the moment, yes. But I want the not to be played (heard) after the cascade happens.

*note. So the animation leads the pianist to the key press.

Just let me summarize how this should work for my benefit.

  1. The MIDI file is read from the SD card using the MD_MIDIFile library. This passes each MIDI event back to the application through a callback.
  2. To process each MIDI event (typically note on/off) in the application callback, save the event in a queue with a time stamp of when it arrived. I woulod suggest that you convert the NOTE ON with ZERO velocity to a NOTE OFF before putting it in the queue.

To process the event queue you will need to have a Finite State Machine style function (non-blocking). Hereunder is what should happen in each state
a. If the time is right to play the next event (from the timestamp and time difference from start of last processed note). If NOTE_ON, move to state b. If NOTE_OFF, move to state e,
b. Light the correct LED and move to state c.
c. If the LED lit time has elapsed turn off the first LED, turn on the second LED and move to state d.
d. If the LED time has elapsed turn off the second LED, turn on the rhird LED and move to state a.
e. Turn off the third LED, move to state a.

Is this what you want to achieve?

I also think that you should try and get a subset of everything working on one CPU before splitting it up. This will make debugging it much easier and less confusing.