[Advice needed] midi controller - EEPROM presets structure

Using Arduino Project Guidance
#1

Hi all,

I'm reviving an old project and I'd like to get some input on how to approach to a few things to plan ahead.

Some background: I'm building (or trying to) a small MIDI stompox to control a couple guitar FX units. I'm working with an Arduino UNO and so far I've been able to connect 5 footswitches, implemented click, double-click, combination and hold functions for these buttons, send/receive MIDI messages (SysEx, CC, program changes) and display the data on a 16x2 LCD.

Now I'd like to go a little further and implement preset handling. Here's where I'd need a hand: how would you approach to it, what would be the better way to handle memory, data structure, etc?

What I want to accomplish is the following: I'd like to be able to set a MIDI message/function for each button, per preset. In other words, each preset would have 5 messages/functions (or a couple more if I can consider combinations) triggered by 5 buttons. Each message consists on a few bytes, containing the midi channel, the message type (program change, CC) and a couple values. I was also thinking to add some special messages, where I could set, for example, 2 CC values and make some kind of curve transition to affect an FX parameter like delay time, etc. Ideally, it'd be nice to have 128 presets available, though it's not that important and I guess it depends on the available memory.

How should I structure all this, considering that I'd like to be able to edit those presets with an encoder and store them (on EEPROM or an SD card maybe)?

EDIT: UPDATED QUESTION USING EEPROM BELOW, ON POST REPLY #9

Any idea, suggestion, input will be greatly appreciated.

Thanks in advance,
Jon

#2

I would go with EEPROM storage and store the first 128 CC values in the first 128 bytes of it. Then for the rest of the memory you can store any system exclusive messages. Editing these is tricky because they can be variable length. If you need variable length messages then store them as a linked list. Then to edit them remove the origional message by overwriting it with the rest of the list and adding the new message to the end of the list.

#3

Hi Mike, Thanks for replying.

Grumpy_Mike:
I would go with EEPROM storage and store the first 128 CC values in the first 128 bytes of it. Then for the rest of the memory you can store any system exclusive messages.

I didn't quite get this, what do you mean when you say 'the rest of the memory'?

Just to clarify, here's breakdown of what I was thinking the structure hierarchy could be:

PRESETS
 │
 └─ Preset (0-127)
        │
        ├─ button 1
        │    │
        │    └─ message
        │           │
        │           ├─ type (PC, CC, special CC, etc)
        │           │
        │           ├─ MIDI channel (0-15)
        │           │
        │           ├─ value 1 
        │           │
        │           ├─ value 2 
        │           │
        │           └─ value 3 
        │  
        │ 
        ├─ button 2
        │
        ├─ button 3
        │
        ├─ button 4
        │
        └─ button 5

So basically each preset stores 5 button assignments, and each button stores a message. Does this makes sense?

If I think about the above structure I imagine something like this:

struct message{
  byte type;
  byte channel;
  byte value1;
  byte value2;
  byte value3;
};

struct preset{
  message btn1;
  message btn2;
  message btn3;
  message btn4;
  message btn5;
};

preset presets[64];

But I suspect this will not be very efficient. By the way, by having 64 presets, global variables use 80% of dynamic memory.

#4

what do you mean when you say 'the rest of the memory'?

The EEPROM is 1K, to store CC values you only need 128 bytes leaving 896 bytes free.

However it looks like you need 128 CC messages for each MIDI channel? Is this correct?

So basically each preset stores 5 button assignments, and each button stores a message. Does this makes sense?

Sorry no it is not very clear.

But I suspect this will not be very efficient.

You are right it is not very efficient. What are the three bytes value about? For a CC preset send the address of the parameter in the EEPROM as the second byte of the CC message and the contents of the EEPROM address as the third byte.

By the way, by having 64 presets, global variables use 80% of dynamic memory.

That is why you do not have these values in SRAM ( dynamic memory ). You make up the message in response to the buttons.

If you are struggling with the concepts of cramming it all in to 1K then you might want to look at having some I2C EEPROM storage.

#5

Grumpy_Mike:
The EEPROM is 1K, to store CC values you only need 128 bytes leaving 896 bytes free.

However it looks like you need 128 CC messages for each MIDI channel? Is this correct?

I need 5 messages per preset, since I'd be using 5 buttons. Ideally I'd like to have 128 presets.
The midi channel is stored as part of the message. Since I have 2 or 3 FX units that would receive MIDI, each of them would be on a different channel.

Grumpy_Mike:
Sorry no it is not very clear.

Imagine the following situation:

I have a preset I created to play a song. Let's call it preset 1.
In this preset, each of the 5 buttons can have a midi 'message' assigned.
For example:

PRESET 1
button 1: would send a program change, to toggle/activate a certain FX on a certain unit.
button 2: would send a program change, to toggle/activate a certain FX on a certain unit.
button 3: would send a CC to a certain unit to modify one of its parameters (i. e. delay time)
button 4: would send a CC to a certain unit to modify one of its parameters (i. e. delay repeats)
button 5: would send a CC to a certain unit to modify one of its parameters (i. e. fx gain)

Now, for another song, I'd navigate to a 2nd. preset, which will assign a complete different message for each button.

PRESET 2
button 1: would send a CC to a certain unit to modify one of its parameters (i. e. filter depth)
button 2: would send a program change, to toggle/activate a certain FX on a certain unit.
button 3: would send a program change, to toggle/activate a certain FX on a certain unit.
button 4: would send a program change, to toggle/activate a certain FX on a certain unit.
button 5: would send a program change, to toggle/activate a certain FX on a certain unit.

Grumpy_Mike:
You are right it is not very efficient. What are the three bytes value about? For a CC preset send the address of the parameter in the EEPROM as the second byte of the CC message and the contents of the EEPROM address as the third byte.
That is why you do not have these values in SRAM ( dynamic memory ). You make up the message in response to the buttons.

If you are struggling with the concepts of cramming it all in to 1K then you might want to look at having some I2C EEPROM storage.

The three values were just for the example, I'm not sure yet how would I structure it. For CC messages I'd only need 2 values (CC# and value) and for Program changes just one.
I'll be reading about I2C EEPROM storage.

Thanks!

#6

I am still not clear what the difference is between

  1. A CC message
    and
  2. A message to toggle/activate a certain FX
#7

Grumpy_Mike:
I am still not clear what the difference is between

  1. A CC message
    and
  2. A message to toggle/activate a certain FX

They're different type of events/messages.
With Control change, you send 2 values, the cc# and a value (0-127) to transmit. In this case, it's mostly used to set an effect parameter, but It depends on each unit's MIDI implementation.

For example, in my current setup, I can send a CC#3 with a value of 0 to set a delay time to its minimum; or send CC#3 127 to set it to the max.
(This works on my strymon timeline delay, if you're curious you can find the midi implementation here, on the last pages)

The same unit, accepts Program change messages, that allow you to change between the available banks/patches. It only sends one value. If i send a Program change #0 I enable the 1st patch. If i send a program change #1 i enable the 2nd patch, and so on. This allow you to switch between different patches, different delay types in this case.

BTW, Here's a nice summary of MIDI messages.

#8

BTW, Here's a nice summary of MIDI messages.

Thanks, but I did write this book so I know about MIDI messages. http://www.apress.com/9781484217207

You are using the words A message to toggle/activate a certain FX as a euphemism for a program change message, that is what confused me.

#9

Grumpy_Mike:
You are using the words A message to toggle/activate a certain FX as a euphemism for a program change message, that is what confused me.

Oh I see, hehe. Yes, I use PC mostly for that, as I mentioned in the example: "would send a program change, to toggle/activate a certain FX on a certain unit".

I'm reading about EEPROM, as you suggested, to see if I can end up with a first approach to the preset handling/storing. I need to calculate how many presets (groups of 5 messages) can i write, and check out an efficient way to handle reading.

Thanks!

#10

Hi Again,

I’ve been reading about EEPROM and I’m still thinking about how could I structure the data. I’ve also found a couple 24LC64 (8kbyte) around to play with. So, keeping these facts in mind:

  • I need to store 120 presets/banks (This is not arbitrary, I’ve got another unit that has 120 presets and I would sync both)

  • each preset/bank would assign an event (a CC, a PC or a special event) to each 5 buttons available.

  • *what I call a special event is something I’d like to do just to experiment with FX. So, for example, I could define a CC#, 2 values (from - to), a time variable and execute a transition between those values. Another example could be sending a random value everytime I push the button.

  • Events could be created and assigned to button on the fly, through a rotary encoder and a simple menu

I’ve come up with 2 options, but I’m not sure which one is better, more flexible and effcicient… or if there’s a better way, of course. So here’s what I’ve got:

option 1)

Define a fixed size for each event, let’s say an array of 12 bytes. This way I’ll be probably wasting some memory, since a regular PC or CC would only take 2-3 bytes… but 12 bytes would allow me to store the above mentioned custom special events. I’d be using 7200 bytes (5 buttons * 12 bytes * 120 presets). The good thing would be that it’d be easy to read/write, since I could get the address by calculating (preset number-1*60) + (button number-1 *12). The data should be stored this way:

preset#      button#      EEPROM address
----------------------------------------------
      1            1             0
----------------------------------------------
      1            2            12
----------------------------------------------
      1            3            24
----------------------------------------------
      1            4            36
----------------------------------------------
      1            5            48
----------------------------------------------
      2            1            60
----------------------------------------------
      2            2            72
----------------------------------------------
      2            3            84
----------------------------------------------
      2            4            96
----------------------------------------------
      2            5           108
----------------------------------------------
      .            .             .
      .            .             .
      .            .             .
----------------------------------------------
    120            1          7140
----------------------------------------------
    120            2          7152
----------------------------------------------
    120            3          7164
----------------------------------------------
    120            4          7176
----------------------------------------------
    120            5          7188

option 2)

Create events with different sizes (i.e. a PC would take 2 bytes, a CC 3 bytes and a special event 12 bytes) and store them sequentially. This option would use the available memory more efficiently (i guess) but I should save somehow each event’s address… And addresses on these chips need 2 bytes, so I should spare 1200 bytes to save each button event’s address (5 buttons * 120 presets * 2 bytes). The rest of the memory could be filled with events.

Now I like this approach, but I’ve got a few questions:

a) what happens if I want to delete an event (consequently unassign from the button and ‘free’ some space)? should I sort / re-allocate the following events and save the new addresses?

b) To calculate the available space in order to restrict event creation, should I read the entire memory?

What do you think? am I on the right direction with any of these options?
Any other option or suggestion will be really appreciated.

Thanks a lot.
Jon

#11

What do you think? am I on the right direction with any of these options?

If you are not memory constrained, I like the simplicity of the structured storage of option 1.

Communication with an external eeproms is with the Wire library, and they can be written or read one byte at a time, or in a block mode up to the 32 byte buffer size of the library. The block read/write mode is faster.

One thing to be aware of with these external eeproms is that they have an internal page structure. For the 24LC64 it is 32 bytes. You can not block write across a page boundary.

This complication is often dealt with by the use of an external eeprom library. I recommend Jack Christensen's library extEEPROM at GitHub - JChristensen/extEEPROM: Arduino library to support external I2C EEPROMs.

#12

In effect you are writing the equivalent of a disc filling system with option 2 and so you face the same problems of fragmentation of the memory. You could run a de-frag routine to recover the space occasionally like disc filling systems do.

But I agree option 1 is simple. You can have multiple chips on the I2C bus so you should be able to get enough memory.

#13