Button matrix works for usb midi but not print

[chagocruz]

So a general overview of what I am trying to currently accomplish: I'm trying to use a button matrix for the "keyboard" part of the synth. In this example, it technically makes it a usb midi controller. With this, I can send MIDI notes via usb to a DAW or something. This works fine but it's not quite what I want to do. Instead of sending midi over usb to my computer, I want my corresponding button to send a frequency (or MIDI note, or note name, etc) to a function that accepts the frequency to do all the synth oscillator stuff.

#include <config.h>
#include <Control_Surface.h>
#include <ResponsiveAnalogRead.h>

// Instantiate a MIDI over USB interface.
USBMIDI_Interface midi;

const int maxTransposition = 4;
const int minTransposition = -1 * maxTransposition;
const int transpositionSemitones = 12;
Transposer<minTransposition, maxTransposition>transposer(transpositionSemitones);

//keyboard notes section
const AddressMatrix<2, 14> noteAddresses = {{
                                                {1, 54, 56, 58, 1, 61, 63, 1, 66, 68, 70, 1, 73, 75},
                                                {53, 55, 57, 59, 60, 62, 64, 65, 67, 69, 71, 72, 74, 76},  
                                            }};

Bankable::NoteButtonMatrix<2, 14> noteButtonMatrix = {
    transposer,
    {ROW_3, ROW_4}, // row pins
    {COL_0, COL_1, COL_2, COL_3, COL_4, COL_5, COL_6, COL_7, COL_8, COL_9, COL_10, COL_11, COL_12, COL_13},    // column pins
    noteAddresses,    // address matrix
    CHANNEL_1,    // channel and cable number
};

//top row buttons section
const AddressMatrix<1,11> presetAddresses = {{
                                                {59, 60, 62, 64, 65, 67, 69, 71, 72, 74, 76}
                                            }};

Bankable::NoteButtonMatrix<1, 11> presetButtonMatrix = {
    transposer,
    {ROW_2}, // row pins
    {COL_3, COL_4, COL_5, COL_6, COL_7, COL_8, COL_9, COL_10, COL_11, COL_12, COL_13},    // column pins
    presetAddresses,    // address matrix
    CHANNEL_1,    // channel and cable number
};

void setup() {
    Serial.begin(9600); 
    Control_Surface.begin(); // Initialize Control Surface
    pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
    Control_Surface.loop(); // Update the Control Surface
    Serial.print("noteButtonMatrix");
    digitalWrite(LED_BUILTIN, HIGH);
    //delay(20);
}

In this example below I am trying to figure out how to accomplish what I said above with print statements to the console. The issue with this code is that sometimes if I press certain buttons, it prints two statements as if I have multiple buttons pressed. If I press the button 'Q', the console shows I have Q and 3 pressed. It should ONLY be Q. I can't seem to figure out why the above example worked fine and I could have all my figures on the keyboard with no issue, but when trying to send it to a print statement, it acts all wonky lol

#include <Arduino.h>
#include <unordered_set>

const int numRows = 3;
const int numCols = 14;

const int rowPins[numRows] = {29, 30, 31};
const int colPins[numCols] = {32, 0, 1, 9, 5, 4, 3, 2, 14, 24, 25, 26, 27, 28};

const char buttonChars[numRows][numCols] = {
  {'\0', '\0', '\0', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N'},
  {'\0', 'P', 'Q', 'R', '\0', 'T', 'U', '\0', 'W', 'X', 'Y', '\0', '!', '?'},
  {'1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '-', '+', '*', '/'}
};

std::unordered_set<char> pressedButtons;

void setup() {
  for (int i = 0; i < numRows; i++) {
    pinMode(rowPins[i], OUTPUT);
    digitalWrite(rowPins[i], HIGH);
  }

  for (int j = 0; j < numCols; j++) {
    pinMode(colPins[j], INPUT_PULLUP);
  }

  Serial.begin(9600);
}

void scanMatrix() {
  pressedButtons.clear();  // Clear the set before scanning

  for (int i = 0; i < numRows; i++) {
    digitalWrite(rowPins[i], LOW);

    for (int j = 0; j < numCols; j++) {
      if (digitalRead(colPins[j]) == LOW) {
        char pressedChar = buttonChars[i][j];
        if (pressedChar != '\0') {
          pressedButtons.insert(pressedChar);
        }
      }
    }

    digitalWrite(rowPins[i], HIGH);
  }

  if (!pressedButtons.empty()) {
    Serial.print("Pressed buttons: ");
    for (char button : pressedButtons) {
      Serial.print(button);
      Serial.print(" ");
    }
    Serial.println();
  } else {
    Serial.println("No buttons pressed");
  }
}

void loop() {
  scanMatrix();
  delay(100);
}

Here is my schematic of my matrix as well:

[xfpd]

What happens if you scan the rows backwards starting with 4 then 3, 2. Maybe the other direction has residual discharge.