morse decoder problem

sbaratheon · August 30, 2015, 9:38pm

I thought it may be fun to write a morse decoder for the arduino (it uses a ldr but thats not particular important).

Here is my code:

#define DOT '.'
#define DASH '-'

#define UNIT_TIME_MS 100

int BASE_LIGHT_LEVEL;
const int LIGHT_SENSITIVITY = 200;

int pin = 0;

void setup() {

  Serial.begin(115200);
  BASE_LIGHT_LEVEL = calibrate_sensor(pin);
  
  Serial.println(BASE_LIGHT_LEVEL);
}

String test;

void loop() {
  
  if (get_light_level(pin,BASE_LIGHT_LEVEL))
  listen_morse();
  
   
}


void listen_morse()
{  
  
  delay (60); // read 60ms into each time unit

  while(true)
  {
    char a = nextChar();
    Serial.println(a);
  }

}

char nextChar()
{
  static char buffer [21] = "";
  
  while (true)
  {
    int len = strlen(buffer);
    buffer [len] = get_light_level(pin,BASE_LIGHT_LEVEL) + 48;
    buffer [len + 1] = '\0';
            
    delay (UNIT_TIME_MS); 
               
     if (strcmp (buffer,"0000000") == 0)  //return space
     {
       buffer[0] = '\0';
       return ' ';
     }
     if (strstr(buffer,"0001"))  //new char
     {
        buffer [strlen(buffer) - 4] = '\0';
        char ascii = morse_to_ascii(raw_to_morse(buffer));
              
        buffer[0] = '1';  
        buffer[1] = '\0';

  
        return ascii;      
     }
     
     else if (strstr(buffer,"0000") && strlen(strstr(buffer,"0000")) == 4 )  // must be space
     {
        buffer [strlen(buffer) - 4] = '\0';
        char ascii = morse_to_ascii(raw_to_morse(buffer));
        
        buffer [4] = '\0'; 
        memset (buffer,'0',4);   
        
        return ascii;
     }
  }
 
 }

char* raw_to_morse (const char* raw_input)
{
  static char morse [10] = "";
  memset (morse,'\0',10); //reset at the beginning of each call

  for (int i=0; i < strlen (raw_input);)
  {
     if (raw_input [i + 1] == '0' || i + 1 == strlen (raw_input)) // if dot
     {
       morse [strlen(morse)] = DOT;

       if (i + 1 >= strlen (raw_input))   //end of morse word
         break;

       else
         i = i + 2;   //move to next morse symbol
     }
     
     else          //if dash
     {
        morse [strlen(morse)] = DASH;

        if (i + 3 >= strlen (raw_input))  //end of morse word
          break;

        else                            //move to next morse symbol
          i = i + 4;
     }
  }
  return morse;
}

char morse_to_ascii (const char* morse)
{
  //letters

  if (strcmp (morse,".-") == 0)
    return 'a';

  if (strcmp (morse,"-...") == 0)
    return 'b';

  if (strcmp (morse,"-.-.") == 0)
    return 'c';

  if (strcmp (morse,"-..") == 0)
    return 'd';

  if (strcmp (morse,".") == 0)
    return 'e';

  if (strcmp (morse,"..-.") == 0)
    return 'f';

  if (strcmp (morse,"--.") == 0)
    return 'g';

  if (strcmp (morse,"....") == 0)
    return 'h';

  if (strcmp (morse,"..") == 0)
    return 'i';

  if (strcmp (morse,".---") == 0)
    return 'j';

  if (strcmp (morse,"-.-") == 0)
    return 'k';

  if (strcmp (morse,".-..") == 0)
    return 'l';

  if (strcmp (morse,"--") == 0)
    return 'm';

  if (strcmp (morse,"-.") == 0)
    return 'n';

  if (strcmp (morse,"---") == 0)
    return 'o';

  if (strcmp (morse,".--.") == 0)
    return 'p';

  if (strcmp (morse,"--.-") == 0)
    return 'q';

  if (strcmp (morse,".-.") == 0)
    return 'r';

  if (strcmp (morse,"...") == 0)
    return 's';

  if (strcmp (morse,"-") == 0)
    return 't';

  if (strcmp (morse,"..-") == 0)
    return 'u';

  if (strcmp (morse,"...-") == 0)
    return 'v';

  if (strcmp (morse,".--") == 0)
    return 'w';

  if (strcmp (morse,"-..-") == 0)
    return 'x';

  if (strcmp (morse,"-.--") == 0)
    return 'y';

  if (strcmp (morse,"--..") == 0)
    return 'z';

  //numbers
  
  
  
  //prosigns
  
  if (strcmp(morse,".-.-."))
    return '+';
}

bool get_light_level (byte pin,int BASE_LIGHT_LEVEL)
{
  if (BASE_LIGHT_LEVEL - analogRead(pin) > LIGHT_SENSITIVITY)
  {
    //Serial.print(true);
    return true;
  }
  
  else
  {
    //Serial.print(false);

    return false;
  } 
  
}

int calibrate_sensor (int pin)
{
    int total_light_val = 0;
    
    for (int i=0; i <3;i++) //3 AVERAGES
    {
      total_light_val = total_light_val + analogRead (pin);
      delay (100);
    }
 
    return total_light_val/3; //return average base light level
}

It still needs a bit of work but seems to fundamentally work except for one strange issue.

The delay in listen_morse (60ms) is designed to offset the read a certain way through each 100ms time unit. However, if I changed it to 50ms for example the decoding seems to break. For the life of me I can’t understand why this is - I could understand if it was equal to or greater than 100ms.

Any ideas on whats going on here?

Thanks

luisilva · August 30, 2015, 9:41pm

Don't use delay() use instead millis() function. Take a look to the example program Blink Without Delay.

nickgammon · August 30, 2015, 10:11pm

void listen_morse()
{ 
 
  delay (60); // read 60ms into each time unit

  while(true)
  {
    char a = nextChar();
    Serial.println(a);
  }

}

This is very timing-specific. I would look for level changes (eg. high to low, low to high), measure the difference, and put them into an array. As I recall, a dash is 3 x the dot length, the gap between letters is a dash length, and word gap is 2 x the dash length. And there is a dot length between dots and dashes.

So the first thing you could deduce is that the gaps will either be a dot length, or if a gap is long (3 times as long) you have a letter gap, and very long (6 times as long) is a word gap.

Now you can work out dots and dashes, since there is quite a big leeway. For example:

on:  100 : dot
off: 100 
on:  100 : dot
off: 100
on:  300 : dash
off: 300  -- end of letter

That gives ..- hence the letter F.

You wouldn't need a very large array, because once you get a gap of 2 x the dot length you can assume you are 2/3 through a dash gap, and therefore at the end of the current letter, which you can now decode.

Of course you wouldn't test for exactly equal to 100, but give-or-take 10 or 20%.

So therefore an interval of 80 ms to 120 ms could be considered a dot, an interval of 280 to 320 ms a dash, and so on (depending on the exact rate at which the morse is transmitted), which you can deduce, by measuring gaps.

sbaratheon · August 30, 2015, 11:02pm

luisilva:
Don’t use delay() use instead millis() function. Take a look to the example program Blink Without Delay.

I’ll try this but I’m not sure how it’ll help.

jurs · August 30, 2015, 11:23pm

sbaratheon:
I'll try this but I'm not sure how it'll help.

I think there are some Arduino morse decoders existing, and the better ones never use delay(), except a very short 2ms delay for a software debouncing of the signal detection. The 'delay()' function has the meaning of "block program and stop program execution for x milliseconds", and this is the worst thing you can do in real-time signal processing.

The better morse decoders also don't use a fixed timing, but are adaptive to the actual speed of the dits and dahs sent. Those morse decoders try to detect the (average) length of a 'dah', and from that they create the timings and tolerances for a dit, a inter-element gap, a inter-character gap and the gap between words.

If you google for "WB7FHC Open-Source" you should find different projects and modifications of a working morse decoder, created as open source for Arduino by a radio amateur.

sbaratheon · August 31, 2015, 12:23am

jurs:
I think there are some Arduino morse decoders existing, and the better ones never use delay(), except a very short 2ms delay for a software debouncing of the signal detection. The 'delay()' function has the meaning of "block program and stop program execution for x milliseconds", and this is the worst thing you can do in real-time signal processing.

The better morse decoders also don't use a fixed timing, but are adaptive to the actual speed of the dits and dahs sent. Those morse decoders try to detect the (average) length of a 'dah', and from that they create the timings and tolerances for a dit, a inter-element gap, a inter-character gap and the gap between words.

If you google for "WB7FHC Open-Source" you should find different projects and modifications of a working morse decoder, created as open source for Arduino by a radio amateur.

Thank you - that's very helpful.

Why is using delay bad though? Why does it matter that the execution is blocked?

aarg · August 31, 2015, 1:16am

sbaratheon:
Why is using delay bad though? Why does it matter that the execution is blocked?

It can easily prevent integration with other functions - UI or hardware interfaces.

econjack · August 31, 2015, 2:49am

Encoding can be much simpler using the ideas presented in post #6 at:

If you are reading char data, do a toupper() on each letter so it falls within the array.

nickgammon · August 31, 2015, 5:50am

Here’s my stab at it. First we need some test data, so I adapted my earlier morse code generator program to output a fixed sequence:

// Morse code generator
// Author: Nick Gammon
// Date: 8th November 2014
// Version: 1

const byte LED = 13;

char * letters [26] = {
   ".-",     // A
   "-...",   // B
   "-.-.",   // C
   "-..",    // D
   ".",      // E
   "..-.",   // F
   "--.",    // G
   "....",   // H
   "..",     // I
   ".---",   // J
   "-.-",    // K
   ".-..",   // L
   "--",     // M
   "-.",     // N
   "---",    // O
   ".--.",   // P
   "--.-",   // Q
   ".-.",    // R
   "...",    // S
   "-",      // T
   "..-",    // U
   "...-",   // V
   ".--",    // W
   "-..-",   // X
   "-.--",   // Y
   "--.."    // Z
};

char * numbers [10] = 
  {
  "-----",  // 0
  ".----",  // 1
  "..---",  // 2
  "...--",  // 3
  "....-",  // 4
  ".....",  // 5
  "-....",  // 6
  "--...",  // 7
  "---..",  // 8
  "----.",  // 9
  };
  
const unsigned long dotLength = 100;  // mS
const unsigned long dashLength = dotLength * 3;  
const unsigned long wordLength = dashLength * 2; 

void dot ()
  {
  digitalWrite (LED, HIGH);
  delay (dotLength);
  digitalWrite (LED, LOW); 
  delay (dotLength);
  }  // end of dot

void dash ()
  {
  digitalWrite (LED, HIGH);
  delay (dashLength);
  digitalWrite (LED, LOW); 
  delay (dotLength);
  }  // end of dash
    
void setup ()
  {
  Serial.begin (115200);
  Serial.println ();
  pinMode (LED, OUTPUT);
  }  // end of setup

void loop ()
  {
  
  char buf [] = "Twas brillig and the slithy toves "
                "Did gyre and gimble in the wabe "
                "All mimsy were the borogoves "
                "And the mome raths outgrabe "
                "Beware the Jabberwock my son "
                "The jaws that bite the claws that catch "
                "Beware the Jubjub bird and shun "
                "The frumious Bandersnatch ";
      
  // for each letter
  for (char * p = buf; *p; p++)
    {
    char c = *p;
    char * sequence = NULL;
    c = toupper (c);
    if (c >= 'A' && c <= 'Z')
      sequence = letters [c - 'A'];
    else if (c >= '0' && c <= '9')
      sequence = numbers [c - '0'];   
    else if (c == ' ')
      {
      delay (wordLength);   // gap between words
      continue;
      }
    
    // ignore not in table
    if (sequence == NULL)
      continue;
      
    // output sequence for one letter
    for (char * s = sequence; *s; s++)
      {
      if (*s == '.')
        dot ();
      else if  (*s == '-')
        dash (); 
      }
    // now a gap
    delay (dashLength);   
    }  // end of for each letter
      
    delay (1000);
  }  // end of loop

Now for the decoder.

// Morse code decoder
// Author: Nick Gammon
// Date: 31 August 2015

const byte SIGNAL_PIN = 2;
const int SIGNAL_COUNT = 10;  // maximum number of dots/dashes in a letter

// calculated later on
unsigned long dotLength = 0;  // mS
unsigned long dashLength;  
unsigned long wordLength; 
unsigned long FUZZ_FACTOR;

volatile unsigned int widths [SIGNAL_COUNT];
volatile byte count;
volatile bool letterDone;
volatile bool haveSpace;
volatile bool adjusting = true;

volatile unsigned long lastPulse;

char * letters [26] = {
   ".-",     // A
   "-...",   // B
   "-.-.",   // C
   "-..",    // D
   ".",      // E
   "..-.",   // F
   "--.",    // G
   "....",   // H
   "..",     // I
   ".---",   // J
   "-.-",    // K
   ".-..",   // L
   "--",     // M
   "-.",     // N
   "---",    // O
   ".--.",   // P
   "--.-",   // Q
   ".-.",    // R
   "...",    // S
   "-",      // T
   "..-",    // U
   "...-",   // V
   ".--",    // W
   "-..-",   // X
   "-.--",   // Y
   "--.."    // Z
};

char * numbers [10] = 
  {
  "-----",  // 0
  ".----",  // 1
  "..---",  // 2
  "...--",  // 3
  "....-",  // 4
  ".....",  // 5
  "-....",  // 6
  "--...",  // 7
  "---..",  // 8
  "----.",  // 9
  };
  
// ISR
void gotPulse ()
  {
    
  unsigned long now = millis ();
  unsigned int width = now - lastPulse;
  lastPulse = now;
  
  byte pinState = digitalRead (SIGNAL_PIN);
  
  if (!adjusting)
    {
    // a long gap means we start again
    if (pinState == HIGH && width >= (dashLength - FUZZ_FACTOR))
      count = 0;
  
    // a really long gap means we a space
    if (pinState == HIGH && width >= (wordLength - FUZZ_FACTOR))
      haveSpace = true;
    }
    
  if (count >= SIGNAL_COUNT)
    return;
    
  if (pinState == LOW && !letterDone)
    widths [count++] = width;
  }
  
void processCharacter ()
  {
  if (haveSpace)
    {
    Serial.print (" ");
    haveSpace = false;
    }
         
  char result [SIGNAL_COUNT + 1];  
  for (int i = 0; i < count; i++)
    {
    unsigned int width = widths [i];
    if (width < dotLength + FUZZ_FACTOR)
      result [i] = '.';
    else
      result [i] = '-';
    }
  result [count] = 0;  // null terminator
  
//  Serial.print (result);
//  Serial.print (" ");
  
  for (int i = 0; i < 26; i++)
    if (strcmp (result, letters [i]) == 0)
      {
      Serial.print (char (i + 'A'));
      return;
      }
      
  for (int i = 0; i < 10; i++)
    if (strcmp (result, numbers [i]) == 0)
      {
      Serial.print (char (i + '0'));
      return;
      }
  
  Serial.print ("?");   
  }  // end of processCharacter
 
void calculateWidths ()
  {
  // ignore the first one, might be spurious
  dotLength = widths [1];
  
  for (int i = 2; i < count; i++)
    {
    unsigned int width = widths [i];
    if (width < dotLength / 2)  // less than half the length?
      dotLength = width;
    }  
    
  dashLength = dotLength * 3;  
  wordLength = dashLength * 2; 
  FUZZ_FACTOR = dotLength / 10;  
  adjusting = false;
  count = 0;
  Serial.print ("Dot width = ");
  Serial.println (dotLength);
  }  // end of calculateWidths
  
void setup ()
  {
  Serial.begin (115200);
  Serial.println ();
  Serial.println ("Starting ...");
  attachInterrupt (0, gotPulse, CHANGE);
  EIFR = bit (INTF0);  // clear flag for interrupt 0
  }  // end of setup

void loop ()
  {
  if (adjusting && count >= SIGNAL_COUNT)
    calculateWidths ();

  if (adjusting)
    return;
    
  if (digitalRead (SIGNAL_PIN) == LOW && 
     (millis () - lastPulse) >= (dotLength * 2) &&
     count > 0
     )
    {
    letterDone = true;
    processCharacter ();
    count = 0;
    letterDone = false;
    }
  }  // end of loop

Connect pin 13 of the generator Uno (also visible on the LED) to pin 2 of the decoder Uno (where it can trigger an interrupt).

First the decoder reads a series of dots/dashes to try to work out which is a dot. Hopefully it gets a dot in that first string.

Then, based on that, it detects on-sequences and stores them in an array. That gets turned into a string, and a string compare deduces which letter/number it is. It’s not the most efficient code, but it works.

Output:

Dot width = 101
GYRE AND GIMBLE IN THE WABE ALL MIMSY WERE THE BOROGOVES AND THE MOME RATHS OUTGRABE BEWARE THE JABBERWOCK MY SON THE

As you can see, it got the dot width right except for one millisecond. And the decoded letters were correct.

It still decoded OK with the sender having a dot length of 20 ms (five times as fast) so I think that shows the general idea is OK.

Paul_KD7HB · September 1, 2015, 8:46pm

Notice this works with computer-sent CW. Hand sent CW is a whole 'nother animal. Mine, for example, will have timing that continually changes.

Paul

nickgammon · September 1, 2015, 9:39pm

I don’t doubt it. However by tweaking the fuzz-factor in the code you should be able to get reasonable results. After all a dash is nominally 3 dots, but you could probably call a dash 2 dots to 4 dots. Also you might continually adjust what you think a dot length is.

If you saved all the incoming lengths to an array (say, of 50 items) you could probably work out what the “running dot length” is.

system · September 1, 2015, 10:38pm

Few years back I wrote CW decoder in Basic (Stamp), I would have to let Windows try to find it on my PC.
I did publish it in QRZ. It may still be there.
I used dot as a basic time unit and calculated the dot to dash ratio dynamically from there.
It was pretty independent from any LID QLF skills.
I find it entertaining, sorry, for people not being able to distinguish between task of dot/ dash decoding and Morse code alphabet look-up table.
73 AA7EJ

econjack · September 2, 2015, 12:06am

My Arduino Project for Amateur Radio book has a decoder in it and we felt lucky to get a little over 30 wpm from it. Even practice from W1AW is tough. At 15wpm, it uses the 3-1 dash-dot ratio plus the standard spacing...no problem. However, when the practice runs are 18wpm or higher, their practice switches to Farnsworth encoding, which pads the character spacing. In actual use, pile on top of that someone who has a bad "fist", QRN (static), and QSB (fading), and it's not a trivial task. Nick: the running dot length idea is worth investigating.

system · May 5, 2021, 9:27pm