New soft serial buffer maxes at 255

Hi; I’m working on a project where I need to receive more than 256 bytes in one buffer from the new soft serial library. I changed the library from the default which helped, but it maxes out at 256 even if I change the library to 512. I am running Teensyduino 2.0++ with arduino 22. I used the memfree program to verify I have lots of RAM left 1.5k Any help would be appreciated.

here is the library I'm using:

ifndef NewSoftSerial_h
#define NewSoftSerial_h

#include <inttypes.h>
#include "Print.h"

/******************************************************************************
* Definitions
******************************************************************************/

#define _NewSS_MAX_RX_BUFF 512 // RX buffer size
#define _NewSS_VERSION 10 // software version of this library
#ifndef GCC_VERSION
#define GCC_VERSION (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__)
#endif

class NewSoftSerial : public Print
{
private:
  // per object data
  uint8_t _receivePin;
  uint8_t _receiveBitMask;
  volatile uint8_t *_receivePortRegister;
  uint8_t _transmitBitMask;
  volatile uint8_t *_transmitPortRegister;

  uint16_t _rx_delay_centering;
  uint16_t _rx_delay_intrabit;
  uint16_t _rx_delay_stopbit;
  uint16_t _tx_delay;

  uint16_t _buffer_overflow:1;
  uint16_t _inverse_logic:1;

  // static data
  static char _receive_buffer[_NewSS_MAX_RX_BUFF]; 
  static volatile uint8_t _receive_buffer_tail;
  static volatile uint8_t _receive_buffer_head;
  static NewSoftSerial *active_object;

  // private methods
  void recv();
  bool activate();
  virtual void write(uint8_t byte);
  uint8_t rx_pin_read();
  void tx_pin_write(uint8_t pin_state);
  void setTX(uint8_t transmitPin);
  void setRX(uint8_t receivePin);

  // private static method for timing
  static inline void tunedDelay(uint16_t delay);

public:
  // public methods
  NewSoftSerial(uint8_t receivePin, uint8_t transmitPin, bool inverse_logic = false);
  ~NewSoftSerial();
  void begin(long speed);
  void end();
  int read();
  uint8_t available(void);
  bool active() { return this == active_object; }
  bool overflow() { bool ret = _buffer_overflow; _buffer_overflow = false; return ret; }
  static int library_version() { return _NewSS_VERSION; }
  static void enable_timer0(bool enable);
  void flush();

  // public only for easy access by interrupt handlers
  static inline void handle_interrupt();
};

// Arduino 0012 workaround
#undef int
#undef char
#undef long
#undef byte
#undef float
#undef abs
#undef round

#endif

Hi; I’m working on a project where I need to receive more than 256 bytes in one buffer from the new soft serial library.

Why are you waiting until the buffer is full to start reading? Even with a 64 character buffer, you should never let it get full. You can read far faster than data can arrive.

Its bieng displayed on a lcd and is manipulated by several aspects of the program and user interface. I guess i could try stuffing it all into a string. I was hoping i could make some small changes to the library to get it to work.

You can't do anything with the data when it is in the serial buffer. You have to read the data and store it somewhere else first. Therefore, making the buffer larger won't help.

If the buffer is filling up, you need to work out handshaking with the sender so that doesn't happen.

Ok, I tried this and am still having issues. I stuff the data into strings as soon as two(my com port) is available. it seems it works better however the data after the 256 buffer starts to break up and does not receive correctly. I have tried several method unable to produce clear data after the buffer. Here is my code:

      while(keyPad != 34 && keyPad != 31)
      {
        c = 0;

dataGather:

        while(two.available() && c <= 160) // gather data for page 1
        { 
          a = two.read();
          data1 += a; 
          c++;
        }     
        while(two.available() && c <= 320) // gather data for page 2
        { 
          a = two.read();
          data2 += a; 
          c++;
        }
        while(two.available() && c <= 480) // gather data for page 3
        { 
          a = two.read();
          data3 += a;
          c++; 
        }
        while(two.available() && c <= 640) // gather data for page 4
        { 
          a = two.read();
          data4 += a; 
          c++;
        }

        delay(20);         // wait for any missed data
        if(two.available())
          goto dataGather;



        Serial.println(data1); // for trouble shooting
        Serial.println(data2); // for trouble shooting
        Serial.println(data3); // for trouble shooting
        Serial.println(data4); // for trouble shooting

        keyPad = 0;

        //----------------------display serial data 1

        if (data1.length() > 0)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);         
          Uart.print(data1);

          while(keyPad != 43 && data1.length() >= 160) // wait for user input
          {
            delay(50);
          }


          keyPad = 0;

        }
        //----------------------display serial data 2
        if (data1.length() >= 160)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);
          Uart.print(data2);           

          while(keyPad != 43 && data2.length() >= 160) // wait for user input
          {
            delay(50);
          }
          keyPad = 0;
        } 

        //----------------------display serial data 3
        if (data2.length() >= 160)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);
          Uart.print(data3);       

          while(keyPad != 43 && data2.length() >= 160) // wait for user input
          {
            delay(50);
          }
          keyPad = 0;

        } 

        //----------------------display serial data 4
        if (data3.length() >= 160)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);
          Uart.print(data4);
        }   


        //----------------------send serial data out

        data1 = ""; // clear string
        data2 = ""; // clear string
        data3 = ""; // clear string
        data4 = ""; // clear string

         keyPad = 0;
         
         while(keyPad == 0) // wait for user input as soon as data is sent to port it will respond
        {
        delay(50);
        } 
         
          if (keyPad == 21)
          {
            two.print("Y");
            Uart.print("Y"); 
          }
          if (keyPad == 22)
          {
            two.print("N");
            Uart.print("N");  
          }
          if (keyPad == 26)
          {
            two.print("A");
            Uart.print("A");  
          }
          if (keyPad == 27)
          {
            two.print("B");
            Uart.print("B");  
          }
          if (keyPad == 28)
          {
            two.print("C");
            Uart.print("C");  
          }
          if (keyPad == 29)
          {
            two.print("D");
            Uart.print("D");  
          }
          if (keyPad == 1)
          {
            two.print("1");
            Uart.print("1");  
          }
          if (keyPad == 2)
          {
            two.print("2");
            Uart.print("2");        
          }
          if (keyPad == 3)
          {
            two.print("3");
            Uart.print("3");  
          }
          if (keyPad == 4)
          {
            two.print("4");
            Uart.print("4");  
          }
          if (keyPad == 5)
          {
            two.print("5");
            Uart.print("5");  
          }
          if (keyPad == 6)
          {
            two.print("6");
            Uart.print("6");  
          }
          if (keyPad == 7)
          {
            two.print("7");
            Uart.print("7");  
          }
          if (keyPad == 8)
          {
            two.print("8");
            Uart.print("8");  
          }
          if (keyPad == 9)
          {
            two.print("9");
            Uart.print("9");  
          }
          if (keyPad == 10)
          {
            two.print("10");
            Uart.print("10");  
          }
          if (keyPad == 32 || keyPad == 44)
          {
            two.print("-");
            Uart.print("-");  
          }

          if (keyPad == 30)
          {
            two.print("\r");
            Uart.print("enter");  
          }
       
      }

my received data looks like this (after "B - DELT" is over 256):

This utility allows a technican to describe the dial face to the meter.
The meter uses these values to determine the correct needle placement.
The OUTER set
of numbers on the dial is the MAIN scale (#1)
The INNER set of numbers on the dial is the DELTA, WOB or 2nd scale
The Main scale is always used in NORMAL MODE

What is the second scale function? [Pick one letter from the list]
A - Normal mode w/ alternate calibration eg: #of lines or English/Metric
B - Delt
a mo- ransivlae ightod -igOnt e - xtsetiresefling
Foornfat culhener' Mal

Get rid of the labels and goto statements, then we'll talk.

done, same thing.

 while(keyPad != 34 && keyPad != 31)
      {
        c = 0;

        while(two.available() && c <= 160) // gather data for page 1
        { 
          a = two.read();
          data1 += a; 
          c++;
        }     
        while(two.available() && c <= 320) // gather data for page 2
        { 
          a = two.read();
          data2 += a; 
          c++;
        }
        while(two.available() && c <= 480) // gather data for page 3
        { 
          a = two.read();
          data3 += a;
          c++; 
        }
        while(two.available() && c <= 640) // gather data for page 4
        { 
          a = two.read();
          data4 += a; 
          c++;
        }

      
        Serial.println(data1); // for trouble shooting
        Serial.println(data2); // for trouble shooting
        Serial.println(data3); // for trouble shooting
        Serial.println(data4); // for trouble shooting

        keyPad = 0;

        //----------------------display serial data 1

        if (data1.length() > 0)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);         
          Uart.print(data1);

          while(keyPad != 43 && data1.length() >= 160) // wait for user input
          {
            delay(50);
          }


          keyPad = 0;

        }
        //----------------------display serial data 2
        if (data1.length() >= 160)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);
          Uart.print(data2);           

          while(keyPad != 43 && data2.length() >= 160) // wait for user input
          {
            delay(50);
          }
          keyPad = 0;
        } 

        //----------------------display serial data 3
        if (data2.length() >= 160)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);
          Uart.print(data3);       

          while(keyPad != 43 && data2.length() >= 160) // wait for user input
          {
            delay(50);
          }
          keyPad = 0;

        } 

        //----------------------display serial data 4
        if (data3.length() >= 160)
        {
          lcd.clearLCD();
          lcd.gotoPosition(0,0);
          Uart.print(data4);
        }   


        //----------------------send serial data out

        data1 = ""; // clear string
        data2 = ""; // clear string
        data3 = ""; // clear string
        data4 = ""; // clear string

         keyPad = 0;
         
         while(keyPad == 0) // wait for user input as soon as data is sent to port it will respond
        {
        delay(50);
        } 
         
          if (keyPad == 21)
          {
            two.print("Y");
            Uart.print("Y"); 
          }
          if (keyPad == 22)
          {
            two.print("N");
            Uart.print("N");  
          }
          if (keyPad == 26)
          {
            two.print("A");
            Uart.print("A");  
          }
          if (keyPad == 27)
          {
            two.print("B");
            Uart.print("B");  
          }
          if (keyPad == 28)
          {
            two.print("C");
            Uart.print("C");  
          }
          if (keyPad == 29)
          {
            two.print("D");
            Uart.print("D");  
          }
          if (keyPad == 1)
          {
            two.print("1");
            Uart.print("1");  
          }
          if (keyPad == 2)
          {
            two.print("2");
            Uart.print("2");        
          }
          if (keyPad == 3)
          {
            two.print("3");
            Uart.print("3");  
          }
          if (keyPad == 4)
          {
            two.print("4");
            Uart.print("4");  
          }
          if (keyPad == 5)
          {
            two.print("5");
            Uart.print("5");  
          }
          if (keyPad == 6)
          {
            two.print("6");
            Uart.print("6");  
          }
          if (keyPad == 7)
          {
            two.print("7");
            Uart.print("7");  
          }
          if (keyPad == 8)
          {
            two.print("8");
            Uart.print("8");  
          }
          if (keyPad == 9)
          {
            two.print("9");
            Uart.print("9");  
          }
          if (keyPad == 10)
          {
            two.print("10");
            Uart.print("10");  
          }
          if (keyPad == 32 || keyPad == 44)
          {
            two.print("-");
            Uart.print("-");  
          }

          if (keyPad == 30)
          {
            two.print("\r");
            Uart.print("enter");  
          }
       
      }





    }

I might have missed it, but what is data1, et al? If they are Strings, that might be your problem since you are doing a lot of malloc'ing and free'ing behind the scenes. Better to roll your own char arrays.

Sorry, my whole program is way to long to post. Can you provide an example?

unsigned long time = 0;
    long injectorLow;
    long injectorHigh;
    long injectorMid;
    int curOutInputRaw;
    byte b = 0;
    int c = 0;
    byte d = 99;
    char a;
    String data1;
    String data2;
    String data3;
    String data4;
    char order1[] = {
      'N','N','N','Y','A','N'                    };
    long order2[] = {
      -17750,35586,-6672,6672                    };
    byte manuel = 0;

instrumentek:
Sorry, my whole program is way to long to post.

In that case most of the sketch is probably not relevant to the problem. Extract the relevant parts into a simple sketch that just demonstrates the problem, and post that. (Make sure the sketch does actually compile and run and demonstrates the problem.)

instrumentek:
Sorry, my whole program is way to long to post.

You can attach a file to a post. Look at Additional Options... bottom/left of the edit window. That gets round the size limitation on posting long code.

char data1[256];
int data1_idx = 0;
int c=0;

...
     while(two.available() && c <= 160) // gather data for page 1
        { 
          a = two.read();
          data1[data1_idx++] = a; 
          data1[data1_idx] = '\0'; //Put in nul 
          if (data1_idx == 255) 
          {
                  //Error! Do something!
          }
          c++;
        }

The two.available() condition may never be true. Or, it may not be true before keyPad takes on one of the non-proscribed values.

Where is keyPad assigned a value? What do the values 34 and 31 mean?

What is connected to the instance called two? Are you, somewhere outside the code you've posted so far, waiting for two.available() to return a value greater than 0?

hey, i made a small sketch that does not use keypad. still has same issue i'm going to post it shortly. 31 and 34 are the exit keys. your last question i will post the complete sketch demonstrating this issue shortly

Thanks for help so far, I made this program as you suggested. it has the same issue the data output on the longer streams is garbled

#include <NewSoftSerial.h>
#include <serialGLCD.h>

int mode = 0;
byte b = 0;
int c = 0;
char a;
String data1;
String data2;
String data3;
String data4;

HardwareSerial Uart = HardwareSerial();
serialGLCD lcd;

void setup() {

  Uart.begin(115200); // for display
  lcd.resetLCD();

  pinMode(9,OUTPUT); // serial power
  digitalWrite(9,0); // Serial Power

}

void loop() {

  // ____________________________________________________________Weight Gauge____________________________

weightGauge:

  lcd.clearLCD();
  NewSoftSerial two(27, 7);
  two.begin(19200);
  digitalWrite(9, 1);
  mode = 0;


  Uart.print("turn on gauge");  // wait to start device
  delay(5000);


  two.print("N"); // Type the capital letters NOI to start
  delay(10);

  two.print("O"); // Type the capital letters NOI to start
  delay(10);

  two.print("I"); // Type the capital letters NOI to start
  delay(10);


  while (mode < 5) // loop for data gather to lcd after device ready
  {
    c = 0;
    mode = mode+1;
    delay(2000);

    if (mode == 1)
      two.print("N"); // after data is sent device will respond

    if (mode == 2)
      two.print("N"); // after data is sent device will respond


    if (mode == 3)
      two.print("N"); // after data is sent device will respond

    if (mode == 4)      
      two.print("Y"); // after data is sent device will respond



    while(two.available() && c <= 160) // gather data for page 1
    { 
      a = two.read();
      data1 += a; 
      c++;
    }     
    while(two.available() && c <= 320) // gather data for page 2
    { 
      a = two.read();
      data2 += a; 
      c++;
    }
    while(two.available() && c <= 480) // gather data for page 3
    { 
      a = two.read();
      data3 += a;
      c++; 
    }
    while(two.available() && c <= 640) // gather data for page 4
    { 
      a = two.read();
      data4 += a; 
      c++;
    }

    Serial.print("data1 = "); 
    Serial.println(data1); // for trouble shooting
    Serial.print("data2 = "); 
    Serial.println(data2); // for trouble shooting
    Serial.print("data3 = "); 
    Serial.println(data3); // for trouble shooting
    Serial.print("data4 = "); 
    Serial.println(data4); // for trouble shooting

    //----------------------display serial data 1

    if (data1.length() > 0)
    {
      lcd.clearLCD();
      lcd.gotoPosition(0,0);         
      Uart.print(data1);
      delay(3000);
    }
    //----------------------display serial data 2
    if (data1.length() >= 160)
    {
      lcd.clearLCD();
      lcd.gotoPosition(0,0);
      Uart.print(data2);
      delay(3000);   
    } 

    //----------------------display serial data 3
    if (data2.length() >= 160)
    {
      lcd.clearLCD();
      lcd.gotoPosition(0,0);
      Uart.print(data3); 
      delay(3000);  
    } 

    //----------------------display serial data 4
    if (data3.length() >= 160)
    {
      lcd.clearLCD();
      lcd.gotoPosition(0,0);
      Uart.print(data4);
      delay(3000);
    } 
    //----------------------send serial data out

    data1 = ""; // clear string
    data2 = ""; // clear string
    data3 = ""; // clear string
    data4 = ""; // clear string

  }

}
HardwareSerial Uart = HardwareSerial();

There is already an instance of the HardwareSerial class, called Serial. Why are you creating a new one? This isn't even the proper way of doing it.

We seem to have gotten away from the thread topic. Is the point of this thread to illustrate how many ways you can do things wrong?

this was used by an example on PJRC website on how to use the teencyduino serial port. This drives the on board serial port which I use for the LCD.

http://www.pjrc.com/teensy/td_uart.html

I have no formal training in programming and do not do it at a professional level, so yes there will be a lot of mistakes...

here is a sample of return data where the return data goes funny at the longer string:

data1 =
Do you want to download updated firmware?

data2 =
data3 =
data4 =
data1 =
Do you want to select or change the operating mode of the unit?

data2 =
data3 =
data4 =
data1 =
Do you want to change the Dial Face of the unit?

data2 =
data3 =
data4 =
data1 = This utility allows a technican to describe the dial face to the meter.
The meter uses these values to determine the correct needle placement.
The OUTER set
data2 = of numbers on the dial is the MAIN scale (#1)
The INNER set of numbers on the dial is the DELTA, WOB or 2nd scale
The Main scale is always used in NORMAL MODE
data3 =
What is the second scale function? [Pick one letter from the list]
A - Normal mode w/ alternate calibration eg: #of lines or English/Metric
B - Del
data4 = m - Etrenivelveigmo
C eg OitdeEr seit rrsefti
Fmoreinman suthwns ual

instrumentek:

String data1;

String data2;
String data3;
String data4;

The String class is currently not safe to use; it can cause memory corruption leading to erratic behaviour. Since you're experiencing erratic behaviour, removing all use of String should be your first step.

One final question, before I bow out of this thread. Are you testing this code only on a non-Arduino? The Teensy is NOT an Arduino. It can be programmed using the Arduino IDE but there are differences in the hardware and the software serial handling.

If your problems are strictly limited to the Teensy, you should be on the prjc forum, not here.