Saving sensor logs of unknown length, conflict between SD card and WiFi shield

I have a sensor giving me values (ints) which I want to keep, and send to a computer via the WiFi shield when requested via HTTP. I have the HTTP thing working, however, I do not know how to have a String or whatever able to store what is basically an array of ints, when I do not know any maximum size.

I have read that using Strings on Arduino is not recommended because how RAM works and such things, so I translated all my String uses to C-style arrays (as it was recommended), this solved my memory issues, however, C-style strings are not resizable.

While browsing the forums, I found that for similar problems the recommended way of handling things would be to create an array of max size instead of dynamically growing the array, but I do not know the maximum size in this case.

Here is what I exactly want :

  • A way of storing each values (in order) so they can be sent later after an HTTP request (but they can get requested multiple times so I can't delete them then),
  • Whatever storage way is fine by me (Strings, arrays of ints, ...), as long as it works.

I thought about printing the values to the SD card, since there I have far enough memory, but for some reason I can't get the WiFi and the SD card to work together (as soon as I init the SD library, the WiFi library doesn't work anymore). I'm aware this is longer (as it's a SD card instead of RAM), but I have around 100ms per request so there should be no problems there.

The shield is up-to-date (WiFi.firmwareVersion() -> 1.1.0).

PS.
I'm quite new to C++ and Arduino, but I'm not too bad at Java.

Here are the codes;

void setup(){
  //stuff
  newFile();
  beginWiFi();
}

void endProgram(){
  Serial.println(F("Order has been made to terminate."));
  while(true);
}

void beginWiFi(){                    //works perfectly if called before newFile();
  const char ssid[] = "MiniStepper";            // Nom du réseau
  const char pass[] = "Test1234";             // Code ou clef
  const int keyIndex = 0;                       // Network key
  int statut = WL_IDLE_STATUS;
  
  // S'il n'y a pas de WiFi shield
  if(WiFi.status() == WL_NO_SHIELD){
    Serial.println(F("The WiFi shield can't be found ..."));
    endProgram();
  }

  // Working stuff
  
  // Connexion ...
  int essai = 1;
  while(statut != WL_CONNECTED){
    Serial.print(F("Trying to connect to network : try #"));
    Serial.println(String(essai));
    switch(cas){
      case WIFI_NO_ENCRYPTION:
        statut = WiFi.begin(ssid);
        break;
      case WIFI_WPA:
        statut = WiFi.begin(ssid, pass);
        break;
      case WIFI_WEP:
        statut = WiFi.begin(ssid, keyIndex, pass);
        break;
      default:
        Serial.println(F("Impossible connexion type ..."));
        endProgram();
    }

    // Attendre de voir si c'est réussi (10 secondes)
    delay(10000);
    essai++;
  }
  Serial.println(F("Connexion successful !"));
  
  //stuff

  delay(100);
  // Creating HTTP server
  Serial.println(F("Creating HTTP server ..."));
  server.begin();
  Serial.println(F("HTTP server created"));
}

char filename[21];      // content/chanXXXX.txt
                        // ^      ^    ^   ^   ^
                        // 0      0    1   1   2
                        // 0      7    2   6   0

void newFile(){                     //Setup of the SD card, works if called before beginWiFi();

  pinMode(10, OUTPUT);   //tried this to solve the conflict,
  digitalWrite(10, HIGH);   //doesn't change anything

  Serial.println(F("Initializing SD card ..."));
  if(!SD.begin(6)){
    Serial.println(F("Couldn't initialize SD card, stopping ..."));
    endProgram();
  }
  Serial.println(F("SD card ready !"));

  // searching an available filename looking like chanXXXX.txt where the X are digits (this name is 8 characters long with a 3 chars extension, so it is valid)
  
  Serial.print(F("Saving data into : ["));
  Serial.print(filename);
  Serial.println(F("]"));

  Serial.println(F("Creating current file ..."));
  File current = SD.open(filename, FILE_WRITE);

  if(!current){Serial.println(F("Couldn't open file...")); endProgram();}
  
  //writing in file
  current.println(F("VERSION 1\nSERIES_NUMBER 1 Stepper\n"));
  current.println(F("NAMES Steps:Time"));
  current.println(F("UNITS Unit:ms\n"));
  current.close();
  Serial.println(F("File created and pre-filled."));
}

and the output, when newFile(); is called :

Booting up ...
Initializing SD card ...
SD card ready !
// searching for filename
Saving data into : [content/chan0014.txt]
Creating current file ...
File created and pre-filled.
The WiFi shield can't be found ...
Order has been made to terminate.

and when the newFile(); function is not called AND the SD card is not plugged in :

Booting up ...
WiFi firmware version : 1.1.0
Targetting network : MiniStepper
Targetting encryption : WPA
Trying to connect to network : try #1
Connexion successful !
Creating HTTP server ...
HTTP server created
Ready ...

(if newFile() is not called, but the SD card is plugged in, the WiFi library doesn't work either, however I read somewhere that this was intended and there was a pin to set to HIGH to solve this, I'm not going to do it as it has no use to me, either I use the SD library, either there's no SD card plugged, not both)

This runs on Arduino Uno + WiFi shield, both purchased a few months ago. I updated the WiFi shield.

(The main goal is to send the data to StatsReader (on my blog))

Finally, my question is ; what is the best (or at least a working one) way of achieving this, and if it is using the SD card, how can I solve my conflict ? It's probably obvious, but as a beginner I don't see it.

however, I do not know how to have a String or whatever able to store what is basically an array of ints, when I do not know any maximum size.

A String would be the stupidest possible way to store them. You have a limited amount of memory to store the data in. You need to use that as efficiently as possible.

You need to decide/understand just mow much data you can save between requests, and simply allocate an array of that size.

Shouldn't your sketch have some other stuff, like including the WiFi and SD libraries? And maybe a loop() function?

Is the SD socket on the WiFi card? Doesn't that SD card socket use Pin 4 for Slave Select? You seem to be specifying Pin 6. If you are using an add-on SD module, many of them cannot share the SPI bus with other devices because their MISO line is active even when Slave Select is LOW.