Why do both x and y end up being equal to 42 when the only information in the x1.txt file is 420 and 210 in the y1.txt file?
I can't see your file, so I can't answer that. What I suspect, though, is that both files have the same last character, whatever that character is (asciitable.com says that 42 is a '*', so look for that in your files).
How can I set my variables, x and y, to be equal to one entire line instead of one character?
x and y have to be char arrays. You need to write to the next element of the array each time you read a character, and append a NULL after that character. Then, use atof() to convert the array contents to a float value.
I imagine it can be done with a for loop instead of a while loop.
Actually, it needs to be done with two while loops - one reading the whole file, and the other reading each character, ending when it finds an end of record marker. Process the data read after the inner while loop ends.
Does the SD.remove("unwanted.txt") function for deleting files work in a similar way so that it only deletes one character at a time or does it simply delete the entire file at once?
It deletes the file.
If it does it all at once, how can I overwrite one line or delete one line at a time when the number of entries gets to a desired level?
That depends on whether the file has fixed length records, or not. Writing the nth record of a file when the records are all the same length is a matter of positioning to the start of the nth record, based on the length of the preceding n-1 records (all the same) and writing the new record.
For variable length records, you need to open the file that contains the data to keep, read the part to keep, writing it to another file, then closing the first file. Then, you write the new data to the second file, and close it. Delete the first file, and rename the second file.
By now, I'm sure you see the advantage of fixed length records.