How to read data received after sending a FIN?

My current project is inspired by https://www.instructables.com/id/Arduino-stoplight-web-server/. There is however one big difference: while the named source contains a web server on the Arduino through which the traffic light can be changed, I want to have the Arduino to fetch the status from the Xymon server.

Thus a single query is sent to the Xymon server and the response from the server is used to control the traffic light. The sequence of networking events in more detail is as follows. A TCP session to port 1984 on the Xymon server is build. The query is sent, the last packet has the PUSH flag set. Then the client stops (!) the session by sending a packet with the FIN flag set. The Xymon server responds with the answer to the query, again with the PUSH flag set in the last packet. Then a packet with the FIN flag set is sent to complete the termination of the TCP session. Thus the client sends a FIN to signal the end of the query.

Class EthernetClient has only one method to send a FIN, method stop(). After invoking method stop(), the response to the query is sent to the Arduino. This is confirmed by using tcpdump on the Xymon server. However, the response never reaches the traffic-light-program as both methods connected() and available() are zero once method stop() is invoked.

The environment is:

  • Xymon server on Raspberry Pi 3B+
  • Arduino Mega2560 with Ethernet shield W5100
  • Arduino IDE 1.8.12 on Linux Mint 19.3

How can one in this case retrieve the response?

The following hack in the Ethernet.+ modules does solve my problem for the time being:

wim@ticiv ~/Arduino/libraries/Ethernet $ git diff Ethernet.h
diff --git a/libraries/Ethernet/Ethernet.h b/libraries/Ethernet/Ethernet.h
index 376e6c5..7112a28 100644
--- a/libraries/Ethernet/Ethernet.h
+++ b/libraries/Ethernet/Ethernet.h
@@ -222,6 +222,7 @@ public:
        virtual int availableForWrite(void);
        virtual size_t write(uint8_t);
        virtual size_t write(const uint8_t *buf, size_t size);
+       virtual uint16_t writeEof();
        virtual int available();
        virtual int read();
        virtual int read(uint8_t *buf, size_t size);
wim@ticiv ~/Arduino/libraries/Ethernet $ git diff EthernetClient.cpp
diff --git a/libraries/Ethernet/EthernetClient.cpp b/libraries/Ethernet/EthernetClient.cpp
index e2406d7..bf70d6d 100644
--- a/libraries/Ethernet/EthernetClient.cpp
+++ b/libraries/Ethernet/EthernetClient.cpp
@@ -88,6 +88,31 @@ size_t EthernetClient::write(const uint8_t *buf, size_t size)
        return 0;
 }
 
+/*
+ * Method EthernetClient::writeEof sends a packet with the FIN flag set. This
+ * will be seen as an end-of-file at the peer. This closes the data stream from
+ * this host to the peer, but the data stream in the other direction is still
+ * open. Wait for the acknowledge, which might include additional data from the
+ * peer.
+ */
+uint16_t EthernetClient::writeEof() {
+  uint16_t sra;                                // Amount of unread data
+ // Signal end-of-file to peer.
+  Ethernet.socketDisconnect( sockindex );
+
+ // Wait for an acknowledge of the FIN packet. As a response is expected, the
+ // availability of received data is a good indicator that an ACK has been
+ // received. Wait for at most 0.5 seconds.
+  uint32_t start = millis();
+  while ( 1 ) {
+    delay( 1 );
+    sra= Ethernet.socketRecvAvailable(sockindex);
+    if ( sra            >   0 ) break;
+    if ( millis()-start > 500 ) break;
+  }  // while
+  return sra;
+}
+
 int EthernetClient::available()
 {
        if (sockindex >= MAX_SOCK_NUM) return 0;
@@ -156,8 +181,9 @@ uint8_t EthernetClient::connected()
        if (sockindex >= MAX_SOCK_NUM) return 0;
 
        uint8_t s = Ethernet.socketStatus(sockindex);
-       return !(s == SnSR::LISTEN || s == SnSR::CLOSED || s == SnSR::FIN_WAIT ||
-               (s == SnSR::CLOSE_WAIT && !available()));
+       return !(s == SnSR::LISTEN || s == SnSR::CLOSED ||
+               (s == SnSR::FIN_WAIT   && !available()) ||
+               (s == SnSR::CLOSE_WAIT && !available()) );
 }
 
 uint8_t EthernetClient::status()
wim@ticiv ~/Arduino/libraries/Ethernet $

This hack is rather specific to my use case: after sending a FIN, the Xymon server will respond within milliseconds with a response. Thus the new method EthernetClient::writeEof() waits for the response to arrive.

I hope that someone with a better understanding of Arduino, Ethernet and C++ is able to create a more generic solution for this use case, that is work for a short time with a half-duplex TCP session.