I located the file inside the library of my project and changed the text a bit.
Turns out that for some reason the head tag does not "take" when I use it like this:
static const char serverIndex[] PROGMEM =
R"(<html><head><title>WiFi module firmware updater</title></head>
<body>Select firmware file, then click the Update button!
<form method='POST' action='' enctype='multipart/form-data'>
<input type='file' name='update'>
<input type='submit' value='Update'>
</form>
</body></html>)";
This displays two lines of text above the buttons, i.e. the supposed title is just printed inside the body of the document.
Strange....
But if I copy the text above into an html file on my disk and double-click it so it opens in FireFox then the title is displayed in the browser tab as expected.