VS Code by itself is essentially just a text editor. You can use it to open, create, modify, and save the content of text files.
So the VS Code base application does not have a built-in sketch verification and upload capabilities. The application doesn't know or care anything whatsoever about Arduino.
You can use the VS Code base application to edit Arduino sketch code files though.
VS Code + Arduino IDE
You could leave it there and continue to use Arduino IDE to compile and upload the Arduino sketches you wrote using VS Code. In fact, some community members do exactly this, and Arduino IDE has always been designed to support the use of an external editor in order to accommodate people who don't like Arduino IDE's built-in editor.
VS Code + Arduino CLI
Another option would be to use VS Code in combination with Arduino CLI instead of the combination of VS Code and Arduino IDE. You would use Arduino CLI from the command line to perform verifications (arduino-cli compile) and uploads (arduino-cli upload). VS Code has an integrated Terminal, which can be used to run command line commands from a panel inside the VS Code window. This can be quite convenient for a use case like this where you are frequently switching back and forth between using a text editor and a terminal.
Complete Embedded Systems IDE
However, you also have the option of using VS Code as a complete IDE for embedded systems projects, equivalent to Arduino IDE.
PlatformIO IDE
You already are aware of the "PlatformIO IDE" extension, which makes VS Code into an IDE for embedded systems projects based on the PlatformIO framework. The creators of PlatformIO have made some efforts to implement the Arduino framework. However, there is not 100% compatibility, so you would find that some work is often needed to use the sketch projects that were already working fine with Arduino IDE.
Arduino Maker Workshop
Another option is the "Arduino Maker Workshop" extension. This is intended to make VS Code into something equivalent to Arduino IDE, with full compatibility for standard Arduino sketch projects.
Arduino Community Edition
Microsoft created a "Arduino" VS Code extension to provide a full Arduino IDE equivalent. It was quite popular for years, but Microsoft eventually stopped maintaining it, and then eventually formally retired the project. Community members created the "Arduino Community Edition" extension as a fork of Microsoft's project in order to continue the maintenance of the extension.
Unfortunately, after the initial burst of work following the retirement of Microsoft's project, there hasn't been much activity from the maintainers of the Arduino Community Edition extension. I'm sure the extension is perfectly fine in its current state, but this does not bode well for the future of the project. So it might be a good idea to look at the other more actively maintained extensions first.
BoardLab
"BoardLab" is another extension that makes VS Code into a full Arduino IDE equivalent. It is a very young project still in the alpha development phase. However, it is quite notable because it was created by @dankeboy36, who is the same person who brought us the "ESP Exception Decoder" extension for Arduino IDE 2.x, as well as other very significant and high quality contributions to the Arduino ecosystem.