On a newly created project, opening a C# script via double clicking it won't correctly load VSCode's extensions. (this is due to parts of the sln file not being generated.)
GoingCreate at least one C# script, then go to 'Assets' > 'Open C# Project' generateswill generate the correct sln file and after a quick reload of VSCode, intellisense starts working.
After doing this one time, intellisense will continue to work as expected, double clicking C# scripts etc.