If your system supports secure boot and also allows you to use custom signing certificates and to disable the default certificates, you can disable the usual ways that would be used for password reset (or for any other kind of access).
This is an ‘advanced’ task. This repository seems to have some useful information about the process: M-P-P-C/Signing-an-Ubuntu-Kernel-for-Secure-Boot: A step-by-step guide on how to install and sign a linux kernel to boot with Secure Boot, because it shouldn’t be so hard to have the latest drivers for your machine.
I believe this answers your question as you scoped it, but note that with physical access this isn’t a very strong solution which can be bypassed in lots of ways (as you mentioned, for example, by accessing the disk directly).
ETA: This process might require that you compile your own kernel, or at least that you won’t be able to update the kernel without it being signed again. If you do this, you probably also want to disable Grub as well and boot Linux directly from UEFI. See for example systemd-boot - ArchWiki.
See also (pretty detailed, but it’ll be a bit different on Mint as this is Gentoo-specific):
ETA 2: On second thought, the second part of @SteveR’s answer might just be sufficient and you may not need to bother with secure boot if you can restrict booting from all external media (and physical access isn’t a concern).
The reason secure boot came to mind is that it does almost exactly what you asked for (anything that boots needs to be properly signed), which would prevent your machine from being booted from USB, CD/DVD, floppy, network, external drive, etc. and internal drives. However, I now think this is overkill without other protections like FDE.
I’d still just in case try to boot directly without Grub (or alternatively, as well as removing the ‘advanced options’, see menu - Restricting on-the-fly editing of grub2 menuentries - Ask Ubuntu). The reason for this is that it’s too easy to edit boot entries to gain access. If you disable editing or don’t use something with editing capabilities to begin with, this is no longer a concern.