Note that the TOTP codes are stored on the key, not in the Authenticator. Yubikey 5 series keys with the latest firmware (non-upgradable) can hold only 64 TOTP credentials. Keys with an older firmware are limited to 32 TOTP credentials. You may find that number to be too limiting.
Personally, the added friction of having to insert the key for TOTP is not worth the security gain. A software-based TOTP Authenticator like Ente Auth would be my choice.