fun fact: the webcrypto api does allow marking a key as non-exportable. So you could imagine deriving the key from the passphrase and having it only exist within the WebCrypto framework.
It is however going to be hard to prevent a malicious active attacker from stealing the passphrase.
It is however going to be hard to prevent a malicious active attacker from stealing the passphrase.