Running Gramine Vault in Raft Mode

[dzobbe]

Hello everyone,

Did you try to setup Gramine Vault in HA mode using the Raft storage type? I tried to do it but I am still unable to make it work. I also opened a discussion in Gramine's repo. Would be nice to know if you tried something similar. It would enrich this repo ;)

Thanks.
Regards.

[klassiker]

Hello @dzobbe,

we used the file storage because it was the only thing working without modification. Every other storage engine was unusable at the time this repo was initially created. So no, unfortunately we didn't check if the raft storage could work with newer gramine versions yet (given the right parameters).

It might be easier to use SEV-SNP instead of SGX for this workload depending on your requirements, it would increase stability if you are aiming for high availability. If you prefer SGX and are interested in integrating it with vault, we also have an (currently private) authentication plugin for vault to use SGX attestations as a login method.

Thank you for checking out our repository,
Kind Regards

[dzobbe]

Hello @klassiker,

Thanks for the reply. We want to use SGX for our requirements. The solution I thought for now is to adopt Vault HA using Consul. It is a bit more complicated. I hope Gramine's folks will help me in making Raft work. Regarding this login addon I'd be interested. Can you give more details? Do you have some docs?

[klassiker]

Getting consul running in SGX might be a problem too, but we haven't tried that yet. Looking at the storage engines again, some SQL backends could also do the trick. If I remember correctly, we only tested integrated storage engines. MySQL would probably work and we already have a working MariaDB SGX container, Postgres could also work, but at the time we tried enclaving it with gramine there was no EDMM amongst other things which made it very complicated.

Also, maybe Raft just needs a lot more resources, we've run applications with 256GB and more sgx.enclave_size back then I think (and that without EDMM).

Regarding the authentication plugin, it's a full setup with premains for vault and the applications to perform attestation and login, including a client to verify vault and build up an attested PKI. It provisions certificates for applications to transparently configure mTLS in a network of attested services. I've messaged the person responsible for repository and documentation access to provide it to you.

[dzobbe]

Actually I was thinking to run Consul externally to gramine and Vault inside gramine because from my understanding the encryption is made at Vault-level. I will wait for the doc, thanks.

[dzobbe]

Hi @klassiker how did you manage the "register" of vault plugin under Gramine?

Thanks

[klassiker]

Regarding encryption at vault level, you are right of course. Documentation will take some time, sorry for that, I couldn't find an up to date version that is ready to share.

External plugin registration under gramine can be a little bit tricky, apart from the standard vault plugin register -sha256="$HASH" auth plugin-name, vault will fork that plugin, doubling the memory requirements on the enclave. Without EDMM, you might run into trouble. If you need help with that, please open a new issue with a description of your problems.