Initialized repository, and added Vault documentation moved from linux-files repo.
This commit is contained in:
231
README.md
Normal file
231
README.md
Normal file
@@ -0,0 +1,231 @@
|
|||||||
|
# HashiCorp Vault
|
||||||
|
|
||||||
|
The Vault is deployed as a small Ubuntu server (Proxmox VM) and is available at [https://vault.koszewscy.waw.pl](https://vault.koszewscy.waw.pl) from inside the Lazurowa network. This Vault instance is used as the primary source for storing secrets and is referred to as "Vault" or "main Vault" in the documentation.
|
||||||
|
|
||||||
|
The main Vault is automatically unsealed using Auto Unseal with the transit secrets engine in another Vault running on Proxmox Backup Server. The second Vault instance is referred to by the term "KMS Vault" in the documentation. It is available over HTTPS as [http://pbs.koszewscy.waw.pl:8200](http://pbs.koszewscy.waw.pl:8200) from inside the Lazurowa network.
|
||||||
|
|
||||||
|
Both instances are exposed to the network without using a reverse proxy. The TLS listener is bound to the private IP address of the host, and the HTTP listener is bound to the localhost interface only.
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```shell
|
||||||
|
curl -s https://apt.releases.hashicorp.com/gpg | gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
|
||||||
|
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(grep -oP '(?<=VERSION_CODENAME=).*' /etc/os-release || lsb_release -cs) main" | tee /etc/apt/sources.list.d/hashicorp.list
|
||||||
|
apt update && apt install vault
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Note:** The original HashiCorp-supplied commands will fail on Debian/Proxmox because `UBUNTU_CODENAME` is not defined on those systems. The above command uses `VERSION_CODENAME` from `/etc/os-release` or falls back to `lsb_release -cs` (which must be installed).
|
||||||
|
|
||||||
|
## Development Mode
|
||||||
|
|
||||||
|
Vault can be started in development mode for testing purposes using the following (recommended) command:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
vault server -dev -dev-no-store-token -dev-root-token-id="root-token" -dev-listen-address="127.0.0.1:8200"
|
||||||
|
```
|
||||||
|
|
||||||
|
- `-dev` - starts Vault in development mode
|
||||||
|
- `-dev-no-store-token` - prevents storing the root token on disk (`~/.vault-token`)
|
||||||
|
- `-dev-root-token-id="root-token"` - sets the root token to a known value for easy access
|
||||||
|
- `-dev-listen-address="127.0.0.1:8200"` - binds the Vault listener to localhost only, but can be changed to `0.0.0.0:8200` to allow external access
|
||||||
|
|
||||||
|
Then before running any Vault commands, set the following environment variables:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
export VAULT_ADDR='http://127.0.0.1:8200' VAULT_TOKEN='root-token'
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Warning:** Development mode is designed for experimentation only. Data is stored in-memory and will be lost when the Vault process is stopped.
|
||||||
|
|
||||||
|
## TLS Certificate
|
||||||
|
|
||||||
|
Use standard Lego ACME installation instructions available at [Let's Encrypt directory](../letsencrypt/README.md) for both the main and the KMS Vault servers.
|
||||||
|
|
||||||
|
Although Proxmox Backup Server requests and provisions a public TLS certificate for the **pbs.koszewscy.waw.pl** domain using Let's Encrypt, the certificate file is stored in a location not accessible to Vault. Therefore, a separate TLS certificate must be created for the Vault server.
|
||||||
|
|
||||||
|
Create an executable certificate hook script for both Vault servers at `/usr/local/bin/certificate-hook`:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
#!/usr/bin/bash
|
||||||
|
|
||||||
|
HOSTNAME=vault.koszewscy.waw.pl
|
||||||
|
install -o vault -g vault -m 0600 /etc/letsencrypt/certificates/${HOSTNAME}.crt /opt/vault/tls/tls.crt
|
||||||
|
install -o vault -g vault -m 0600 /etc/letsencrypt/certificates/${HOSTNAME}.key /opt/vault/tls/tls.key
|
||||||
|
systemctl restart vault
|
||||||
|
```
|
||||||
|
|
||||||
|
The script will be run after each successful certificate renewal. You will have to unseal the KMS Vault manually after each restart. You can add an email notification to the certificate renewal process to remind you about unsealing the KMS Vault.
|
||||||
|
|
||||||
|
## Auto Unseal Configuration
|
||||||
|
|
||||||
|
The default Vault unseal method uses Shamir's Secret Sharing, which requires manual entry of unseal keys after each Vault restart. To avoid this, the Auto Unseal feature is used, which leverages the transit secrets engine of another Vault instance (the KMS Vault).
|
||||||
|
|
||||||
|
### KMS Vault Setup
|
||||||
|
|
||||||
|
1. Enable the transit secrets engine:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
vault secrets enable transit
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Create a new encryption key:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
vault write -f transit/keys/transit-unseal
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Create a policy allowing unseal operations as a file named `transit-unseal-policy.hcl`:
|
||||||
|
|
||||||
|
```hcl
|
||||||
|
path "transit/decrypt/unseal-key" {
|
||||||
|
capabilities = ["update"]
|
||||||
|
}
|
||||||
|
|
||||||
|
path "transit/encrypt/unseal-key" {
|
||||||
|
capabilities = ["update"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Apply the policy:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
vault policy write transit-unseal-policy transit-unseal-policy.hcl
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Create a token with the policy attached:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
vault token create -policy="transit-unseal-policy"
|
||||||
|
```
|
||||||
|
|
||||||
|
Save the generated token for later use.
|
||||||
|
|
||||||
|
6. Verify connectivity from the main Vault to the KMS Vault.
|
||||||
|
|
||||||
|
7. Store the KMS Vault unseal key and root token securely. Make an offline backup of the KMS Vault.
|
||||||
|
|
||||||
|
### Enabling audit logging (optional)
|
||||||
|
|
||||||
|
To enhance security and monitoring, enable audit logging on the KMS Vault:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
mkdir -p /var/log/vault
|
||||||
|
chown -R vault:vault /var/log/vault
|
||||||
|
chmod 750 /var/log/vault
|
||||||
|
vault audit enable file file_path=/var/log/vault/audit.log
|
||||||
|
```
|
||||||
|
|
||||||
|
Then monitor the audit log online:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
tail -F /var/log/vault/audit.log | jq -r '. | select(.type == "response") | [ .time, .request.path, .request.operation, .request.remote_address] | @tsv'
|
||||||
|
```
|
||||||
|
|
||||||
|
or offline:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
jq -r '. | select(.type == "response") | [ .time, .request.path, .request.operation, .request.remote_address] | @tsv' /var/log/vault/audit.log | column -t -N "time,path,operation,remote_addr"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Main Vault Configuration
|
||||||
|
|
||||||
|
Depending on main Vault state (new or existing), some of the following steps are mutually exclusive.
|
||||||
|
|
||||||
|
1. If the main Vault is already initialized, shut it down and back up its data directory and configuration file.
|
||||||
|
|
||||||
|
2. Update the main Vault configuration file (usually located at `/etc/vault.d/vault.hcl`) to include the Auto Unseal configuration:
|
||||||
|
|
||||||
|
```hcl
|
||||||
|
seal "transit" {
|
||||||
|
address = "https://pbs.koszewscy.waw.pl:8200"
|
||||||
|
key_name = "transit-unseal"
|
||||||
|
mount_path = "transit/"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
3. Put the KMS Vault token created earlier into the environment file `/etc/vault.d/vault.env`:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
VAULT_TOKEN="s.xxxxxxx"
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Ensure that the Vault service systemd unit file includes the environment file. If not, create an override file at `/etc/systemd/system/vault.service.d/override.conf`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Service]
|
||||||
|
EnvironmentFile=/etc/vault.d/vault.env
|
||||||
|
```
|
||||||
|
|
||||||
|
5. If the main Vault was already initialized, start it and unseal with `-migrate` parameter.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
systemctl start vault
|
||||||
|
vault operator unseal -migrate
|
||||||
|
```
|
||||||
|
|
||||||
|
6. Verify that the main Vault is unsealed and operational. The Seal Type should show `transit`, and Recovery Seal Type should show `shamir`.
|
||||||
|
|
||||||
|
7. Uninitialized main Vault will automatically encrypt the root key with the transit key from the KMS Vault during initialization and present recovery keys for Shamir's Secret Sharing.
|
||||||
|
|
||||||
|
## Offline Backup
|
||||||
|
|
||||||
|
Vault installs the following directories:
|
||||||
|
|
||||||
|
- `/etc/vault.d` - configuration files
|
||||||
|
- `/opt/vault/data` - default file storage backend
|
||||||
|
- `/opt/vault/tls` - TLS certificates
|
||||||
|
|
||||||
|
The simplest method of backing up Vault is to stop the Vault service, back up the above directories, and then start the service again. This can be automated using systemd service and timer units.
|
||||||
|
|
||||||
|
Systemd service unit `/etc/systemd/system/backup-vault.service`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Make an off-line backup of HashiCorp Vault
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
EnvironmentFile=/etc/proxmox-backup/pbc-vault.env
|
||||||
|
ExecStartPre=/usr/bin/systemctl stop vault.service
|
||||||
|
ExecStart=/usr/bin/proxmox-backup-client backup --backup-id "vault" --backup-type host config.pxar:/etc/vault.d data.pxar:/opt/vault/data tls.pxar:/opt/vault/tls
|
||||||
|
ExecStartPost=/usr/bin/systemctl start vault.service
|
||||||
|
RemainAfterExit=no
|
||||||
|
Type=oneshot
|
||||||
|
```
|
||||||
|
|
||||||
|
Systemd timer unit `/etc/systemd/system/backup-vault.timer`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Make an off-line backup of HashiCorp Vault (timer)
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
Persistent=true
|
||||||
|
OnCalendar=*-*-* 2:40
|
||||||
|
RandomizedDelaySec=1h
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
```
|
||||||
|
|
||||||
|
The KMS Vault is protected by regular unseal key(s) and therefore cannot be backed up automatically in the same way. Use the script below to create a backup of the KMS Vault and start it up afterwards.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
#!/usr/bin/bash
|
||||||
|
|
||||||
|
PBC_ENV="/etc/proxmox-backup/pbc-vault.env"
|
||||||
|
if [[ -f $PBC_ENV ]]; then
|
||||||
|
# Import variables and export them for the proxmox-backup-client usage
|
||||||
|
export $(grep -v ^# $PBC_ENV)
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Stop Vault service
|
||||||
|
systemctl stop vault
|
||||||
|
|
||||||
|
# Create a backup
|
||||||
|
proxmox-backup-client backup --backup-id "vault-kms" --backup-type host confg.pxar:/etc/vault.d data.pxar:/opt/vault/data tls.pxar:/opt/vault/tls
|
||||||
|
|
||||||
|
# Start Vault service
|
||||||
|
systemctl start vault
|
||||||
|
|
||||||
|
# Unseal the Vault
|
||||||
|
VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user