Refactored scripts to focues on compatibility with the real product.

This commit is contained in:
2026-02-28 09:34:52 +01:00
parent c65c347ca5
commit ae7542100b
10 changed files with 135 additions and 246 deletions

View File

@@ -3,15 +3,30 @@
# Replace "__ACCOUNT_NAME__" with a desired storage account name if needed
__ACCOUNT_NAME__.blob.core.windows.net {
tls __AZURITE_DIR__/__ACCOUNT_NAME___cert.pem __AZURITE_DIR__/__ACCOUNT_NAME___key.pem
reverse_proxy 127.0.0.1:10000
reverse_proxy https://__ACCOUNT_NAME__.blob.core.windows.net:10000 {
transport http {
tls
tls_trust_pool file __AZURITE_DIR__/ca_cert.pem
}
}
}
__ACCOUNT_NAME__.queue.core.windows.net {
tls __AZURITE_DIR__/__ACCOUNT_NAME___cert.pem __AZURITE_DIR__/__ACCOUNT_NAME___key.pem
reverse_proxy 127.0.0.1:10001
reverse_proxy https://__ACCOUNT_NAME__.queue.core.windows.net:10001 {
transport http {
tls
tls_trust_pool file __AZURITE_DIR__/ca_cert.pem
}
}
}
__ACCOUNT_NAME__.table.core.windows.net {
tls __AZURITE_DIR__/__ACCOUNT_NAME___cert.pem __AZURITE_DIR__/__ACCOUNT_NAME___key.pem
reverse_proxy 127.0.0.1:10002
reverse_proxy https://__ACCOUNT_NAME__.table.core.windows.net:10002 {
transport http {
tls
tls_trust_pool file __AZURITE_DIR__/ca_cert.pem
}
}
}

View File

@@ -2,13 +2,25 @@
This is a simple guide that will help you set up and run the Azure Storage Emulator on your local machine. The Azure Storage Emulator allows you to develop and test your applications that use Azure Storage services without needing an actual Azure subscription nor Internet connection.
## Prerequisites
Review the [Documentation](#reference) for more details on how to use the emulator and its features. This document will cover a scenario where you want to run the emulator as close to the real Azure Storage service as possible, which means using triple HTTPS endpoints and OAuth simulation.
- Any system that supports fairly recent versions of Node.js (LTS vesions are recommended).
- A Bash shell (a Linux or macOS system or Windows with WSL2 installed).
- Docker Community Edition or Docker Desktop (for running the emulator in a containerized environment).
- Apple `container` command (for running the containerized version of the emulator on macOS).
- (Optional) Caddy HTTP server (for running the emulator with triple HTTPS endpoints, as it would be in Azure).
## Docker Installation
To run the Azure Storage Emulator in a Docker container, follow these steps:
1. Ensure that a container runtime is installed. This repository supports both Docker and Apple `container` command.
2. Build the emulator image using the provided Dockerfile:
```bash
./build.sh
```
3. Run the emulator container:
```bash
./run.sh
```
## Native Installation
@@ -47,37 +59,43 @@ To install the Azure Storage Emulator natively on your machine, follow these ste
rm -rf azurite
```
## Running the Emulator
6. Install the Caddy HTTP server.
There are two ways to run the Azure Storage Emulator: as a development server exposing the API on localhost using high ports (10000-10002) or as a fake Azure Storage service. The first one is easy, just run the `azurite` command in your terminal and that's it. The second one is a bit more complex, but it allows you to use the emulator as if it were the real Azure Storage service, and connect to it applications that do not support custom endpoints (Terraform azurerm state storage backend for example).
7. Create an `accounts.env` file in the same directory as the `run-server.sh` script with the following content:
Run the `make-certs.sh` script to generate a self-signed CA certificate and a server certificate signed by that CA. The server certificate will be used by the emulator to serve HTTPS traffic, and the CA certificate can be imported into your system's trusted root certificates store to avoid security warnings when connecting to the emulator.
```bash
AZURITE_ACCOUNTS=accountname:accountkey
```
The `make-certs.sh` script will create a storage directory for the emulator and place the generated PEM files there. The script will create a certificate for the `devstoreaccount1.blob.core.windows.net`. You can change that by adding you own storage account name as a single positional parameter. You will also need to map that name to `127.0.0.1` in your hosts file.
Replace `accountname` with the desired account name. Use OpenSSL to generate an account key.
```bash
./make-certs.sh myaccount
```
```bash
openssl rand -base64 32
```
```
127.0.0.1 devstoreaccount1.blob.core.windows.net devstoreaccount1.queue.core.windows.net devstoreaccount1.table.core.windows.net
```
You can also generate a deterministic account key using any string as a seed:
Next, create an empty `accounts.env` file in the same directory. This file will be used by the emulator to retrieve the storage accounts names and access keys. You can use a single key or a pair of keys for each account. The format of the file is as follows:
```bash
echo -n "your-seed-string" | base64
```
```shell
AZURITE_ACCOUNTS="account1:key1,key2;account2:key1,key2;..."
```
8. Add the following line to your `/etc/hosts` file to map the custom domain names to localhost:
Use `add-account.sh` script to add at least one account to the `accounts.env` file. The script will generate random key for each added account. You can use `list-accounts.sh` script to list all accounts and their keys. You can also specify a password that will be used as the value for the key. The key is always Base64 encoded, and you password will also be Base64 encoded before being stored in the `accounts.env` file. The benefit of using a password is that the account key will be deterministic, so you can easily recreate the same account with the same key if needed.
```
127.0.0.1 <accountname>.blob.core.windows.net <accountname>.queue.core.windows.net <accountname>.table.core.windows.net
```
Finally, run the emulator using the `run-emulator.sh` script.
9. Run the server:
You can specify `--oauth` to enable simulation of Entra ID Authentication. It requires the endpoints to be directly accessed via HTTPS.
```bash
./run-server.sh
```
The `run-emulator.sh` will detect if Caddy HTTP server is installed on your system, and if it is, it will use it to serve the emulator on triple HTTPS endpoints (Blob, Queue and Table services) as it would be in Azure.
You can add the `--oauth` or `-o` flag to simulate OAuth authentication.
Use `--no-caddy` to disable Caddy even if it is installed. It is required to test Entra ID Authentication simulation.
```bash
./run-server.sh --oauth
```
## Reference

View File

@@ -1,27 +0,0 @@
#!/usr/bin/env bash
if [[ -z "$1" ]]; then
echo "Usage: $0 <account-name> [ password ]"
exit 1
fi
if [[ -z "$2" ]]; then
PASSWORD=$(openssl rand -base64 32)
else
PASSWORD=$(echo -n "$2" | base64)
fi
AZURITE_DIR="storage"
mkdir -p "$AZURITE_DIR"
AZURITE_ACCOUNTS_FILE="$AZURITE_DIR/accounts.env"
if [[ -f "$AZURITE_ACCOUNTS_FILE" ]]; then
. "$AZURITE_ACCOUNTS_FILE"
STORAGE_ACCOUNTS=($(echo "$AZURITE_ACCOUNTS" | tr ';' ' '))
else
# No accounts file, start with an empty array
STORAGE_ACCOUNTS=()
fi
STORAGE_ACCOUNTS+=("$1:$PASSWORD")
printf 'AZURITE_ACCOUNTS="%s"\n' $(IFS=';'; echo "${STORAGE_ACCOUNTS[*]}") > "$AZURITE_ACCOUNTS_FILE"

View File

@@ -46,7 +46,7 @@ function make_server_cert() {
-addext "basicConstraints=critical,CA:FALSE" \
-addext "keyUsage=digitalSignature,keyEncipherment" \
-addext "extendedKeyUsage=serverAuth,clientAuth" \
-addext "subjectAltName=DNS:${ACCOUNT_NAME}.blob.core.windows.net,DNS:${ACCOUNT_NAME}.queue.core.windows.net,DNS:${ACCOUNT_NAME}.table.core.windows.net,DNS:localhost,IP:127.0.0.1" \
-addext "subjectAltName=DNS:${ACCOUNT_NAME}.blob.core.windows.net,DNS:${ACCOUNT_NAME}.queue.core.windows.net,DNS:${ACCOUNT_NAME}.table.core.windows.net,DNS:${ACCOUNT_NAME}.blob.localhost,DNS:${ACCOUNT_NAME}.queue.localhost,DNS:${ACCOUNT_NAME}.table.localhost,DNS:localhost,IP:127.0.0.1" \
| openssl x509 \
-req \
-CA "$CERT_DIR/ca_cert.pem" \

View File

@@ -19,15 +19,12 @@ ACCOUNT_NAME=$(echo "$AZURITE_ACCOUNTS" | cut -f 1 -d ';' | cut -f 1 -d ':')
# Ensure /etc/hosts contains an entry the Azure endpoint names,
# so Caddy can route requests to the correct service based on the hostname.
if ! grep -qE "${ACCOUNT_NAME}\.blob\.core\.windows\.net" /etc/hosts ||
! grep -qE "${ACCOUNT_NAME}\.queue\.core\.windows\.net" /etc/hosts ||
! grep -qE "${ACCOUNT_NAME}\.table\.core\.windows\.net" /etc/hosts; then
echo -e "\n127.0.0.1\t${ACCOUNT_NAME}.blob.core.windows.net ${ACCOUNT_NAME}.queue.core.windows.net ${ACCOUNT_NAME}.table.core.windows.net" >> /etc/hosts
fi
sed -i -E "/${ACCOUNT_NAME}/d" /etc/hosts
cat <<EOF >> /etc/hosts
127.0.0.1 ${ACCOUNT_NAME}.blob.core.windows.net ${ACCOUNT_NAME}.queue.core.windows.net ${ACCOUNT_NAME}.table.core.windows.net ${ACCOUNT_NAME}.blob.localhost ${ACCOUNT_NAME}.queue.localhost ${ACCOUNT_NAME}.table.localhost
EOF
CADDY=true
AZURITE_SSL=""
CERT_ARGS=()
OAUTH_ARGS=()
while [[ $# -gt 0 ]]; do
@@ -35,19 +32,6 @@ while [[ $# -gt 0 ]]; do
--oauth)
OAUTH_ARGS=("--oauth" "basic")
# Ensure Caddy is disabled when using OAuth, as Azurite does not support OAuth behind a reverse proxy.
CADDY=""
AZURITE_SSL=true
shift
;;
--ssl)
AZURITE_SSL=true
# Ensure Caddy is disabled when using Azurite's built-in SSL support.
CADDY=""
shift
;;
--no-caddy)
# Disable Caddy, but do not enable SSL for Azurite.
CADDY=""
shift
;;
*)
@@ -58,36 +42,27 @@ while [[ $# -gt 0 ]]; do
done
# Ensure certificates are generated before starting Azurite or Caddy.
if [[ -n "$AZURITE_SSL" || -n "$CADDY" ]]; then
if ! make_ca "$AZURITE_DIR"; then
echo "Error: Failed to create CA certificate and key." >&2
exit 1
fi
if ! make_server_cert "$ACCOUNT_NAME" "$AZURITE_DIR"; then
echo "Error: Failed to create server certificate and key." >&2
exit 1
fi
if ! make_ca "$AZURITE_DIR"; then
echo "Error: Failed to create CA certificate and key." >&2
exit 1
fi
if [[ -n "$CADDY" ]]; then
# If using Caddy, it will reverse proxy blob, queue, and table endpoints to different ports on localhost,
# allowing simultaneous access to all services on a single HTTPS port.
# Generate a Caddyfile configuration based on the account name and storage directory.
sed -E "s/__ACCOUNT_NAME__/${ACCOUNT_NAME}/g; s|__AZURITE_DIR__|${AZURITE_DIR}|g" /app/Caddyfile.example > "$AZURITE_DIR/Caddyfile"
echo "Starting Caddy server..."
caddy start --config "$AZURITE_DIR/Caddyfile" # Use start not run, start does not block the shell process.
else
# If not using Caddy, configure Azurite to listen on all interfaces and use the generated self-signed certificate.
# This mode does not require Caddy, but it will not allow simultaneous access to the table and queue services on the same HTTPS port.
if [[ -n "$AZURITE_SSL" ]]; then
CERT_ARGS=("--key" "$AZURITE_DIR/${ACCOUNT_NAME}_key.pem" "--cert" "$AZURITE_DIR/${ACCOUNT_NAME}_cert.pem")
fi
if ! make_server_cert "$ACCOUNT_NAME" "$AZURITE_DIR"; then
echo "Error: Failed to create server certificate and key." >&2
exit 1
fi
# Generate a Caddyfile configuration based on the account name and storage directory.
sed -E "s/__ACCOUNT_NAME__/${ACCOUNT_NAME}/g; s|__AZURITE_DIR__|${AZURITE_DIR}|g" /app/Caddyfile.example > "$AZURITE_DIR/Caddyfile"
caddy start --config "$AZURITE_DIR/Caddyfile" # Use start not run, start does not block the shell process.
HOST_ARGS=("--blobHost" "127.0.0.1" "--queueHost" "127.0.0.1" "--tableHost" "127.0.0.1")
PORT_ARGS=("--blobPort" "10000" "--queuePort" "10001" "--tablePort" "10002")
# Start Azurite with the appropriate arguments based on the configuration.
exec node /app/azurite/src/azurite.js \
--disableTelemetry \
--location "$AZURITE_DIR" \
--blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 \
"${CERT_ARGS[@]}" "${OAUTH_ARGS[@]}"
--key "$AZURITE_DIR/${ACCOUNT_NAME}_key.pem" --cert "$AZURITE_DIR/${ACCOUNT_NAME}_cert.pem" \
"${HOST_ARGS[@]}" "${PORT_ARGS[@]}" "${OAUTH_ARGS[@]}"

View File

@@ -1,51 +0,0 @@
#!/usr/bin/env bash
CERT_DIR="./storage"
mkdir -p "$CERT_DIR"
ACCOUNT_NAME="${1:-devstoreaccount1}"
# Generate CA certificate and key if they don't exist
if [[ ! -f "$CERT_DIR/ca_cert.pem" || ! -f "$CERT_DIR/ca_key.pem" ]]; then
echo "Generating CA certificate and key..."
if ! openssl req \
-x509 \
-newkey rsa:4096 \
-keyout "$CERT_DIR/ca_key.pem" \
-out "$CERT_DIR/ca_cert.pem" \
-days 3650 \
-nodes \
-subj "/CN=Azurite CA" \
-text \
-addext "basicConstraints=critical,CA:TRUE,pathlen:0"; then
echo "Error: Failed to generate CA certificate and key." >&2
exit 1
fi
fi
# Generate server certificate and key if they don't exist
if [[ ! -f "$CERT_DIR/server_cert.pem" || ! -f "$CERT_DIR/server_key.pem" ]]; then
echo "Generating server certificate and key..."
if ! openssl req \
-newkey rsa:4096 \
-keyout "$CERT_DIR/server_key.pem" \
-nodes \
-subj "/CN=${ACCOUNT_NAME}.blob.core.windows.net" \
-addext "basicConstraints=critical,CA:FALSE" \
-addext "keyUsage=digitalSignature,keyEncipherment" \
-addext "extendedKeyUsage=serverAuth,clientAuth" \
-addext "subjectAltName=DNS:${ACCOUNT_NAME}.blob.core.windows.net,DNS:${ACCOUNT_NAME}.queue.core.windows.net,DNS:${ACCOUNT_NAME}.table.core.windows.net,DNS:localhost,IP:127.0.0.1" \
| openssl x509 \
-req \
-CA "$CERT_DIR/ca_cert.pem" \
-CAkey "$CERT_DIR/ca_key.pem" \
-set_serial "0x$(openssl rand -hex 16)" \
-copy_extensions copyall \
-days 365 \
-text \
-out "$CERT_DIR/server_cert.pem"; then
echo "Error: Failed to generate server certificate and key." >&2
exit 1
fi
fi

View File

@@ -11,7 +11,7 @@ if [[ ! -d "$AZURITE_DIR" ]]; then
exit 0
fi
if [[ -f "$SCRIPT_DIR/accounts.env" ]]; then
if [[ -z "$AZURITE_ACCOUNTS" && -f "$SCRIPT_DIR/accounts.env" ]]; then
set -a
. "$SCRIPT_DIR/accounts.env"
set +a
@@ -31,54 +31,22 @@ if ! grep -qE "${ACCOUNT_NAME}\.blob\.core\.windows\.net" /etc/hosts ||
fi
if ! command -v azurite &> /dev/null; then
echo "Azurite is not installed. Please install it with 'npm install -g azurite'"
echo "ERROR: Azurite is not installed. Please install it with 'npm install -g azurite'"
exit 1
fi
if command -v caddy &> /dev/null; then
CADDY=true
else
CADDY=""
if ! command -v caddy &> /dev/null; then
echo "ERROR: Caddy is not installed. Please install it from https://caddyserver.com/docs/install or with 'brew install caddy' on macOS."
exit 1
fi
AZURITE_SSL=""
CERT_ARGS=()
OAUTH_ARGS=()
PORTS_ARGS=("--blobPort" "10000" "--queuePort" "10001" "--tablePort" "10002")
while [[ $# -gt 0 ]]; do
case "$1" in
--oauth)
case "$2" in
blob)
PORTS_ARGS=("--blobPort" "443" "--queuePort" "10001" "--tablePort" "10002")
;;
queue)
PORTS_ARGS=("--blobPort" "10000" "--queuePort" "443" "--tablePort" "10002")
;;
table)
PORTS_ARGS=("--blobPort" "10000" "--queuePort" "10001" "--tablePort" "443")
;;
*)
echo "Error: --oauth must be followed by 'blob', 'queue', or 'table'." >&2
exit 1
;;
esac
--oauth|-o)
OAUTH_ARGS=("--oauth" "basic")
# Ensure Caddy is disabled when using OAuth, as Azurite does not support OAuth behind a reverse proxy.
CADDY=""
AZURITE_SSL=true
shift 2
;;
--ssl)
AZURITE_SSL=true
# Ensure Caddy is disabled when using Azurite's built-in SSL support.
CADDY=""
shift
;;
--no-caddy)
# Disable Caddy, but do not enable SSL for Azurite.
CADDY=""
shift
;;
*)
@@ -89,36 +57,30 @@ while [[ $# -gt 0 ]]; do
done
# Ensure certificates are generated before starting Azurite or Caddy.
if [[ -n "$AZURITE_SSL" || -n "$CADDY" ]]; then
if ! make_ca "$AZURITE_DIR"; then
echo "Error: Failed to create CA certificate and key." >&2
exit 1
fi
if ! make_server_cert "$ACCOUNT_NAME" "$AZURITE_DIR"; then
echo "Error: Failed to create server certificate and key." >&2
exit 1
fi
if ! make_ca "$AZURITE_DIR"; then
echo "Error: Failed to create CA certificate and key." >&2
exit 1
fi
if [[ -n "$CADDY" ]]; then
# If using Caddy, it will reverse proxy blob, queue, and table endpoints to different ports on localhost,
# allowing simultaneous access to all services on a single HTTPS port.
# Generate a Caddyfile configuration based on the account name and storage directory.
sed -E "s/__ACCOUNT_NAME__/${ACCOUNT_NAME}/g; s|__AZURITE_DIR__|${AZURITE_DIR}|g" "$SCRIPT_DIR/Caddyfile.example" > "$AZURITE_DIR/Caddyfile"
echo "Starting Caddy server..."
caddy start --config "$AZURITE_DIR/Caddyfile" # Use start not run, start does not block the shell process.
trap "echo 'Stopping Caddy server...'; caddy stop" EXIT INT TERM HUP KILL STOP
CERT_ARGS=()
OAUTH_ARGS=() # Ensure OAuth is disabled when using Caddy, as Azurite does not support OAuth in reverse proxy mode.
else
# If not using Caddy, configure Azurite to listen on all interfaces and use the generated self-signed certificate.
# This mode does not require Caddy, but it will not allow simultaneous access to the table and queue services on the same HTTPS port.
CERT_ARGS=("--key" "$AZURITE_DIR/server_key.pem" "--cert" "$AZURITE_DIR/server_cert.pem")
if ! make_server_cert "$ACCOUNT_NAME" "$AZURITE_DIR"; then
echo "Error: Failed to create server certificate and key." >&2
exit 1
fi
CERT_ARGS=()
# Generate a Caddyfile configuration based on the account name and storage directory.
sed -E "s/__ACCOUNT_NAME__/${ACCOUNT_NAME}/g; s|__AZURITE_DIR__|${AZURITE_DIR}|g" "$SCRIPT_DIR/Caddyfile.example" > "$AZURITE_DIR/Caddyfile"
echo "Starting Caddy server..."
caddy start --config "$AZURITE_DIR/Caddyfile" # Use start not run, start does not block the shell process.
trap "echo 'Stopping Caddy server...'; caddy stop" EXIT INT TERM HUP KILL STOP
# Start Azurite
echo "Starting Azurite..."
azurite \
--disableTelemetry \
--location "$AZURITE_DIR" \
--blobHost 127.0.0.1 --queueHost 127.0.0.1 --tableHost 127.0.0.1 \
"${PORTS_ARGS[@]}" "${CERT_ARGS[@]}" "${OAUTH_ARGS[@]}"
--key "$AZURITE_DIR/${ACCOUNT_NAME}_key.pem" --cert "$AZURITE_DIR/${ACCOUNT_NAME}_cert.pem" \
"${OAUTH_ARGS[@]}"

30
run.sh
View File

@@ -1,36 +1,12 @@
#!/usr/bin/env bash
# By default Caddy exposes the SSL termination endpoint and
# uses DNS name to determine which service to route to.
EXPOSED_PORTS=("-p" "443:443")
CONTAINER_ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
-o|--oauth)
# OAuth support
case "$2" in
blob)
EXPOSED_PORTS=("-p" "443:10000")
;;
queue)
EXPOSED_PORTS=("-p" "443:10001")
;;
table)
EXPOSED_PORTS=("-p" "443:10002")
;;
*)
echo "Error: --oauth must be followed by 'blob', 'queue', or 'table'." >&2
exit 1
;;
esac
CONTAINER_ARGS+=("--oauth")
shift 2
;;
--ssl|-s|--no-caddy|-n)
CONTAINER_ARGS+=("$1")
EXPOSED_PORTS=("-p" "10000:10000" "-p" "10001:10001" "-p" "10002:10002")
shift
;;
*)
@@ -40,12 +16,10 @@ while [[ $# -gt 0 ]]; do
esac
done
echo "Using exposed ports: ${EXPOSED_PORTS[*]}"
if command -v dockerd &> /dev/null; then
docker run --rm -it --name azurite --env-file accounts.env "${EXPOSED_PORTS[@]}" -v ./storage:/storage azurite:latest "${CONTAINER_ARGS[@]}"
docker run --rm -it --name azurite --env-file accounts.env -p 443:443 -v ./storage:/storage azurite:latest "${CONTAINER_ARGS[@]}"
elif command -v container &> /dev/null; then
container run --rm -it --name azurite --env-file accounts.env "${EXPOSED_PORTS[@]}" --mount type=bind,source=./storage,target=/storage azurite:latest "${CONTAINER_ARGS[@]}"
container run --rm -it --name azurite --env-file accounts.env -p 443:443 --mount type=bind,source=./storage,target=/storage azurite:latest "${CONTAINER_ARGS[@]}"
else
echo "Neither supported container runtime found." >&2
exit 1

View File

@@ -1,4 +1,3 @@
storage_account_name = "terraform"
container_name = "tfstate"
key = "test.tfstate"
use_azuread_auth = true

View File

@@ -1,3 +1,27 @@
#!/usr/bin/env bash
terraform init -backend-config backend.config
ACCESS_KEY=""
while [[ $# -gt 0 ]]; do
case "$1" in
--key|-k)
if [[ ! -f "../accounts.env" ]]; then
echo "Error: accounts.env file not found at ../accounts.env" >&2
exit 1
fi
# Load the accounts.env file to get the ACCESS_KEY variable.
source "../accounts.env"
ACCESS_KEY=$(echo "$AZURITE_ACCOUNTS" | cut -f 1 -d ';' | cut -f 2 -d ':')
;;
*)
echo "Unknown argument: $1"
exit 1
;;
esac
shift
done
if [[ -z "$ACCESS_KEY" ]]; then
terraform init -backend-config="backend.config" -backend-config="use_azuread_auth=true" -reconfigure
else
terraform init -backend-config="backend.config" -backend-config="access_key=$ACCESS_KEY" -reconfigure
fi