diff --git a/README.md b/README.md index 551f4df..8f9657e 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Review the [Documentation](#reference) for more details on how to use the emulat ## Installation -You can install and use the emulator in a few different ways, depending on your preferences and environment. The recommended way is to use a container runtime or Kubernetes, but you can also install it natively using Node.js. +You can install and use the emulator in a few different ways, depending on your preferences and environment. The recommended way is to use a container runtime or Kubernetes, but you can also install it natively using Node.js and Caddy HTTP Server. ### Using a container runtime @@ -14,13 +14,29 @@ To run the Azure Storage Emulator in a container, follow these steps: 1. Ensure that a container runtime is installed. This repository supports both Docker and Apple `container` command. -2. Build the emulator image using the provided Dockerfile: +2. Create a custom non-public certificate for the emulator. Use the provided `make-cert.sh` script to generate a self-signed CA certificate and a server certificate for the specified storage account name. The script will use `CA_DIR` (default: `./storage`), `CA_NAME` (default: `Azurite Emulator CA`), and `STORAGE_ACCOUNT_NAME` (default: `azuritelocal`) environment variables to determine the storage location for the certificates, the name of the CA, and the storage account name for which the server certificate will be generated. For example: + + ```bash + ./make-cert.sh + ``` + + or + + ```bash + CA_DIR=./myca CA_NAME="My Custom CA" STORAGE_ACCOUNT_NAME=myaccount ./make-cert.sh + ``` + + You can run the script multiple times with different `STORAGE_ACCOUNT_NAME` values to generate certificates for multiple storage accounts if needed. Just make sure to use the same `CA_DIR` and `CA_NAME` for all of them (or use the defaults) to ensure they are signed by the same CA. + +3. Build the emulator image using the provided Dockerfile: ```bash ./build.sh ``` -3. Run the emulator container: + or pull it from the Docker Hub registry using image name: `skoszewski/azurite:latest`. + +4. Run the emulator container: ```bash ./start-azurite @@ -28,36 +44,7 @@ To run the Azure Storage Emulator in a container, follow these steps: You can also use the included example `compose.yaml` file for running it using `docker compose` (or any other compose compatible CLI). -### Using Kubernetes - -Use the example manifest `azurite-deployment.yaml`. - -1. Create the required Secret from your local env file: - - ```bash - kubectl create secret generic azurite-accounts \ - --from-env-file="$HOME/.azurite/accounts.env" - ``` - -2. Deploy the manifest: - - ```bash - kubectl apply -f azurite-deployment.yaml - ``` - -3. Get the Kubernetes Service external IP: - - ```bash - kubectl get svc azurite -o jsonpath='{.status.loadBalancer.ingress[0].ip}' - ``` - -4. Add `/etc/hosts` entries on your workstation and point all Azurite account FQDNs to that Service IP (not `127.0.0.1`): - - ``` - .blob.core.windows.net .queue.core.windows.net .table.core.windows.net - ``` - -### Using Node.js +### Using Node.js and Caddy HTTP Server To install the Azure Storage Emulator natively on your machine, ensure you have Node.js (with npm) and Caddy HTTP Server installed, and follow these steps: @@ -118,18 +105,14 @@ To install the Azure Storage Emulator natively on your machine, ensure you have 127.0.0.1 .blob.core.windows.net .queue.core.windows.net .table.core.windows.net ``` -8. Run the server: +8. Create a certificate for the specified account name using the `make-cert.sh` as described in the container runtime installation steps. + +9. Run the server: ```bash ./run-server.sh ``` - You can add the `--oauth` or `-o` flag to simulate OAuth authentication. - - ```bash - ./run-server.sh --oauth - ``` - ## Accessing the blob storage ### RClone @@ -198,33 +181,6 @@ terraform { ## Command Reference -### `run-server.sh` - -The script is the entry point for starting the Azure Storage Emulator natively. It discovers the account name and key from the `accounts.env` file, generates the necessary SSL certificates, configures Caddy for HTTPS endpoints, and starts the Azurite server with the appropriate settings. - -It accepts the following optional flag: `--oauth` or `-o`. It enables OAuth simulation for the emulator. When this flag is set, the emulator simulates OAuth authentication flows, allowing you to test scenarios that involve Entra ID authentication. It does not implement a real authentication flow; instead, it accepts any valid token. - -The script assumes that both Azurite and Caddy are installed and available in the system's PATH. It also assumes that the `accounts.env` file is properly configured with at least one account name and key, and that the `/etc/hosts` file contains the necessary entries for the custom domain names, e.g. `accountname.blob.core.windows.net`, `accountname.queue.core.windows.net`, and `accountname.table.core.windows.net`. - -The storage location is determined by the `AZURITE_DIR` environment variable, which defaults to the relative `storage` subdirectory of the current directory if not set. Ensure that the specified directory is writable by the user running the script, as Azurite will need to create and manage files for the emulated storage accounts. - -The script will generate self-signed SSL certificates for the specified account name and store them in the storage directory. Caddy will be configured to use these certificates for the HTTPS endpoints. The emulator will be accessible at the following endpoints: - -- Blob service: `https://accountname.blob.core.windows.net` -- Queue service: `https://accountname.queue.core.windows.net` -- Table service: `https://accountname.table.core.windows.net` - -You need to add `storage/ca_cert.pem` as a trusted root certificate in your system to avoid SSL errors when connecting to the emulator. - -For Linux-based systems, you can use the following commands to add the certificate to the trusted store: - -```bash -sudo cp storage/ca_cert.pem /usr/local/share/ca-certificates/azurite_ca_cert.crt -sudo update-ca-certificates -``` - -For macOS, you can use the Keychain Access application to import the certificate and mark it as trusted. Windows users can use the Certificate Manager to import the certificate into the "Trusted Root Certification Authorities" store. - ### `build.sh` The script builds a container image for the Azure Storage Emulator using the provided Dockerfile. The image includes the Azurite server and Caddy HTTP server, configured to run the emulator with triple HTTPS endpoints and optional OAuth simulation. It does not require Azurite or Caddy to be installed on the host machine, as they are included in the container image. @@ -236,21 +192,48 @@ Accepted flags: - `--latest`: Uses the latest released version of Azurite from GitHub as the base for the container image. This flag cannot be used together with `--version`. - `--registry`: Specifies the container registry to which the built image will be pushed. If not provided, the image will only be built locally and not tagged with registry prefix. +Used environment variables: + +- `AZURITE_IMAGE`: Overrides the default image name (`azurite:latest`) for the built container image. This can be useful if you want to use a different naming convention or push to a specific registry. The `--registry` flag will replace the registry part of the image name, but it will not override the entire name, so you can still use a custom image name with the registry prefix if needed. + ### `start-azurite` -The script runs the Azure Storage Emulator using a supported container runtime (Docker or Apple `container` command). It accepts the same flag as `run-server.sh` to enable OAuth simulation (`--oauth` or `-o`). It also assumes `AZURITE_DIR` is either set in the environment or empty (not set), in which case it will default to the `./storage` subdirectory of the current directory. The script mounts the specified storage directory into the container, allowing you to persist data and access the generated SSL certificates on the host machine. - -The default image name is `azurite:latest`, but it can be overridden by setting the `AZURITE_IMAGE` environment variable before running the script. For example: - -```bash -AZURITE_IMAGE=myregistry/azurite:latest ./start-azurite -``` - -Both `AZURITE_DIR` and `AZURITE_IMAGE` should be set in the shell profile if you are running the emulator as a local Azure Storage replacement for development purposes. +The script runs the Azure Storage Emulator using a supported container runtime (Docker or Apple `container` command). It enables OAuth and starts Caddy by defualt. > **Remember**: Make backups of the storage directory when the container is not running. -You have to use the same procedure as for `run-server.sh` to install the generated CA certificate as a trusted root certificate in your system to avoid SSL errors when connecting to the emulator. +Accepted flags: + +- `--no-oauth` - disables OAuth simulation in the emulator. When this flag is set, you have to use the account key for authentication. +- `--no-caddy` - disables Caddy server and runs Azurite with its built-in HTTP server. This will result in the emulator being accessible over three HTTPS endpoints (ports: 10000, 10001, 10002). Use this flag if you want to run the emulator and proxy the endpoints yourself. Note that you have to configure your proxy to accept the certificate supplied for the emulator. Caddy uses `tls_trust_pool file ` directive. + +Used environment variables: + +- `AZURITE_DIR`: Specifies the directory on the host machine where the emulator will store its data. This directory will be mounted as `/storage` in the container, allowing the emulator to persist data across container restarts. +- `AZURITE_IMAGE`: Specifies the name of the container image to use when running the emulator. If not set, it defaults to `azurite:latest`, which is the default tag used by the `build.sh` script. + +### `run-server.sh` + +The script is the entry point for starting the Azure Storage Emulator natively. It discovers the account name and key from the `accounts.env` file, checks for the necessary SSL certificates, configures Caddy for HTTPS endpoints, and starts the Azurite server with the appropriate settings. + +The script assumes that both Azurite and Caddy are installed and available in the system's PATH. It also assumes that the `accounts.env` file is properly configured with at least one account name and key, and that the `/etc/hosts` file contains the necessary entries mapped to `127.0.0.1` for the custom domain names, e.g. `accountname.blob.core.windows.net`, `accountname.queue.core.windows.net`, and `accountname.table.core.windows.net`. + +The storage location is determined by the `AZURITE_DIR` environment variable. Data files are stored in the `storage` subdirectory. The directory structure pointed to by `AZURITE_DIR` will be created if it does not exist. + +The script will use the certificate for the listed endpoints. Caddy will be configured to use that certificate for all HTTPS endpoints, therefore the certificate must have all the required SANs (Subject Alternative Names) for the endpoints and must be trusted by the system. The emulator will be accessible at the following endpoints: + +- Blob service: `https://accountname.blob.core.windows.net` +- Queue service: `https://accountname.queue.core.windows.net` +- Table service: `https://accountname.table.core.windows.net` + +For Debian-based systems, you can use the following commands to add the certificate to the trusted store: + +```bash +sudo cp storage/ca_cert.pem /usr/local/share/ca-certificates/azurite_ca_cert.crt +sudo update-ca-certificates +``` + +For macOS, you can use the Keychain Access application to import the certificate and mark it as trusted. Windows users can use the Certificate Manager to import the certificate into the "Trusted Root Certification Authorities" store. ## Reference diff --git a/azurite-deployment.yaml b/azurite-deployment.yaml deleted file mode 100644 index 71a2d69..0000000 --- a/azurite-deployment.yaml +++ /dev/null @@ -1,69 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: azurite-storage - labels: - app.kubernetes.io/name: azurite - app.kubernetes.io/component: storage-emulator -spec: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 100Mi ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: azurite - labels: - app.kubernetes.io/name: azurite - app.kubernetes.io/component: storage-emulator -spec: - replicas: 1 - selector: - matchLabels: - app.kubernetes.io/name: azurite - template: - metadata: - labels: - app.kubernetes.io/name: azurite - app.kubernetes.io/component: storage-emulator - spec: - containers: - - name: azurite - image: azurite:latest - imagePullPolicy: IfNotPresent - args: - - --oauth - ports: - - name: https - containerPort: 443 - protocol: TCP - envFrom: - - secretRef: - name: azurite-accounts - volumeMounts: - - name: storage - mountPath: /storage - volumes: - - name: storage - persistentVolumeClaim: - claimName: azurite-storage ---- -apiVersion: v1 -kind: Service -metadata: - name: azurite - labels: - app.kubernetes.io/name: azurite - app.kubernetes.io/component: storage-emulator -spec: - type: LoadBalancer - selector: - app.kubernetes.io/name: azurite - ports: - - name: https - port: 443 - targetPort: 443 - protocol: TCP