diff --git a/README.md b/README.md index 4628e0e..35bdf53 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ * **Format** a file system * **Label** a file system -* **Resize** a file system (when a threshold is reached) +* **Resize** a file system * **Mount** a block device -* **Ownership** and **Permissions** management of the mount point +* Manage **ownership** and **permissions** of the mount point Currently, the following file systems are supported for querying and modification... @@ -45,21 +45,19 @@ curl -L \ sudo install -m755 /tmp/ebs-bootstrap /usr/local/sbin/ebs-bootstrap ``` -## Modes +## Documentation -// TODO - -## Configuration - -// TODO + + + ## Use Cases ### `systemd` -A potential way of operating `ebs-bootstrap` is through a `systemd` service. This is so we can configure it as a `oneshot` service type that executes after the file system is ready and after `clout-init.service` writes any config files to disk. The latter is essential as `ebs-bootstrap` consumes a config file that is located at `/etc/ebs-boostrap/config.yml` by default. +A potential way of operating `ebs-bootstrap` is through a `systemd` service. This is so we can configure it as a `oneshot` service type that executes after the file system is ready and `clout-init.service` writes any config files to disk. The latter is essential as `ebs-bootstrap` consumes a config file that is located at `/etc/ebs-boostrap/config.yml` by default. -`ExecStopPost=-...` con point towards a script that is executed when the `ebs-bootstrap` service exits on either success or failure. This is a suitable place to include logic to notify an individual that the configured devices failed their relevant healthchecks and the underlying application failed to launch in the process. +`ExecStopPost=-...` can point towards a script that is executed when the `ebs-bootstrap` service exits on either success or failure. This is a suitable place to include logic to notify an individual that the configured devices failed their relevant healthchecks and the underlying application failed to launch in the process. ```ini [Unit] @@ -79,17 +77,9 @@ ExecStopPost=-/etc/ebs-bootstrap/post-hook.sh WantedBy=multi-user.target ``` -``` -cat /etc/ebs-bootstrap/post-hook.sh -#!/bin/sh -if [ "${EXIT_STATUS}" = "0" ]; then - echo "🟢 Post Stop Hook: Success" -else - echo "🔴 Post Stop Hook: Failure" -fi -``` +It is then possible to configure another `systemd` service to only start if the `ebs-bootstrap` service is successful. Certain databases support the ability to spread database chunks across multiple devices that need to be mounted to pre-defined directories with the correct ownership and permissions. -It is then possible to configure another `systemd` service to only start if the `ebs-bootstrap` service is successful. Certain databases support the ability to spread database chunks across multiple devices that need to be mounted to pre-defined directories with the correct ownership and permissions. In this particular use-case, the database could be configured as a `systemd` service that relies on the `ebs-bootstrap.service` to succeed before attempting to start. This can be achieved by specifiying `ebs-boostrap.service` as a dependency in the `Requires=` and `After=` parameters. +In this particular use-case, the database could be configured as a `systemd` service that relies on the `ebs-bootstrap.service` to succeed before attempting to start. This can be achieved by specifiying `ebs-boostrap.service` as a dependency in the `Requires=` and `After=` parameters. ```ini [Unit] @@ -111,59 +101,67 @@ WantedBy=multi-user.target ### `cloud-init` -By default, `ebs-bootstrap` consumes a configuration file located at `/etc/ebs-boostrap/config.yml`. `cloud-init` can be configured to write a config to this location, using the `write_files` module. Ensure that `ebs-bootstrap` is installed on your Instance via the process of baking it into your Golden AMI or downloading it early in the boot process, using the `runcmd` module. +By default, `ebs-bootstrap` consumes a configuration file located at `/etc/ebs-boostrap/config.yml`. `cloud-init` can be configured to write a config to this location, using the `write_files` module. Ensure that `ebs-bootstrap` is installed on your Instance via the process of baking it into your Golden AMI or downloading it early in the boot process, using the `bootcmd` module. The advent of Instance Store provided Nitro-enabled EC2 instances the ability to harness the power of high speed NVMe. For a stateful workload like a database, you might want a fast and ephemeral space for temporary tables, alongside a stateful EBS volume declared in a different CloudFormation Stack. However, these Instance Store devices were ephemeral and had to be formatted and mounted on each startup cycle. From the perspective of a **sceptical** Platforms Engineer, you do not mind a tool like `ebs-bootstrap` automating the task of formatting and mounting an ephemeral device. However, you personally draw the line on automation executing modifications to a stateful device, **without** the prior consent of a human. `ebs-bootstrap` empowers this Platform Engineer by allowing them to specify the execution mode, on a **device-by-device** basis: Instance Store (`force`) and EBS Volume (`healthcheck`) ```yaml +Parameters: + LatestUbuntuAmi: + Type: AWS::SSM::Parameter::Value + Default: /aws/service/canonical/ubuntu/server/20.04/stable/current/amd64/hvm/ebs-gp2/ami-id + Resources: Instance: Type: AWS::EC2::Instance - ... - InstanceType: m5ad.large # Nitro Instance Type - Volumes: - - Device: /dev/sdb # EBS Volume (Stateful) - VolumeId: !ImportValue EbsVolumeId - BlockDeviceMappings: - - DeviceName: /dev/sdh # Instance Store (Ephemeral) - VirtualName: ephemeral0 - UserData: - Fn::Base64: !Sub - - |+ - #cloud-config - write_files: - - content: | - devices: - /dev/sdb: - fs: ${FileSystem} - mountPoint: /mnt/ebs - mountOptions: ${MountOptions} - user: ec2-user - group: ec2-user - permissions: 755 - label: stateful - mode: healthcheck - /dev/sdh: - fs: ${FileSystem} - mountPoint: /mnt/instance-store - mountOptions: ${MountOptions} - user: ec2-user - group: ec2-user - permissions: 755 - label: ephemeral - mode: force - path: /etc/ebs-bootstrap/config.yml - runcmd: - - curl -L -o /tmp/ebs-bootstrap "${EbsBootstrapUrlPrefix}-$(uname -m)" - - install -m755 /tmp/ebs-bootstrap /usr/local/sbin/ebs-bootstrap - bootcmd: - - /usr/local/sbin/ebs-bootstrap - - FileSystem: ext4 - MountOptions: defaults,nofail,x-systemd.device-timeout=5 - EbsBootstrapUrlPrefix: > - https://github.com/reecetech/ebs-bootstrap/releases/download/latest/ebs-bootstrap-linux + Properties: + ... + ImageId: !Ref LatestUbuntuAmi + InstanceType: m5ad.large # Nitro Instance Type + Volumes: + - Device: /dev/sdb # EBS Volume + VolumeId: !ImportValue StatefulVolumeId + BlockDeviceMappings: + - DeviceName: /dev/sdh # Instance Store + VirtualName: ephemeral0 + UserData: + Fn::Base64: !Sub + - |+ + #cloud-config + write_files: + - content: | + devices: + /dev/sdb: + fs: ${FileSystem} + mountPoint: /mnt/ebs + mountOptions: ${MountOptions} + user: ubuntu + group: ubuntu + permissions: 755 + label: stateful + mode: healthcheck + /dev/sdh: + fs: ${FileSystem} + mountPoint: /mnt/instance-store + mountOptions: ${MountOptions} + user: ubuntu + group: ubuntu + permissions: 755 + label: ephemeral + mode: force + path: /etc/ebs-bootstrap/config.yml + bootcmd: + - curl -L -o /tmp/ebs-bootstrap "${EbsBootstrapUrlPrefix}-$(uname -m)" + - install -m755 /tmp/ebs-bootstrap /usr/local/sbin/ebs-bootstrap + runcmd: + - /usr/local/sbin/ebs-bootstrap + mounts: + - [ "LABEL=stateful", /mnt/ebs, ${FileSystem}, "${MountOptions}", "0", "2"] + - FileSystem: ext4 + MountOptions: defaults,nofail,x-systemd.device-timeout=5 + EbsBootstrapUrlPrefix: https://github.com/reecetech/ebs-bootstrap/releases/latest/download/ebs-bootstrap-linux ``` Assuming this is the very first launch, `ebs-bootstrap` would refuse to perform any modifications associated to the EBS device as it was assigned the `healthcheck` mode. However, we can temporarily override this behaviour with the `-mode=prompt` option. This allows the Platform Engineer to approve any suggested changes by `ebs-bootstrap`. @@ -184,7 +182,8 @@ Assuming this is the very first launch, `ebs-bootstrap` would refuse to perform 🟢 Passed all validation checks ``` -By inspecting the output of `lsblk`, we can verify that `ebs-bootstrap` was able to recover the CloudFormation assigned block device mappings (`/dev/sdb` and `/dev/sdh`) from both EBS and Instance Store NVMe devices (`/dev/nvme1n1` and `/dev/nvme2n1`) and format/label/mount the respective devices +By inspecting the output of `lsblk`, we can verify that `ebs-bootstrap` was able to recover the CloudFormation assigned block device mappings (`/dev/sdb` and `/dev/sdh`) from both EBS and Instance Store NVMe devices (`/dev/nvme1n1` and `/dev/nvme2n1`) and format/label/mount the respective devices. + ``` [~] lsblk -o NAME,FSTYPE,MOUNTPOINT,LABEL,SIZE NAME FSTYPE MOUNTPOINT LABEL SIZE diff --git a/assets/ebs-bootstrap.png b/assets/ebs-bootstrap.png index 070f60e..c197f9b 100644 Binary files a/assets/ebs-bootstrap.png and b/assets/ebs-bootstrap.png differ diff --git a/internal/action/format.go b/internal/action/format.go index 6409c25..67b18f8 100644 --- a/internal/action/format.go +++ b/internal/action/format.go @@ -39,9 +39,9 @@ func (a *FormatDeviceAction) Prompt() string { } func (a *FormatDeviceAction) Refuse() string { - return fmt.Sprintf("Refused to format to %s", a.fileSystemService.GetFileSystem()) + return fmt.Sprintf("Refused to format %s to %s", a.device, a.fileSystemService.GetFileSystem()) } func (a *FormatDeviceAction) Success() string { - return fmt.Sprintf("Successfully formatted to %s", a.fileSystemService.GetFileSystem().String()) + return fmt.Sprintf("Successfully formatted %s to %s", a.device, a.fileSystemService.GetFileSystem().String()) } diff --git a/internal/action/format_test.go b/internal/action/format_test.go index 2d3c02d..afcb8b2 100644 --- a/internal/action/format_test.go +++ b/internal/action/format_test.go @@ -36,12 +36,12 @@ func TestFormatDeviceActionMessages(t *testing.T) { { Name: "Refuse", Message: fda.Refuse(), - ExpectedOutput: "Refused to format to ext4", + ExpectedOutput: "Refused to format /dev/xvdf to ext4", }, { Name: "Success", Message: fda.Success(), - ExpectedOutput: "Successfully formatted to ext4", + ExpectedOutput: "Successfully formatted /dev/xvdf to ext4", }, } for _, subtest := range subtests { diff --git a/internal/action/label.go b/internal/action/label.go index 49aeb11..d3d8cb0 100644 --- a/internal/action/label.go +++ b/internal/action/label.go @@ -41,9 +41,9 @@ func (a *LabelDeviceAction) Prompt() string { } func (a *LabelDeviceAction) Refuse() string { - return fmt.Sprintf("Refused to label to '%s'", a.label) + return fmt.Sprintf("Refused to label %s to '%s'", a.device, a.label) } func (a *LabelDeviceAction) Success() string { - return fmt.Sprintf("Successfully labelled to '%s'", a.label) + return fmt.Sprintf("Successfully labelled %s to '%s'", a.device, a.label) } diff --git a/internal/action/label_test.go b/internal/action/label_test.go index 8a84bd7..d9a869d 100644 --- a/internal/action/label_test.go +++ b/internal/action/label_test.go @@ -36,12 +36,12 @@ func TestLabelDeviceActionMessages(t *testing.T) { { Name: "Refuse", Message: lda.Refuse(), - ExpectedOutput: "Refused to label to 'example'", + ExpectedOutput: "Refused to label /dev/xvdf to 'example'", }, { Name: "Success", Message: lda.Success(), - ExpectedOutput: "Successfully labelled to 'example'", + ExpectedOutput: "Successfully labelled /dev/xvdf to 'example'", }, } for _, subtest := range subtests {