Add a new AWS backend using Amazon Machine Images

The existing AWS backend uses system containers backed on to the AWS
hosted NFS service (EFS). This has some advantages, but also some
disadvantages.

Using the EFS service allows building a container on one instance,
with the state being held on the EFS, and then launching a new
instance to run the container. Using EFS also provides persistence, at
least beyond individual EC2 instances.

However, build performance when using EFS is poor compared with a
local store without the overhead of the network latency. Additionally,
the startup speed of the container running off EFS is slow compared to
local storage.

This backend doesn't use EFS, instead the Guix store sits on instance
storage. Rather than using a system container for a Mini Environment,
an Amazon Machine Image (AMI) is built instead. The fast local storage
makes builds faster, and using EBS storage for the Mini Envirnoments
as well as not having the overhead of starting Ubuntu, then starting
the container also makes startup faster.

2 files changed, 554 insertions, 0 deletions

diff --git a/app/views/backends/terraform_aws_using_ami/new.html.erb b/app/views/backends/terraform_aws_using_ami/new.html.erb
new file mode 100644
index 0000000..929d69f
--- /dev/null
+++ b/app/views/backends/terraform_aws_using_ami/new.html.erb
@@ -0,0 +1,192 @@
+<%#
+
+GOV.UK Mini Environment Admin
+Copyright © 2018, 2019  Christopher Baines <mail@cbaines.net>
+
+This file is part of the GOV.UK Mini Environment Admin.
+
+The GOV.UK Mini Environment Admin is free software: you can
+redistribute it and/or modify it under the terms of the GNU Affero
+General Public License as published by the Free Software Foundation,
+either version 3 of the License, or (at your option) any later
+version.
+
+The GOV.UK Mini Environment Admin is distributed in the hope that it
+will be useful, but WITHOUT ANY WARRANTY; without even the implied
+warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+the GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public
+License along with the GOV.UK Mini Environment Admin.  If not, see
+<http://www.gnu.org/licenses/>.
+
+%>
+
+<a href="<%= setup_path %>" class="btn btn-lg btn-default pull-right">
+  Back to setup
+</a>
+
+<h1>Create a new AWS backend</h1>
+
+<div class="row">
+  <div class="col-md-10">
+    <%= form_with(model: @backend,
+                  url: { action: "create" },
+                  html: { class: "form-horizontal" }) do |f|
+    %>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :label, class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :label,
+            class: 'form-control',
+            placeholder: 'Label for this backend'
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :domain, class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :domain,
+            class: 'form-control',
+            placeholder: 'Domain within which to host mini environments'
+          ) %>
+          <span class="help-block">
+            <p>
+              For example, if you entered <samp>example.com</samp> for
+              the backend domain, and then created a mini environment
+              called "Test" using this backend, then the application
+              domain for the mini environment would be
+              <samp>test.example.com</samp>.
+            </p>
+            <p>
+              A Route53 Hosted Zone will be created for this domain,
+              and records added for the mini environments.
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :aws_region, 'AWS Region', class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :aws_region,
+            class: 'form-control',
+            placeholder: 'What region to use'
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :vpc_id,
+          'VPC ID',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :vpc_id,
+            class: 'form-control',
+            placeholder: 'The ID of the VPC (Virtual Private Cloud) to use'
+          ) %>
+          <span class="help-block">
+            <p>
+              This VPC (Virtual Private Cloud) should be the one to
+              use for all resources.
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :route_53_zone_id,
+          'Route 53 Zone ID',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :route_53_zone_id,
+            class: 'form-control',
+            placeholder: 'The ID of the Route 53 Zone to use'
+          ) %>
+          <span class="help-block">
+            <p>
+              This zone should be authoritive for the domain this
+              backend is using. Entries in this zone will be created
+              for the mini environments.
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :aws_access_key_id, 'AWS Access Key ID', class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :aws_access_key_id,
+            class: 'form-control',
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :aws_secret_access_key, 'AWS Secret Access Key', class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.password_field(
+            :aws_secret_access_key,
+            class: 'form-control',
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :ssh_public_key,
+          'SSH Key, public part',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_area(
+            :ssh_public_key,
+            class: 'form-control',
+            placeholder: 'The public part of the SSH key to use'
+          ) %>
+          <span class="help-block">
+            <p>
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :ssh_private_key,
+          'SSH Key, private part',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_area(
+            :ssh_private_key,
+            class: 'form-control',
+            placeholder: 'The private part of the SSH key to use'
+          ) %>
+          <span class="help-block">
+            <p>
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <div class="col-sm-offset-4 col-sm-8">
+          <%= f.submit "Create", class: 'btn btn-lg btn-success' %>
+        </div>
+      </div>
+    <% end %>
+  </div>
+</div>
diff --git a/app/views/backends/terraform_aws_using_ami/show.html.erb b/app/views/backends/terraform_aws_using_ami/show.html.erb
new file mode 100644
index 0000000..14810d1
--- /dev/null
+++ b/app/views/backends/terraform_aws_using_ami/show.html.erb
@@ -0,0 +1,362 @@
+<%#
+
+GOV.UK Mini Environment Admin
+Copyright © 2018, 2019  Christopher Baines <mail@cbaines.net>
+
+This file is part of the GOV.UK Mini Environment Admin.
+
+The GOV.UK Mini Environment Admin is free software: you can
+redistribute it and/or modify it under the terms of the GNU Affero
+General Public License as published by the Free Software Foundation,
+either version 3 of the License, or (at your option) any later
+version.
+
+The GOV.UK Mini Environment Admin is distributed in the hope that it
+will be useful, but WITHOUT ANY WARRANTY; without even the implied
+warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+the GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public
+License along with the GOV.UK Mini Environment Admin.  If not, see
+<http://www.gnu.org/licenses/>.
+
+%>
+
+<a href="<%= setup_path %>" class="btn btn-lg btn-default pull-right">
+  Back to Setup
+</a>
+
+<h1>Backend: <%= @backend.label %></h1>
+<% status = @backend.status %>
+
+<br>
+
+<div class="row">
+  <div class="col-md-3">
+
+    <div class="panel panel-default">
+      <div class="panel-heading">
+        Current Status<br>
+        <small>updated at <%= status[:updated_at] %></small>
+      </div>
+      <div class="panel-body">
+        <% if status[:running] %>
+          <div class="alert alert-success text-center"
+               role="alert"
+               style="margin-bottom: 0px;">
+            Backend available
+          </div>
+        <% else %>
+          <div class="alert alert-warning text-center"
+               role="alert"
+               style="margin-bottom: 0px;">
+            Backend down
+          </div>
+        <% end %>
+
+        <br>
+        <a href="<%= terraform_http_backend_show_history_path(
+                 @backend.terraform_state_id
+                 ) %>">
+          View Terraform state information
+        </a>
+      </div>
+    </div>
+
+    <%= form_with(
+      url: perform_action_terraform_aws_using_ami_backend_path(@backend),
+      local: true,
+      method: "post"
+    ) do %>
+
+      <div class="panel panel-default">
+        <div class="panel-heading">Actions</div>
+
+        <ul class="list-group">
+          <li class="list-group-item">
+            <%= submit_tag('Deploy',
+                           role: 'button',
+                           style: 'margin-bottom: 5px;',
+                           class: 'btn btn-lg btn-success btn-block')
+            %>
+            <p class="list-group-item-text">
+              Run Terraform to deploy this backend, ensuring everything
+              is setup to create new mini environments.
+            </p>
+          </li>
+          <li class="list-group-item">
+            <%= submit_tag("Stop",
+                           role: 'button',
+                           style: 'margin-bottom: 5px;',
+                           class: 'btn btn-lg btn-warning btn-block')
+            %>
+            <p class="list-group-item-text">
+              Run Terraform to stop this backend, terminating the
+              build machine used to build new mini
+              environments. Existing environments will continue to
+              run, but new environments cannot be created.
+            </p>
+          </li>
+          <li class="list-group-item">
+            <%= submit_tag("Destroy",
+                           role: 'button',
+                           style: 'margin-bottom: 5px;',
+                           class: 'btn btn-lg btn-danger btn-block')
+            %>
+            <p class="list-group-item-text">
+              Run Terraform to destroy this backend, note that this will
+              remove all cached data and disrupt all mini environments
+              using this backend.
+            </p>
+          </li>
+          <li class="list-group-item">
+            <%= submit_tag("Refresh state",
+                           role: 'button',
+                           style: 'margin-bottom: 5px;',
+                           class: 'btn btn-lg btn-info btn-block')
+            %>
+            <p class="list-group-item-text">
+              Run Terraform to refresh the state information for this
+              backend.
+            </p>
+          </li>
+        </ul>
+      </div>
+    <% end %>
+  </div>
+  <div class="col-md-9">
+    <%= render(
+      partial: 'shared/jobs',
+      locals: {
+        jobs: Backends::TerraformAwsJob.jobs(@backend.id).order(id: :desc)
+      }
+    ) %>
+  </div>
+</div>
+
+<div class="row">
+  <div class="col-md-8">
+    <h3>Update details</h3>
+    <br>
+
+    <%= form_with(model: @backend,
+                  url: { action: "update" },
+                  html: { class: "form-horizontal" }) do |f|
+    %>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :label, class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :label,
+            class: 'form-control',
+            placeholder: 'Label for this backend'
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :domain, class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :domain,
+            class: 'form-control',
+            placeholder: 'Domain within which to host mini environments',
+            readonly: true
+          ) %>
+          <span class="help-block">
+            <p>
+              For example, if you entered <samp>example.com</samp> for
+              the backend domain, and then created a mini environment
+              called "Test" using this backend, then the application
+              domain for the mini environment would be
+              <samp>test.example.com</samp>.
+            </p>
+            <p>
+              A Route53 Hosted Zone will be created for this domain,
+              and records added for the mini environments.
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :aws_region, 'AWS Region', class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :aws_region,
+            class: 'form-control',
+            placeholder: 'What region to use',
+            readonly: true,
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :vpc_id,
+          'VPC ID',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :vpc_id,
+            class: 'form-control',
+            placeholder: 'The ID of the VPC (Virtual Private Cloud) to use',
+            readonly: true
+          ) %>
+          <span class="help-block">
+            <p>
+              This VPC (Virtual Private Cloud) should be the one to
+              use for all resources.
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :route_53_zone_id,
+          'Route 53 Zone ID',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :route_53_zone_id,
+            class: 'form-control',
+            placeholder: 'The ID of the Route 53 Zone to use',
+            readonly: true
+          ) %>
+          <span class="help-block">
+            <p>
+              This zone should be authoritive for the domain this
+              backend is using. Entries in this zone will be created
+              for the mini environments.
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :aws_access_key_id, 'AWS Access Key ID', class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.text_field(
+            :aws_access_key_id,
+            class: 'form-control',
+          ) %>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label :aws_secret_access_key, 'AWS Secret Access Key', class: 'col-sm-4 control-label' %>
+        <div class="col-sm-8">
+          <%= f.password_field(
+            :aws_secret_access_key,
+            class: 'form-control',
+            placeholder: 'Secret key hidden',
+          ) %>
+          <span id="helpBlock" class="help-block">
+            The AWS Secret Access Key is not accessible once entered.
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :ssh_public_key,
+          'SSH Key, public part',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= f.text_area(
+            :ssh_public_key,
+            class: 'form-control',
+            placeholder: 'The public part of the SSH key to use',
+            readonly: true
+          ) %>
+          <span class="help-block">
+            <p>
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <%= f.label(
+          :ssh_private_key,
+          'SSH Key, private part',
+          class: 'col-sm-4 control-label'
+        ) %>
+        <div class="col-sm-8">
+          <%= text_area_tag(
+            :ssh_private_key,
+            'Secret key hidden',
+            class: 'form-control',
+            disabled: true
+          ) %>
+          <span class="help-block">
+            <p>
+            </p>
+          </span>
+        </div>
+      </div>
+
+      <div class="form-group form-group-lg">
+        <div class="col-sm-offset-2 col-sm-10">
+          <%= f.submit "Save", class: 'btn btn-lg btn-success' %>
+        </div>
+      </div>
+    <% end %>
+  </div>
+  <div class="col-md-4">
+    <h3>Delete backend</h3>
+    <br>
+
+    <% unless @backend.mini_environments.empty? %>
+      <p>
+        Unable to delete backend, as mini environments using this
+        backend still exist.
+      </p>
+
+      <p>
+        To delete this backend, first delete all the mini environments
+        using it.
+      </p>
+    <% end %>
+
+    <%= form_with(model: @backend,
+                  url: { action: "destroy" },
+                  html: { class: "form-horizontal", method: :delete }) do |f|
+    %>
+      <%= f.submit(
+        "Delete",
+        class: (
+          'btn btn-lg btn-danger' +
+          (@backend.mini_environments.empty? ? '' : ' disabled')
+        )
+      ) %>
+    <% end %>
+
+  </div>
+</div>
+
+<h3>Mini environments</h3>
+
+<table class="table table-striped">
+  <tr>
+    <th>Name</th>
+    <th></th>
+  </tr>
+  <% @backend.mini_environments.each do |mini_environment| %>
+    <tr>
+      <td><%= mini_environment.name %></td>
+      <td>
+        <a class="btn btn-default btn-lg pull-right"
+           role="button"
+           href="<%= mini_environment_path(mini_environment) %>">
+          Show details
+        </a>
+      </td>
+    </tr>
+  <% end %>
+</table>