Post Heading
+ +Lorem ipsum dolor sit amet, consectetur adipiscing elit.
+elixir or the iex command. Both will execute the code, but the second command opens an interactive IEx shell afterward. What if, we won't know until runtime whether we want a shell or not? How can we start an IEx session even when we use elixir, instead of iex, to run our script?"
+...
+
+{
+ id: "open-an-iex-shell-from-an-elixir-script"
+}
+
+## Method 1
+
+Here's a quick test script:
+
+`run.exs`
+ +```elixir +:shell.start_interactive({IEx, :start, []}) + +System.no_halt(true) +``` + +`System.no_halt(true)` is needed so that the interactive session doesn't die when the process running the script ends. You could also use the `elixir` command with the `--no-halt` option instead to achieve the same effect. + +When we run this script with just `elixir` (instead of `iex`), we should still see an interactive IEx session start. + +```bash +$ elixir run.exs +Erlang/OTP 26 [erts-14.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit:ns] + +Interactive Elixir (1.15.6) - press Ctrl+C to exit (type h() ENTER for help) +iex(1)> +``` + +## Method 2 + +While researching, I found another way that works, but it's even more obscure than the above method. + +```elixir +:user_drv.start_shell(%{initial_shell: {IEx, :start, []}}) +``` + +I can't find documentation for `user_drv`, I had to browse the [code](https://github.com/erlang/otp/blob/master/lib/kernel/src/user_drv.erl) directly. diff --git a/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md b/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md new file mode 100644 index 0000000..36ee86c --- /dev/null +++ b/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md @@ -0,0 +1,51 @@ +--- +title: "Start Erlang's Dialyzer With GUI From A Docker Container" +blurb: "Everything in OTP is command-line driven, so using containers during development has been without issue. But, Dialyzer, Erlang's static analysis tool, actually has a Graphical User Interface. How can we still use Dialyzer and its GUI even though Elixir is running inside a container?" +... + +{ + id: "start-erlangs-dialyzer-with-gui-from-a-docker-container" +} + +I use Docker mostly when working on software projects and I figured out how to get Erlang's Dialyzer GUI working in a Docker container. + +1. Start a container with X11 forwarding: + + ``` + $ docker run -it --rm --net=host -e "DISPLAY" -e "NO_AT_BRIDGE=1" \ + -v "$XAUTHORITY:/root/.Xauthority:rw" elixir bash + ``` + +2. Build ["the recommended minimal PLT for Erlang/OTP"](https://www.erlang.org/doc/apps/dialyzer/dialyzer_chapter#the-persistent-lookup-table): + + ``` + # dialyzer --build_plt --apps erts kernel stdlib mnesia + Creating PLT /root/.cache/erlang/.dialyzer_plt ... + ... + done (warnings were emitted) + ``` + +3. Start the Dialyzer GUI: + + a. From the command line: + + # dialyzer --gui + + b. From inside an IEx session: + + 1. Start the Dialyzer application: + + ```elixir + iex(1)> Application.load(:dialyzer) + :ok + ``` + + 2. Start the Dialyzer GUI: + + ```elixir + iex(2)> :dialyzer.gui() + ``` + +You should now see the Dialyzer GUI. + + diff --git a/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md b/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md new file mode 100644 index 0000000..d95bcfa --- /dev/null +++ b/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md @@ -0,0 +1,109 @@ +--- +blurb: "One thing we learned from a week of trying to make isometric vector drawings." +... + +{ + id: "fix-distortion-introduced-when-transforming-multiview-projections-to-isometric" +} + +## Objective + +Construct an isometric vector image of an object from top, front, and side view projections (like [this](https://workforce.libretexts.org/Bookshelves/Drafting_and_Design_Technology/Introduction_to_Drafting_and_AutoCAD_3D_(Baumback)/02%3A_Part_2/2.01%3A_Module_7-_Visualizing_Multiview_Drawings), but using Inkscape instead of AutoCAD). + +This is an explanation of one of the issues encountered and how it was dealt with. + +## A Short Primer On Graphical Projection + +Drawing a 3D object as it looks head-on from the top, the front, and the side views is called [multiview orthographic projection](https://en.wikipedia.org/wiki/Multiview_orthographic_projection){target=_local}. Figure 1, below, is [an example from 1912](https://commons.wikimedia.org/wiki/File:3D_projection_views.png#/media/File:3D_projection_views.png){target=_local} that illustrates the concept. Each drawing is a view of the house from a different axis in three-dimensional space. Note the various other names, by which these views are known, indicated on the labeling. + +{width=300px} + +Another type of graphical projection, that you may have seen before, is called [isometric](https://en.wikipedia.org/wiki/Isometric_projection). Many 2D video games use isometric projection to simulate a 3D environment. One of the characteristics of isometric projection is, the angles between all three axes are equal. This results in the outline of a perfect cube forming a hexagon. Another is, it has no vanishing point, so there is no concern for things like draw-distance as there is in 3D games. + + + +## A Naive Solution + +We start with the multiview projections of the 3D object we want to turn into an isometric drawing: + + + +The task seems pretty straight-forward. First, we skew the front and side views. We want our truck to face to the right, so we need the top edge of the front view to match the angle of the blue axis from figure 2. To reach this angle, the skew must be 30°. The angle of the top of the side view must, therefore, match the red axis from figure 2. To achieve this, it has to be skewed 30° in the other direction (-30°, technically). + + + +Then, we transform the top view to fit our modified front and side views. + + + +All the angles for our box truck's skewed views look correct, but when we try to align them all together, we discover a problem. While the front and top views line up fine, when we try to align the top to the side view, we find that they do not fit together because they are different sizes. + + + +## The Problem + +The pieces don't fit together because when we skew a 2-dimensional rectangle we are actually stretching one of the axes. The top and bottom edges of the skewed side-face are longer than they were before the transformation. + + + +The skewed front and top views of our box truck also become stretched on one axis but because they both stretch the same axis, the pieces still fit together. + +## The Quick Fix + +We can fix the box truck's elongated side view by resizing it before we skew it. We need to shrink it along the axis that is going to be stretched by the skew. But how do we know what value to scale by? Here's how to find it: + + 1.  + Draw a line the same length as the truck side view. + + 2.  + Duplicate the line and rotate it by 30°. + + 3.  + Join one end of the two lines together. + + 4.  + Measure the horizontal distance between the other ends of the lines. This is the amount the side must be shrunk by so that it will be the correct size when stretched by skewing. + + 5.  + Shrink the side view by the distance we measured. + + 6.  + Now we can skew the side as we did before. + +This time when we try to line up the 3 skewed views of our isometric drawing, they should fit together. + + + +However, all the distortion in our render has not been eliminated. While this solution might look good, it is not true proportionally to the top and front views. They are still distorted from the skew along the axis perpendicular to the side view. + +## The Full Fix + +So, if the top and front views are still distorted, we will have to size them down and then re-skew them. + +Comparing the edges where the front view and the top view meet, we see that they are actually the same length. To find the distance the views need to be shrunk by, we will only need to make this calculation once. + + + +Now we can shrink the front view and then skew it as we did before. + + + +Likewise, we shrink the top view's width (in this case, this is the height of the diagram), skew it, and rotate it to fit. + + + +When we put all the pieces together, we can see that this version of our isometric model is less wide than before. This is because its width is no longer being stretched out of proportion. + + + +## Practical Applications + +Is this useful? Multiview projections of vehicles, buildings, etc. are easy to find online. If you want to build proportionally-accurate isometric drawings then this information will help. For example, here is an isometric taxi cab made with this technique and some found [diagrams](https://drawingdatabase.com/checker-marathon-taxi/). + + + +## Conclusion + +Making isometric drawings from blueprints of real objects is more fun and enjoyable when they come out looking like the real thing. Using the technique outlined here should eliminate distortion introduced from skewing multiview projections and retain their true proportions. Hopefully, it will improve your work as it has done mine. + + diff --git a/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md b/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md new file mode 100644 index 0000000..249b0f1 --- /dev/null +++ b/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md @@ -0,0 +1,368 @@ +--- +title: "Deploy Elixir-Generated HTML With Docker On DigitalOcean" +blurb: "This is a simple proof of concept where we create a boilerplate HTML file with Elixir, containerize our build process with Docker, and deploy our markup live with DigitalOcean's hosting service." +... + +{ + id: "deploy-elixir-generated-html-with-docker-on-digitalocean" +} + +## Introduction + +DigitalOcean has this [App Platform](https://www.digitalocean.com/products/app-platform) service that can host a static website, as well as build it from a Docker image, if provided a `Dockerfile`. We thought a static website built by an Elixir app could be an instructive project. To explore if the idea is viable, we wrote a small Elixir application that generates a simple `index.html` file and deployed it live on DigitalOcean's service. + +## Requirements + +* Docker +* GitHub account +* DigitalOcean account + +This is not an endorsement of these vendors. We don't have special affinity for any of them. We just like containers, git, and PaaS. This post is specific to DigitalOcean, though, because that's where we deploy our site. + +## Procedure + +The instructions are divided into four parts. + +1. [Create the mix project](#create-the-mix-project) +2. [Create the build task](#create-the-build-task) +3. [Add the `Dockerfile`](#add-the-dockerfile) +4. [Deploy live](#deploy-live) + +::: warning +We're going to do this whole thing with containers. That means Docker is all we need to have installed on our machines. Be warned, though, that also means we'll be running Elixir through the `docker` command. +::: + +### 1. Create the mix project + +We use the `mix new` command to create our Elixir project. If we were using Elixir installed natively on our computer, it would just be the last part, `mix new static_site`. Since we are using Docker, the command looks like this: + +``` +$ docker run --rm -w /opt -v $PWD:/opt -u $(id -u):$(id -u) elixir mix new static_site +``` + +That might look a bit overwhelming, so let's explain each part of the command. + +* `docker run elixir` + + `docker run` creates and starts a new container from an image. `elixir` uses the latest [official Docker elixir image](https://hub.docker.com/_/elixir). + +* `--rm` + + Since `docker run` creates a new container just to run this single command, this option will delete the container once it has finished. Using `--rm` keeps us from creating a bunch of containers that have to be cleaned up later. + +* `-v $PWD:/opt` + + We want the files Mix generates to be accessible to us so we can edit and keep them in version control. We need to bring them out of the container somehow. We do this by mount binding a volume. This option binds the result of `$PWD`, which is the current directory on our filesystem, to the `/opt` directory on the container filesystem. Any changes made in the container to the `/opt` directory will be reflected on our filesystem. + +* `-w /opt` + + This option sets the directory that the command will run in. Since we mounted our project files in the container's `/opt` directory, we want to set it as the working directory. + +* `-u $(id -u):$(id -u)` + + This option sets the container user and group to match our operating system's current user. If we don't do this, the files generated will all belong to `root` and be uneditable to us without using `sudo`. + +* `mix new static_site` + + The command we want to run in the container. + +After the command finishes, we have a new directory on our filesystem called `static_site`. Let's change into that directory and make sure we can run the tests. We don't care if the files `mix` creates in `_build` are owned by `root`, so we don't bother setting the user and group with the `-u` option when we run the command this time. + +``` +$ cd static_site +$ docker run --rm -w /opt -v $PWD:/opt elixir mix test +``` + +We should see a successful test result. + +With our Mix project files generated, we move on to implementing creating our static HTML file. + +### 2. Create the build task + +Because our output will only contain static markup, our Elixir application will not be a long running process in production. It will only run once, during the build phase of deployment. A one-off job is the perfect role for a Mix task. The [`Task` module documentation](https://hexdocs.pm/mix/Mix.Task.html) shows an example file to start with and even tells us where to put it (`lib/mix/tasks`). + +We name the task simply `build` and create a file called `build.ex`. It uses Elixir's `File` module to first create a directory called `/public`. Then, it writes a minimal `index.html` file at that location. + +::: filename-for-code-block +`lib/mix/tasks/build.ex` +::: + +```elixir +defmodule Mix.Tasks.Build do + @moduledoc "Creates a /public directory and places an HTML index file there" + @shortdoc "Builds static HTML file" + + use Mix.Task + + @output_directory "/public" + + @markup """ + + + + hello world + + """ + + @impl Mix.Task + def run(_args) do + Mix.shell().info("running build task") + + File.mkdir_p!(@output_directory) + @output_directory |> Path.join("index.html") |> File.write!(@markup) + end +end +``` + +The easiest way to test our task is to run `mix build` and then inspect the contents of `/public/index.html` with the `cat` command, but `docker run` only accepts a single command. We can combine both into one with `bash -c "mix build && cat /public/index.html"`. + +``` +$ docker run --rm -w /opt -v $PWD:/opt elixir bash -c "mix build && cat /public/index.html" +``` + +If all went well, our output should be: + +``` +running build task + + + + hello world + +``` + +With creating the Mix task complete, it is time to add a `Dockerfile` to our project. + +### 3. Add the `Dockerfile` + +We could commit HTML files to our repo directly and deploy them that way. It would not require Docker at all. But if we want to use Elixir to generate hypertext markup programatically, we will have to add a `Dockerfile` for building our project in production. + +#### Dependencies + +In the last section, we built the HTML file by calling our Mix task with the command, `mix build`. Let's make sure our build handles dependencies by adding one to the project. We add a `plug` dependency in the `mix.exs` file and try building again. + +`mix.exs`
+ +```elixir +defmodule StaticSite.MixProject do + use Mix.Project + ... + + defp deps do + [ + {:plug, ">= 0.0.0"} + ] + end +end +``` + +We will also have to add a call to `mix deps.get` in our command. Again, we combine the two commands into one with `&&`: + +``` +$ docker run --rm -w /opt -v $PWD:/opt elixir bash -c "mix deps.get && mix build" +* creating /root/.mix/archives/hex-2.0.6 +Resolving Hex dependencies... +... +* Getting plug (Hex package) +... +Compiling 2 files (.ex) +Generated static_site app +running build task +``` + +We should see hex installing, dependencies being fetched, the project compiled, and our mix task being ran. + +#### Environments + +There is a problem with this method, however. What if the dependencies are only needed during development? We don't want `dev` environment dependencies being included when we deploy to production. To test this, change `{:plug, ">= 0.0.0"}` to `{:plug, ">= 0.0.0", only: :dev}` in `mix.exs`. + +##### Development (`dev`) + +We will re-run our command, but we will add a call to `mix deps` to see what dependencies our project builds. + +``` +$ docker run --rm -w /opt -v $PWD:/opt elixir bash -c "mix deps.get && mix build && mix deps" +... +Compiling 2 files (.ex) +Generated static_site app +running build task +* mime 2.0.5 (Hex package) (mix) + locked at 2.0.5 (mime) da0d64a3 + ok +* plug 1.15.2 (Hex package) (mix) + locked at 1.15.2 (plug) 02731fa0 + ok +* plug_crypto 2.0.0 (Hex package) (mix) + locked at 2.0.0 (plug_crypto) 53695bae + ok +* telemetry 1.2.1 (Hex package) (rebar3) + locked at 1.2.1 (telemetry) dad9ce9d + ok +``` + +Our `dev` environment has the dependency we added. + +##### Production (`prod`) + +Now, we will run the command again after we set the environment variable `MIX_ENV` to `prod`. We do this with the `-e` option: + +``` +$ docker run --rm -w /opt -v $PWD:/opt -e MIX_ENV=prod elixir bash -c "mix deps.get && mix build && mix deps" +* creating /root/.mix/archives/hex-2.0.6 +Resolving Hex dependencies... +... +* Getting plug (Hex package) +* Getting mime (Hex package) +* Getting plug_crypto (Hex package) +* Getting telemetry (Hex package) +... +Compiling 2 files (.ex) +Generated static_site app +running build task +``` + +`mix deps` doesn't list any dependencies in the `prod` environment, which is what we want. However, we don't want it *getting* any dependencies that aren't used, either. + +##### Using the `--only` option + +It turns out that `mix deps.get` [has an `--only` option](https://hexdocs.pm/mix/Mix.Tasks.Deps.Get.html) that can be used to fetch dependencies only for a specific environment. We try our command again with that option. + +``` +$ docker run --rm -w /opt -v $PWD:/opt -e MIX_ENV=prod elixir bash -c "mix deps.get --only prod && mix build && mix deps" +* creating /root/.mix/archives/hex-2.0.6 +All dependencies are up to date +Compiling 2 files (.ex) +Generated static_site app +running build task +``` + +We don't see any dependencies being fetched, so that works as we want it to. We can set `MIX_ENV` to `prod` in production and use `mix deps.get --only $MIX_ENV` in our `Dockerfile` to fetch dependencies only belonging to that environment. + +#### Writing the file + +`Dockerfiles` have 2 different ways to use variables, `ENV` and `ARG`. `ENV` variables will be available in the running container, while `ARG` variables are only available during the image build process. Since we only need the variable to be available during the deployment, we need to use `ARG` in our `Dockerfile`. We can even set a default, so that we don't need to set `MIX_ENV` explicitly when we are developing. Here is our complete `Dockerfile`: + +`Dockerfile`
+ +```dockerfile +FROM elixir:slim + +WORKDIR /opt +COPY lib ./lib +COPY mix.exs ./mix.exs + +ARG MIX_ENV=dev + +RUN mix deps.get --only $MIX_ENV && mix build +``` + +Here's the explanation: + +* `FROM elixir:slim` + + This is the image we are basing ours on. The `slim` version is slightly smaller, so we chose it to save space. + +* `WORKDIR /opt` + + Just like the `-w` option in our `docker run` command, this sets the working directory to `/opt`. + +* `COPY lib ./lib` + `COPY mix.exs ./mix.exs` + + These copy our project files into the current working directory (`/opt`). + +* `ARG MIX_ENV=dev` + + Makes the `MIX_ENV` variable available in the `Dockerfile` if it is set in the environment. If it is not set, this tells Docker to use `dev` as the default value. + +* `RUN mix deps.get --only $MIX_ENV && mix build` + + Fetches depencies for our mix project and runs our build task. + +With that complete, we can now build an image. + +#### Building the image + +##### `dev` environment + +This first build will not have the `MIX_ENV` variable set, so we expect it to default to `dev` and to find the `plug` dependency installed. + +``` +$ docker build -t hw_dev . +``` + +Let's see what dependencies our image contains: + +``` +$ docker run --rm hw_dev mix deps + +* mime 2.0.5 (Hex package) (mix) + locked at 2.0.5 (mime) da0d64a3 + ok +* plug 1.15.2 (Hex package) (mix) + locked at 1.15.2 (plug) 02731fa0 + ok +* plug_crypto 2.0.0 (Hex package) (mix) + locked at 2.0.0 (plug_crypto) 53695bae + ok +* telemetry 1.2.1 (Hex package) (rebar3) + locked at 1.2.1 (telemetry) dad9ce9d + ok +``` + +That looks good. And let's also check that our HTML file was written: + +``` +$ docker run --rm hw_dev ls /public +index.html +``` + +##### `prod` environment + +Excellent. Now let's build a production image. We pass in the value to set for the `MIX_ENV` argument with the `--build-arg` option: + +``` +$ docker build -t hw_prod --build-arg MIX_ENV=prod . +``` + +And now we check as before. We have to set `MIX_ENV` in the container if we want our mix command to run in that environment. We do this with the `-e` option. + +``` +$ docker run --rm -e MIX_ENV=prod hw_prod mix deps +$ +``` + +This shows no dependencies, as expected. And check our index.html: + +``` +$ docker run --rm hw_prod ls /public +index.html +``` + +It works! Now we can deploy. + +### 4. Deploy live + +1. First, we push our git repo to GitHub. It doesn't have to be a public repo, we will give DigitalOcean access to it in a moment. +2. We [log in](https://cloud.digitalocean.com/login) to DigitalOcean, go to our Apps page and click "Create App". +3. On the "Create Resource From Source Code" screen, we click on "Edit Your GitHub Permissions". +4. We do what GitHub calls "installing" the DigitalOcean app on our GitHub account. +5. We add the repo to the "Repository access" list and click "Save". We should be redirected back to the DigitalOcean "Create Resource From Source Code" screen. +6. We select "GitHub" as the "Service Provider". Our repo should now appear in the list. We select it and click "Next". +7. It detected our Dockerfile and thinks we want to run a web service. We need to edit the resource to tell it it's a static site that just needs Docker to build. We click on the "Edit" button. +8. Under "Resource Type", we click "Edit" and select "Static Site". Then we click "Save". +9. We edit the "Output Directory" and set it to `/public`, the location of the static files in the container. Then click "< Back". +10. Under the "App" section, we should see "Starter Plan" and "Static Site". We click "Next". +11. On the "Environment Variables" page, we can set `MIX_ENV`, the variable our Dockerfile will need during the build process, to `prod`. It probably works the same whether set under "Global" or local to the app, but we just set it on the `static-site` app. +12. We don't have a reason to change anything on the "Info" screen, so we click "Next". If we wanted to change the resource name, this would be the time to do it. +13. At the bottom of the "Review" screen, we click "Create Resources". + +After deployment finishes successfully, we should be able to visit the link that DigitalOcean gives us and we will be greeted by our "hello world" HTML page. + +## Making changes + +If we want to make changes, we can commit our updates and push the code to GitHub. This will trigger a rebuild and deploy the changes, automatically. + +## Conclusion + +Our Elixir-generated HTML file is live and hosted. We have completed our proof of concept. If we wanted to take advantage of DigitalOcean's platform and host an Elixir-generated static website, this is the blueprint we could follow. It's relatively simple if familiar with Docker, and once set up, deploying changes with a `git push` is simply magical. We look forward to using what we have learned in a future project. diff --git a/docs/2023-11-15-test-mix-task-file-modify.md b/docs/2023-11-15-test-mix-task-file-modify.md new file mode 100644 index 0000000..fc4a5d4 --- /dev/null +++ b/docs/2023-11-15-test-mix-task-file-modify.md @@ -0,0 +1,326 @@ +--- +title: "Temporary Directories For Testing Mix Tasks That Modify Files" +blurb: "Writing a test for a simple Mix task gets surprisingly complex. Application environment variables, temporary test directories, and IO capture are all involved." +... + +{ + id: "test-mix-task-file-modify" +} + +## Intro + +Last time, we added a Mix task to our project that writes an HTML file to a directory `/public` in the container's filesystem. Today, we will write a test for that task. + +Here is the code we want to write a test for. + +```elixir +@markup """ + + + + hello world + +""" + +@output_directory "/public" + +@impl Mix.Task +def run(_args) do + Mix.shell().info("running build task") + + File.mkdir_p!(@output_directory) + @output_directory |> Path.join("index.html") |> File.write!(@markup) +end +``` + +## First pass + +The task writes some HTML markup to a file, `/public/index.html`, so our test should ensure that after the task is ran, that file exists. + +::: filename-for-code-block +`test/mix/tasks/build_test.exs` +::: + +```elixir +defmodule Mix.Tasks.BuildTest do + use ExUnit.Case + + test "creates output files" do + Mix.Tasks.Build.run([]) + + assert File.exists?("/public/index.html") + end +end +``` + +Let's try running the test. + +``` +$ docker run --rm -w /opt -v $PWD:/opt hw_dev mix test +running build task +... +Finished in 0.02 seconds (0.02s async, 0.00s sync) +1 doctest, 2 tests, 0 failures +``` + +While it passes, this is not a valid test. `/public/index.html` already exists before our test runs the build task, because our `Dockerfile` tells Docker to run the task when it builds our image. To make the test valid, we need it to use a clean directory. + +If we make the output directory configurable during runtime, we can create a temporary directory to use as a mock output directory during test runs. We need to move where the output directory is stored, from a hard-coded module attribute into something configureable at runtime. + +## Separating `test` and `dev` environment data + +Right now the value for the output directory is stored in a module attribute in the task module. If we had it stored in a variable, we could change it when we run our tests. + +### Configuring the output directory + +#### 1. Store the value in an application environment variable + +To store it in an application environment variable, add this in `mix.exs` + +```elixir + def application do + [ + env: [output_directory: "/public"], + ... + ] + end +``` + +Then, in `run()` in `lib/mix/tasks/build.ex`, we fetch the output directory from the application environment instead of hard-coding it in a module attribute + +```elixir + def run(_args) do + Mix.shell().info("running build task") + + output_directory = Application.fetch_env!(:static_site, :output_directory) + + File.mkdir_p!(output_directory) + output_directory |> Path.join("index.html") |> File.write!(@markup) + end +``` + +Now that the output directory is stored in an application env var, let's see about changing it for our test run. + + +#### 2. Change the value when running tests + +Now we can dynamically set the output directory in our test. Really we just have to set it a relative path instead of an absolute. Once we make it a relative path, that mix task will create the output directory relative to the current directory. + +```elixir + setup do + output_dir = "public" + Application.put_env(:static_site, :output_directory, output_dir) + + [output_dir: output_dir] + end + + + test "creates output file", %{output_path: output_dir} do + ... + assert File.exists?(Path.join(output_dir, "index.html")) + end +``` + +#### 3. Restore the original value + +But this changes the output directory application-wide, and it's good practice to put anything the tests change back to the way they were before. We can use `on_exit()` to set the original value when the test run finishes. We have to save it before changing the application variable and pass it to the call in `on_exit()` + +```elixir +setup do + output_dir = "public" + original = Application.fetch_env!(:static_site, :output_directory) + ... + on_exit(fn -> + Application.put_env(:static_site, :output_directory, original) + end) + ... +end +``` + +Now that we can change the output directory in our tests, we can point to a temporary directory that is separate from our dev files. But where should we put it? + +### Adding a temporary `tmp` directory for test artifacts + +#### 1. Locate the temporary test directory + +Before we spend too much time thinking about it, let's take a look how it's done in the Elixir source code. In [`lib/mix/test/test_helper.exs`](https://github.com/elixir-lang/elixir/blob/main/lib/mix/test/test_helper.exs) there's + +```elixir + def tmp_path do + Path.expand("../tmp", __DIR__) + end + + def tmp_path(extension) do + Path.join(tmp_path(), remove_colons(extension)) + end +``` + +`__DIR__` is whatever directory the current file is in, so for our task test in `test/mix/tasks` the temp directory would be `test/mix/tmp`. + +That seems like a good place to us. Let's copy `tmp_path()` into our test file. We can leave off the call to `remove_colons()`, since we don't have any colons to deal with. + +```elixir +defp tmp_path, do: Path.expand("../tmp", __DIR__) +defp tmp_path(extension), do: Path.join(tmp_path(), extension) +``` + +Now let's see about changing the location of the output directory. + +#### 2. Change the current directory during the test + +We can use `tmp_path/1` to create a temporary directory specifically for this test. + +```elixir +test "creates output file", %{output_dir: output_dir} do + File.mkdir_p!(tmp_path("build")) + File.cd!(tmp_path("build")) + ... +``` + +And at the end of our test, we need to change back into the test file's directory with + +```elixir + ... + File.cd!(__DIR__) +end +``` + +#### 3. Add fixture for missing file + +And since we're no longer in the project root directory, our task will not find the root `index.html` file. We need to add a fixture that our task can read from. + +```elixir +File.write("index.html", """ + + + + hello world + +""") +``` + +After running the test, + +``` +$ docker run --rm -w /opt -v $PWD:/opt hw_dev mix test +running build task +... +Finished in 0.02 seconds (0.02s async, 0.00s sync) +1 doctest, 2 tests, 0 failures +``` + +we have our test directory, `test/mix/tmp`, + +``` +$ find test/mix/tmp +test/mix/tmp +test/mix/tmp/build +test/mix/tmp/build/public +test/mix/tmp/build/public/index.html +test/mix/tmp/build/index.html +``` + +and our fixture file. + +``` +$ cat test/mix/tmp/build/index.html + + + + hello world + +``` + +## Cleaning up temporary directories + +Our tests are pretty solid now, but it is a good idea to clean the `tmp` directory up when the tests are done. We can do that in the `on_exit()` call in our test setup block + +```elixir +on_exit(fn -> + ... + File.rm_rf!(tmp_path()) +end) +``` + +Now, after we run the tests, all the files and directories created during the test run have been removed. + +``` +$ find test/mix/tmp/ +find: ‘test/mix/tmp/’: No such file or directory +``` + + +## Suppressing IO messages + +Our mix task outputs a useful message when it runs, but we don't want that to clutter up the test results. We can prevent those messages by "capturing" the output with the `ExUnit.CaptureIO` module. + +Add `import ExUnit.CaptureIO` to the top of our test file. Then, in our tests, we can pass our call to the mix task as a function to `capture_io()` + +```elixir +capture_io(fn -> Mix.Tasks.Build.run([]) end) +``` + +When we run our tests now, there are no more IO messages cluttering up the test results. + +``` +$ docker run --rm -w /opt -v $PWD:/opt hw_dev mix test +... +Finished in 0.02 seconds (0.02s async, 0.00s sync) +1 doctest, 2 tests, 0 failures +``` + + +## Final form + +Here's what the final draft of our test looks like. + +::: filename-for-code-block +`test/mix/tasks/build_test.exs` +::: + +```elixir +defmodule Mix.Tasks.BuildTest do + use ExUnit.Case + + import ExUnit.CaptureIO + + setup do + output_dir = "public" + original = Application.fetch_env!(:static_site, :output_directory) + Application.put_env(:static_site, :output_directory, output_dir) + + on_exit(fn -> + Application.put_env(:static_site, :output_directory, original) + File.rm_rf!(tmp_path()) + end) + + [output_dir: output_dir] + end + + test "creates output file", %{output_dir: output_dir} do + File.mkdir_p!(tmp_path("build")) + File.cd!(tmp_path("build")) + + File.write("index.html", """ + + + + hello world + + """) + + capture_io(fn -> Mix.Tasks.Build.run([]) end) + + assert File.exists?(Path.join(output_dir, "index.html")) + + File.cd!(__DIR__) + end + + defp tmp_path, do: Path.expand("../tmp", __DIR__) + defp tmp_path(extension), do: Path.join(tmp_path(), extension) +end +``` + +## Conclusion + +This test seemed trivial at first, but increased in complexity quickly. We had to set some of our configuration in application environment variables, change the configuration temporarily before a test run and then change it back after, clean up test artifacts after the run, and capture IO messages that were generated during it. That's enough going on that we thought it would be a good topic for a post. Cheers and happy coding! diff --git a/docs/2023-12-01-build-static-website-generator-part-1.md b/docs/2023-12-01-build-static-website-generator-part-1.md new file mode 100644 index 0000000..5b75a2e --- /dev/null +++ b/docs/2023-12-01-build-static-website-generator-part-1.md @@ -0,0 +1,904 @@ +--- +title: "Build A Static-Website Generator With Elixir, Part 1" +blurb: "We take the first steps in designing and implementing the \"world's simplest static-website generator\". Building on tools and knowledge we acquired previously, and utilizing an incremental and iterative development process, we go through the entire software life-cycle from creating the initial project files to deploying to production. We spare nothing, from spelling out every command, to ensuring application integrity with tests, and even updating the README file. Grab a drink and some snacks, and dive right in!" +... + +{ + id: "build-static-website-generator-part-1" +} + +::: info +This post was originally intended to be the first in a multi-part series. +However, the deeper we got into this project, the more we realized we were +basically implementing our own version of a web framework. Rather than +continuing, we opted to change course and adopted an already-existing +framework, [Phoenix](https://www.phoenixframework.org/), and simply added a +Markdown-to-HTML conversion feature. As a consequence, there are no other parts +to this post, but a description of the solution we chose instead can be found +[here](/posts/publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix). +::: + +This is one of our longer posts, so we've included a table of contents. + +## Table of Contents + +- Introduction + - Incremental Development Concepts + - Iterative Development Concepts +- Initial Planning + - Deliverables +- Initialization + - Planning + - Requirements + - Analysis & Design + - Implementation + - Testing + - Deployment + - Evaluation +- Project Control List + - New Tasks + - Completed Tasks +- Conclusion + +## Introduction + +We want to build the world's simplest static-website generator. Also, we want to use an iterative and incremental development method to do it. Finally, we want to use the Elixir programming language. Before we dive in, here's an extremely short primer on iterative and incremental development processes. We will combine these two ideas as we build this project. + +### Incremental Development Concepts + +- The system is broken down into small pieces +- Requirements are worked through in order of priority + +### Iterative Development Concepts + +- The entire software development lifecycle is traversed in each iteration +- Initialization, the initial "iteration", builds a base version of the system +- All tasks and priorities are tracked in a Project Control List + +We are now in the initialization phase. We will go through each of the seven iteration steps to build a minimum base system. + +1. Planning +2. Requirements +3. Analysis & Design +4. Implementation +5. Testing +6. Deployment +7. Evaluation + +## Initial Planning + +### Deliverables + +#### Functional Requirements + +* A way to publish written work online (ie blog) + +#### Non-Functional Requirements + +* Must use Elixir + + We love Elixir and functional programming. + +* Must be as simple as possible + + It's got to be dead simple. The app and the code. It's so easy to get carried away and over-engineer. Simplicity is a must-have requirement. + +* Must use an iterative and incremental development method + + In other words, short feedback loop/development life-cycle. Rather than releasing a minimum number of features as "version 1", we implement the smallest useful feature possible and release immediately. + +## Initialization + +### Planning + +We want to deliver the smallest increment of functionality possible, so we're going to start simple. We will first add an `index.html` serves as the initial page for our static website. Then, we will add a development server to serve the site to ourselves so we can see how it looks before we deploy it live. + +### Requirements + +#### Functional + +1. Post written text online + +This is the essence of the deliverable. Posting written text online is the first functionality we will deliver. + +#### Non-functional + +* A way to see what the output will look like locally before it is deployed + +This will make our lives easier during development and shorten the feedback loop. + +### Analysis & Design + +#### Analysis + +The smallest bit of functionality we can offer is a static website with a single HTML file. We actually made a [proof of concept](https://github.com/webdevcat-me/docker_static_site_test) that does exactly this already and described the process in detail in an [earlier post](deploy-elixir-generated-html-with-docker-on-digitalocean.html). If we can add a development web server, that will pretty much meet all of our requirements. + +#### Design + +We could list our posts in a single HTML file, sorted in descending order of date, so that the most recent posts are at the top. Readers should have to scroll down to see older posts. + +Ideally, each post would have a title, a short message, and the date it was posted. The text should not stretch across the whole viewport on a wide screen, but should wrap at a point that facilitates readability. The markup should be [well-structured](https://developer.mozilla.org/en-US/docs/Learn/HTML/Introduction_to_HTML/Document_and_website_structure) and valid. The column of text should be centered in the viewport but the text itself should be left-aligned. + +##### Name + +We've chosen "Serval" for the project name. + +### Implementation + +#### 1. Create a new Mix project + +We're going to need a supervisor to start our development server later when we add it, so we use the `--sup` option when we run the `mix new` command to generate our initial project files. We can refer to our [explanation of this command](/posts/deploy-elixir-generated-html-with-docker-on-digitalocean.html#create-the-mix-project) previously, if necessary. + +``` +$ docker run --rm -w /opt -v $PWD:/opt -u $(id -u):$(id -u) elixir mix new serval --sup +``` + +Then, we initialize Git in our project directory. + +``` +$ cd serval +$ git init +``` + +Configure the user name and email, if necessary. + +``` +$ git config user.email "webdevcat@proton.me" +$ git config user.name "Catalin Mititiuc" +``` + +Lastly, create the initial commit. + +``` +$ git add . +$ git commit -m "Initial commit" +``` + +#### 2. Create a GitHub repo + +After creating the remote repo on GitHub, we add it as the origin and push up our initial commit. + +``` +$ git remote add origin git@github.com:webdevcat-me/serval.git +$ git push -u origin master +``` + +#### 3. Add an `index.html` file to the root directory + +Just a simple HTML boilerplate, for now. + +::: filename-for-code-block +`index.html` +::: + +```html + + + + + + + +Lorem ipsum dolor sit amet, consectetur adipiscing elit.
+index.html