diff --git a/docs/2023-08-03-recursively-list-all-files-in-a-directory-with-elixir.md b/docs/2023-08-03-recursively-list-all-files-in-a-directory-with-elixir.md index 87171ec..dc0d96b 100644 --- a/docs/2023-08-03-recursively-list-all-files-in-a-directory-with-elixir.md +++ b/docs/2023-08-03-recursively-list-all-files-in-a-directory-with-elixir.md @@ -39,35 +39,37 @@ end The data structure holding the results of the file search. - iex(1)> directory_tree = Files.find("hello") - {"hello", - [ - "README.md", - {"hello/test", - [ - {"hello/test/support", ["conn_case.ex"]}, - "test_helper.exs", - {"hello/test/hello_web", - [{"hello/test/hello_web/controllers", ["error_json_test.exs"]}]} - ]}, - {"hello/lib", - [ - "hello.ex", - {"hello/lib/hello", ["application.ex"]}, - "hello_web.ex", - {"hello/lib/hello_web", - [ - {"hello/lib/hello_web/controllers", ["error_json.ex"]}, - "telemetry.ex", - "router.ex", - "endpoint.ex" - ]} - ]}, - {"hello/priv", [{"hello/priv/static", ["robots.txt", "favicon.ico"]}]}, - {"hello/config", - ["config.exs", "dev.exs", "test.exs", "prod.exs", "runtime.exs"]}, - "mix.exs" - ]} +``` +iex(1)> directory_tree = Files.find("hello") +{"hello", + [ + "README.md", + {"hello/test", + [ + {"hello/test/support", ["conn_case.ex"]}, + "test_helper.exs", + {"hello/test/hello_web", + [{"hello/test/hello_web/controllers", ["error_json_test.exs"]}]} + ]}, + {"hello/lib", + [ + "hello.ex", + {"hello/lib/hello", ["application.ex"]}, + "hello_web.ex", + {"hello/lib/hello_web", + [ + {"hello/lib/hello_web/controllers", ["error_json.ex"]}, + "telemetry.ex", + "router.ex", + "endpoint.ex" + ]} + ]}, + {"hello/priv", [{"hello/priv/static", ["robots.txt", "favicon.ico"]}]}, + {"hello/config", + ["config.exs", "dev.exs", "test.exs", "prod.exs", "runtime.exs"]}, + "mix.exs" + ]} +``` ## Sorting and printing @@ -101,28 +103,30 @@ end Print all the files sorted. - iex(2)> Paths.puts(directory_tree) - hello/mix.exs - hello/README.md - hello/config/config.exs - hello/config/dev.exs - hello/config/prod.exs - hello/config/runtime.exs - hello/config/test.exs - hello/lib/hello.ex - hello/lib/hello_web.ex - hello/lib/hello/application.ex - hello/lib/hello_web/endpoint.ex - hello/lib/hello_web/router.ex - hello/lib/hello_web/telemetry.ex - hello/lib/hello_web/controllers/error_json.ex - hello/priv/static/favicon.ico - hello/priv/static/robots.txt - hello/test/test_helper.exs - hello/test/hello_web/controllers/error_json_test.exs - hello/test/support/conn_case.ex - :ok +``` +iex(2)> Paths.puts(directory_tree) +hello/mix.exs +hello/README.md +hello/config/config.exs +hello/config/dev.exs +hello/config/prod.exs +hello/config/runtime.exs +hello/config/test.exs +hello/lib/hello.ex +hello/lib/hello_web.ex +hello/lib/hello/application.ex +hello/lib/hello_web/endpoint.ex +hello/lib/hello_web/router.ex +hello/lib/hello_web/telemetry.ex +hello/lib/hello_web/controllers/error_json.ex +hello/priv/static/favicon.ico +hello/priv/static/robots.txt +hello/test/test_helper.exs +hello/test/hello_web/controllers/error_json_test.exs +hello/test/support/conn_case.ex +:ok +``` ## Conclusion -Trying to do this without recursion made it difficult to sort directories first. +Trying to do this without recursion made it difficult to sort directories first. \ No newline at end of file diff --git a/docs/2023-09-15-open-an-iex-shell-from-an-elixir-script.md b/docs/2023-09-15-open-an-iex-shell-from-an-elixir-script.md index 46da21c..20a4aa1 100644 --- a/docs/2023-09-15-open-an-iex-shell-from-an-elixir-script.md +++ b/docs/2023-09-15-open-an-iex-shell-from-an-elixir-script.md @@ -1,7 +1,9 @@ -{ - title: "Open An IEx Shell From An Elixir Script" - blurb: "We can run an Elixir script with either the elixir or the iex command. Both will execute the code, but the second command opens an interactive IEx shell afterward. What if, we won't know until runtime whether we want a shell or not? How can we start an IEx session even when we use elixir, instead of iex, to run our script?" -} +--- +title: "Open An IEx Shell From An Elixir Script" +blurb: "We can run an Elixir script with either the elixir or the iex command. Both will execute the code, but the second command opens an interactive IEx shell afterward. What if, we won't know until runtime whether we want a shell or not? How can we start an IEx session even when we use elixir, instead of iex, to run our script?" +... + +I recently had occasion to want to start an IEx session from an Elixir script. Here's how I was able to do it. ## Method 1 diff --git a/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md b/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md index c1211f5..a02532a 100644 --- a/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md +++ b/docs/2023-10-08-start-erlangs-dialyzer-with-gui-from-a-docker-container.md @@ -1,7 +1,7 @@ -{ - title: "Start Erlang's Dialyzer With GUI From A Docker Container" - blurb: "Everything in OTP is command-line driven, so using containers during development has been without issue. But, Dialyzer, Erlang's static analysis tool, actually has a Graphical User Interface. How can we still use Dialyzer and its GUI even though Elixir is running inside a container?" -} +--- +title: "Start Erlang's Dialyzer With GUI From A Docker Container" +blurb: "Everything in OTP is command-line driven, so using containers during development has been without issue. But, Dialyzer, Erlang's static analysis tool, actually has a Graphical User Interface. How can we still use Dialyzer and its GUI even though Elixir is running inside a container?" +... I use Docker mostly when working on software projects and I figured out how to get Erlang's Dialyzer GUI working in a Docker container. @@ -44,4 +44,4 @@ I use Docker mostly when working on software projects and I figured out how to g You should now see the Dialyzer GUI. -![dialyzer application window](/images/dialyzer.png) +![dialyzer application window](/images/dialyzer.png) \ No newline at end of file diff --git a/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md b/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md index 6b759aa..e241994 100644 --- a/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md +++ b/docs/2023-10-16-fix-distortion-introduced-when-transforming-multiview-projections-to-isometric.md @@ -1,6 +1,6 @@ -{ - blurb: "One thing we learned from a week of trying to make isometric vector drawings." -} +--- +blurb: "One thing we learned from a week of trying to make isometric vector drawings." +... ## Objective diff --git a/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md b/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md index 59dcc01..e3b98fe 100644 --- a/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md +++ b/docs/2023-11-01-deploy-elixir-generated-html-with-docker-on-digitalocean.md @@ -1,7 +1,7 @@ -{ - title: "Deploy Elixir-Generated HTML With Docker On DigitalOcean" - blurb: "This is a simple proof of concept where we create a boilerplate HTML file with Elixir, containerize our build process with Docker, and deploy our markup live with DigitalOcean's hosting service." -} +--- +title: "Deploy Elixir-Generated HTML With Docker On DigitalOcean" +blurb: "This is a simple proof of concept where we create a boilerplate HTML file with Elixir, containerize our build process with Docker, and deploy our markup live with DigitalOcean's hosting service." +... ## Introduction @@ -361,4 +361,4 @@ If we want to make changes, we can commit our updates and push the code to GitHu ## Conclusion -Our Elixir-generated HTML file is live and hosted. We have completed our proof of concept. If we wanted to take advantage of DigitalOcean's platform and host an Elixir-generated static website, this is the blueprint we could follow. It's relatively simple if familiar with Docker, and once set up, deploying changes with a `git push` is simply magical. We look forward to using what we have learned in a future project. +Our Elixir-generated HTML file is live and hosted. We have completed our proof of concept. If we wanted to take advantage of DigitalOcean's platform and host an Elixir-generated static website, this is the blueprint we could follow. It's relatively simple if familiar with Docker, and once set up, deploying changes with a `git push` is simply magical. We look forward to using what we have learned in a future project. \ No newline at end of file diff --git a/docs/2023-11-15-test-mix-task-file-modify.md b/docs/2023-11-15-test-mix-task-file-modify.md index 04e9646..8b01856 100644 --- a/docs/2023-11-15-test-mix-task-file-modify.md +++ b/docs/2023-11-15-test-mix-task-file-modify.md @@ -1,7 +1,7 @@ -{ - title: "Temporary Directories For Testing Mix Tasks That Modify Files" - blurb: "Writing a test for a simple Mix task gets surprisingly complex. Application environment variables, temporary test directories, and IO capture are all involved." -} +--- +title: "Temporary Directories For Testing Mix Tasks That Modify Files" +blurb: "Writing a test for a simple Mix task gets surprisingly complex. Application environment variables, temporary test directories, and IO capture are all involved." +... ## Intro @@ -319,4 +319,4 @@ end ## Conclusion -This test seemed trivial at first, but increased in complexity quickly. We had to set some of our configuration in application environment variables, change the configuration temporarily before a test run and then change it back after, clean up test artifacts after the run, and capture IO messages that were generated during it. That's enough going on that we thought it would be a good topic for a post. Cheers and happy coding! +This test seemed trivial at first, but increased in complexity quickly. We had to set some of our configuration in application environment variables, change the configuration temporarily before a test run and then change it back after, clean up test artifacts after the run, and capture IO messages that were generated during it. That's enough going on that we thought it would be a good topic for a post. Cheers and happy coding! \ No newline at end of file diff --git a/docs/2023-12-01-build-static-website-generator-part-1.md b/docs/2023-12-01-build-static-website-generator-part-1.md index 1750ff5..e84c44f 100644 --- a/docs/2023-12-01-build-static-website-generator-part-1.md +++ b/docs/2023-12-01-build-static-website-generator-part-1.md @@ -1,9 +1,8 @@ -{ - title: "Build A Static-Website Generator With Elixir, Part 1" - blurb: "We take the first steps in designing and implementing the \"world's simplest static-website generator\". Building on tools and knowledge we acquired previously, and utilizing an incremental and iterative development process, we go through the entire software life-cycle from creating the initial project files to deploying to production. We spare nothing, from spelling out every command, to ensuring application integrity with tests, and even updating the README file. Grab a drink and some snacks, and dive right in!" -} - -
$markdown{[[ +--- +title: "Build A Static-Website Generator With Elixir, Part 1" +blurb: "We take the first steps in designing and implementing the \"world's simplest static-website generator\". Building on tools and knowledge we acquired previously, and utilizing an incremental and iterative development process, we go through the entire software life-cycle from creating the initial project files to deploying to production. We spare nothing, from spelling out every command, to ensuring application integrity with tests, and even updating the README file. Grab a drink and some snacks, and dive right in!" +... +::: info This post was originally intended to be the first in a multi-part series. However, the deeper we got into this project, the more we realized we were basically implementing our own version of a web framework. Rather than @@ -12,7 +11,7 @@ framework, [Phoenix](https://www.phoenixframework.org/), and simply added a Markdown-to-HTML conversion feature. As a consequence, there are no other parts to this post, but a description of the solution we chose instead can be found [here](/posts/publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix). -]]}
+::: This is one of our longer posts, so we've included a table of contents. diff --git a/docs/2024-01-01-build-a-neovim-qt-appimage-from-source.md b/docs/2024-01-01-build-a-neovim-qt-appimage-from-source.md new file mode 100644 index 0000000..831dd92 --- /dev/null +++ b/docs/2024-01-01-build-a-neovim-qt-appimage-from-source.md @@ -0,0 +1,152 @@ +--- +title: "Build A Neovim Qt AppImage from Source" +blurb: "Building an AppImage package from source allows us to run the +latest version of Neovim-QT on our machine running the Debian Linux +distribution." +... + +![Neovim Qt](/images/neovim-qt-sample.png) + +## Introduction + +We have [Debian](https://www.debian.org/) installed on our machine and would like to run [Neovim](https://neovim.io/) with the latest version of the [Neovim Qt](https://github.com/equalsraf/neovim-qt) GUI. While a Debian `neovim-qt` package exists, it is not the latest version. To solve this problem, we will build Neovim Qt from source and package it as an AppImage. + +### Requirements + +We will assume Neovim is already installed. Neovim AppImages are available from the [Neovim GitHub repo](https://github.com/neovim/neovim/releases). + +## 1. Install [Toolbox](https://containertoolbx.org/) + +We will have to install all the Neovim Qt build dependencies, so we will use Toolbox to build in a container, keeping our system clean. + + $ sudo apt-get update + $ sudo apt-get install podman-toolbox + +## 2. Download Neovim Qt + +We start by downloading the latest `*.tar.gz` source code asset from the [Neovim Qt GitHub repository](https://github.com/equalsraf/neovim-qt/releases/). + +Then, we unpack and unzip it and `cd` into the directory. + + $ tar -xzvf neovim-qt-0.2.18.tar.gz + $ cd neovim-qt-0.2.18 + +## 3. Create and enter a new Toolbox container + + $ toolbox create --distro debian neovim-qt + $ toolbox enter neovim-qt + +## 4. Add `deb-src` to the sources list + +Toolbox's base Debian image only lists binary archive types in the sources list. We will have to add the archive type for source packages, as well. + + $ sudo nano /etc/apt/sources.list.d/debian.sources + +We change the two lines that read `Types: deb` to `Types: deb deb-src` and save the changes. + + Types: deb deb-src + ... + + Types: deb deb-src + ... + +## 5. Install build dependencies + +We can install all the build dependencies we will need to build Neovim Qt with the `build-dep` option. `fuse` will be needed to build the AppImage package. + + $ sudo apt-get update + $ sudo apt-get build-dep neovim-qt + $ sudo apt-get install fuse + +## 6. Add a build script + +We copy a sample `cmake` build-script from the [appimage.org online documentation](https://docs.appimage.org/packaging-guide/from-source/native-binaries.html#bundle-qtquickapp-with-cmake) into a file called `build-with-cmake.sh`. + +Alternatively, we can download it directly from their GitHub repository with + + $ wget https://raw.githubusercontent.com/linuxdeploy/QtQuickApp/master/travis/build-with-cmake.sh + +We need to make two changes. + +1. On the line that contains + + cmake "$REPO_ROOT" -DCMAKE_INSTALL_PREFIX=/usr + + we add the variable `DCMAKE_BUILD_TYPE` and set it to `Release` (per the [Neovim Qt build instructions](https://github.com/equalsraf/neovim-qt/wiki/Build-Instructions)): + + cmake "$REPO_ROOT" -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release + +2. On the very last line, + + mv QtQuickApp*.AppImage "$OLD_CWD" + + we remove the sample app name, `QtQuickApp`. + + mv *.AppImage "$OLD_CWD" + + (We could, optionally, set it to `mv Neovim-Qt*.AppImage "$OLD_CWD"`, but for our case, it's not necessary). + +## 7. Run the build script + +We make the script runnable and then run it. + + $ chmod +x build-with-cmake.sh + $ ./build-with-cmake.sh + +## 8. Test-run the AppImage + +We should now have an AppImage package in our directory that we can run. + + $ ./Neovim-Qt-x86_64.AppImage + +When we run it we should see Neovim open in a new GUI window. + +![Neovim Qt Running In a Window](/images/neovim-qt-test-run.png) + +## 9. Exit the Toolbox container + + $ exit + logout + +The Toolbox container is still running. We can stop it with + + $ podman stop neovim-qt + +## 10. Add the package to our user-specific executable directory + +According to the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html), user-specific executables belong in `$HOME/.local/bin`. We will place our AppImage in its own directory in `~/.local/neovim-qt` and create a symlink to it in `~/.local/bin`. + + $ mkdir ~/.local/neovim-qt + $ mv ./Neovim-Qt-x86_64.AppImage ~/.local/neovim-qt + $ ln -s ~/.local/neovim-qt/Neovim-Qt-x86_64.AppImage ~/.local/bin/nvim-qt + +We can now run it by calling `nvim-qt` directly from the command line. + + $ nvim-qt --version + NVIM-QT v0.2.18.0 + Build type: Release + Compilation: -Wall -Wextra -Wno-unused-parameter -Wunused-variable + Qt Version: 5.15.8 + ... + +## 11. Add Neovim Qt to the applications menu + +We simply need to copy the `.desktop` file from the source directory. + + $ cp src/gui/nvim-qt.desktop ~/.local/share/applications/ + +## 12. Add an icon + +And finally, we copy over the icon from the source directory as well. + + $ mkdir ~/.local/neovim-qt/icons + $ cp third-party/neovim.png ~/.local/neovim-qt/icons/nvim-qt.png + $ mkdir -p ~/.local/share/icons/hicolor/192x192/apps + $ ln -s ~/.local/neovim-qt/icons/nvim-qt.png ~/.local/share/icons/hicolor/192x192/apps/ + $ xdg-icon-resource forceupdate --mode user + +## Conclusion + +When we search for `neovim` in our applications menu, we should now see an entry we can use to start our new Neovim Qt AppImage. + +![Neovim Qt Application Menu Entry](/images/neovim-qt.png) diff --git a/docs/2024-11-03-set-up-a-gitweb-server.md b/docs/2024-11-03-set-up-a-gitweb-server.md new file mode 100644 index 0000000..5e442d3 --- /dev/null +++ b/docs/2024-11-03-set-up-a-gitweb-server.md @@ -0,0 +1,175 @@ +--- +title: "Set Up A GitWeb Server" +blurb: "Set up a VPS with a simple, web-based code repository visualizer using +Lighttpd and GitWeb." +... +## Introduction + +Git comes with a CGI script called GitWeb, a simple web-based visualizer. +Today we will set up a virtual server on DigitalOcean that will use GitWeb to +visualize a Git repository. + +## 1. Create a DigitalOcean droplet + +Create a new droplet from the DigitalOcean dashboard. For now, we go with the +smallest virtual server currently available. We also add an SSH key so we can +authenticate without a password. + +![DigitalOcean Droplet Creation Form, Image & Size Selection](/images/git-server-do-droplet.png) + +## 2. Log in to the droplet remotely + +After the droplet is created, we can see its IP address on the dashboard. We +use this IP address to log in to our virtual server: + + $ ssh root@XX.XX.XXX.XXX + +## 3. Silence locale warnings + +After successfully logging in, one of the messages we are greeted with is a +warning about the locale. + + -bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) + +We can make this annoying message go away by creating a locale file manually: + + # localedef -i en_US -f UTF-8 en_US.UTF-8 + +## 4. Add a user account to administer the Git repositories + +Next, we create a user account that will administer the git repositories, so we +don't always have to do it as `root`. + + # adduser git --disabled-password + +We have to add our SSH key to the `git` user's `authorized_keys` file. + + # su git + $ cd + $ mkdir .ssh && chmod 700 .ssh + $ touch .ssh/authorized_keys && chmod 600 .ssh/authorized_keys + $ exit + # cat ~/.ssh/authorized_keys >> /home/git/.ssh/authorized_keys + +## 5. Install necessary packages + +Lighttpd is the default web server that GitWeb tries to use if available. For +simplicity, that's what we'll use. + + # apt-get update + # apt-get install git lighttpd gitweb + +## 6. Configure Lighttpd + +We will need to make some changes to Lighttpd's config file. + + # nano /etc/lighttpd/lighttpd.conf + +1. Update the value for `server.document-root`: + + `server.document-root = "/usr/share/gitweb"` + +2. Add `index.cgi` to `index-file.names`: + + `index-file.names = ( "index.php", "index.html", "index.cgi" )` + +## 7. Enable Lighttpd modules + +Since GitWeb uses CGI, we will have to enable Lighttpd's CGI module. We will +also need the setenv module. First we need to configure them. + + # nano /etc/lighttpd/conf-available/05-setenv.conf + +Add the following line: + +::: filename-for-code-block +`/etc/lighttpd/conf-available/05-setenv.conf` +::: + + ... + setenv.add-environment = ( "PATH" => env.PATH, "GITWEB_CONFIG" => env.GITWEB_CONFIG ) + +Next, edit the CGI module config file. + + # nano /etc/lighttpd/conf-available/10-cgi.conf + +Add: + +::: filename-for-code-block +`/etc/lighttpd/conf-available/10-cgi.conf` +::: + + ... + cgi.assign = ( + ".cgi" => "" + ) + +Once that's done, enable `mod_cgi` and `mod_setenv` with: + + # lighty-enable-mod cgi setenv + # service lighttpd force-reload + +## 8. Edit the Lighttpd service init files + +We need to edit the Lighttpd service startup files to define the +`GITWEB_CONFIG` environment variable that we used in the previous step. + + # systemctl edit lighttpd.service + +This will start up an editor and create an `override.conf` file. Add the +following two lines: + +::: filename-for-code-block +`/etc/systemd/system/lighttpd.service.d/override.conf` +::: + + [Service] + Environment="GITWEB_CONFIG=./gitweb_config.perl" + +Then, save the file and exit the editor. To finish, we need to run these two +commands: + + # systemctl daemon-reload + # service lighttpd restart + +## 9. Upload a Git repository + +We are now ready to upload a Git respository we wish to visualize with our +server. First, lets transfer ownership of the respository directory to our user +`git` we created earlier. + + # chown git /var/lib/git/ + # chgrp git /var/lib/git/ + +Now we can log out of the server with `exit`. On our local machine, we clone a +`bare` copy of the repo we want to upload. + + $ git clone --bare my_project my_project.git + +Now we can upload this bare repo to our server. + + $ scp -r my_project.git git@XX.XX.XXX.XXX:/var/lib/git + +We can tell Git to automatically add group write permissions to our repo with: + + $ ssh git@XX.XX.XXX.XXX + $ cd /var/lib/git/my_project.git + $ git init --bare --shared + +## 10. Visit the GitWeb server + +When we visit the server's IP address with our browser, `http://XX.XX.XXX.XXX`, +we should see a GitWeb `projects` page. We can now explore our project's code +with our web browser. + +![GitWeb Projects List](/images/git-server-gitweb.png) + +## Resources + +Here is a list of the resources that were used to figure out how to accomplish +the task in this post. + +- [Git Documentation Book - Chapter 4: Git on the Server](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols) +- [Lighttpd Configuration Tutorial](https://redmine.lighttpd.net/projects/lighttpd/wiki/TutorialConfiguration) +- [LinuxQuestions.org - Perl CGI:Can't locate CGI.pm](https://www.linuxquestions.org/questions/programming-9/perl-cgi-can%27t-locate-cgi-pm-330706/) +- [Nicketa's GitHub Gist - LC_CTYPE.md](https://gist.github.com/nicks9188/a19f39d62780055a68c22b89a9799c25) diff --git a/docs/2024-11-11-resize-a-qemu-disk-image.md b/docs/2024-11-11-resize-a-qemu-disk-image.md new file mode 100644 index 0000000..3eaa0b7 --- /dev/null +++ b/docs/2024-11-11-resize-a-qemu-disk-image.md @@ -0,0 +1,41 @@ +--- +title: "Resize A QEMU Disk Image" +blurb: "Our hosting provider allows us to upload a custom image when +provisioning a new server. We will have to resize the image if it does not +match the size of the server's disk space." +... +## 1. Install `guestfs-tools` + + $ apt-get update + $ apt-get install guestfs-tools + +## 2. Download an image + +Download an image from [https://cdimage.debian.org/images/cloud/](https://cdimage.debian.org/images/cloud/). + +## 3. List partitions + +Check the image disk partitions to see which partition needs resizing. + + $ virt-filesystems --long -h --all -a debian-12-nocloud-amd64.qcow2 + +## 4. Create output container + + $ qemu-img create -f qcow2 \ + -o preallocation=metadata debian-12-nocloud-amd64-resized.qcow2 20G + +## 5. Generate resized image + + $ virt-resize --expand /dev/sda1 \ + debian-12-nocloud-amd64.qcow2 debian-12-nocloud-amd64-resized.qcow2 + +## Conclusion + +You can now open `debian-12-nocloud-amd64-resized.qcow2` with a virtual machine +program and the disk size will be 20G. + +## References + +`virt-resize` manpage + + $ man virt-resize diff --git a/docs/2025-01-18-publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix.md b/docs/2025-01-18-publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix.md new file mode 100644 index 0000000..a15fe0a --- /dev/null +++ b/docs/2025-01-18-publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix.md @@ -0,0 +1,540 @@ +--- +title: "Publish Markdown Documents As Static Web Pages with Pandoc and Phoenix" +blurb: "We thought we wanted a static website generator. It turns out what we +really wanted was Phoenix, with an option to convert markdown to HTML. Here is +our implementation of a solution, using our very own, recently-released, +Pandoc Hex package!" +... + +## Introduction + +A short while ago, we published our latest version of a [`pandoc` installer Hex +package](https://hex.pm/packages/pandoc), modeled after the existing +[`tailwind`](https://hex.pm/packages/tailwind) and +[`esbuild`](https://hex.pm/packages/esbuild) packages. In this post, we will +show how we used our `pandoc` package to add our current markdown publishing +solution to the Phoenix Framework. + +## 1. Generate a new Phoenix project and add the `pandoc` dependency + +Let's start with a new project. We will use the `--no-ecto` option because we +don't need a database for this project. + + $ mix phx.new hello --no-ecto + +Next we add `pandoc` as a dependency to our `mix.exs` file. + +::: filename-for-code-block +`mix.exs` +::: + +```elixir +{:pandoc, "~> 0.3", only: :dev} +``` + +Then we fetch our dependencies. + + $ mix deps.get + +## 2. Configure `pandoc` + +Because the goal is to have multiple documents, the name of the output file +will depend on the name of the input file. For this reason, we cannot simply +use a static value for the `--output` option. To deal with this problem, +`pandoc` accepts a function for the `args` config key that allows us to set the +output filename dynamically for each document. + +::: filename-for-code-block +`config/config.exs` +::: + +```elixir +if config_env() != :prod do + # Configure pandoc (the version is required) + config :pandoc, + version: "3.6.1", + hello: [ + args: fn extra_args -> + {_, [input_file], _} = OptionParser.parse(extra_args, switches: []) + ~w(--output=../priv/static/posts/#{Path.rootname(input_file)}.html) + end, + cd: Path.expand("../documents", __DIR__) + ] +end +``` + +Because [anonymous functions are not supported in +releases](https://elixirforum.com/t/mix-do-compile-release-could-not-read-configuration-file-config-runtime-exs-not-found/37800/3), +we can wrap our config in a `if config_env() != :prod` conditional, since we'll +only convert markdown to HTML at build time. + +Next, we create the directory where our converted HTML documents will live. + + $ mkdir priv/static/posts + +And we add our new directory to `.gitignore` so that Git doesn't save the HTML +output of our documents. + +::: filename-for-code-block +`.gitignore` +::: + +``` +# Ignore documents that are produced by pandoc. +/priv/static/posts/ +``` + +Lastly, we add our `pandoc` watcher to the endpoint so that, in development, +any changes to our documents' markdown will reflect in the browser in +real-time. + +::: filename-for-code-block +`config/dev.exs` +::: + +```elixir +config :hello, HelloWeb.Endpoint, + # ... + watchers: [ + # ... + pandoc: {Pandoc, :watch, [:hello]} + ] +``` + +To make sure everything is working, we can give it a quick test: + +```bash +$ mkdir documents +$ echo "# hello" > documents/hello.md +$ mix pandoc hello hello.md +... [debug] Downloading pandoc from ... +$ cat priv/static/posts/hello.html +

hello

+``` + +## 3. Add new document aliases to `mix.exs` + +Now that we have Pandoc installed, and a Mix task to convert a document from +markdown to HTML, we need a way to call it on all the documents in the +`documents` directory. We can do this by adding a new alias in `mix.exs` that +will scan the directory for files and call the Pandoc Mix task on each. + +::: filename-for-code-block +`mix.exs` +::: + +```elixir +defp aliases do + [ + setup: [ + "deps.get", + "assets.setup", + "assets.build", + "documents.setup", + "documents.build" + ], + # ... + "documents.setup": ["pandoc.install --if-missing"], + "documents.build": &pandoc/1, + "statics.deploy": ["assets.deploy", "documents.build"] + ] +end + +defp pandoc(_) do + config = Application.get_env(:pandoc, :hello) + cd = config[:cd] || File.cwd!() + + cd + |> File.cd!(fn -> + Enum.filter(File.ls!(), &(File.stat!(&1).type != :directory)) + end) + |> Enum.each(&Mix.Task.rerun("pandoc", ["hello", &1])) +end +``` + +Now when we run `mix setup`, Pandoc will convert all the files in our documents +directory to markup and place the output in `priv/static/posts`. + +We also added a `statics.deploy` alias so we'll only have to run a single task +before we build a release. + +## 4. Add context, controller, templates, and routes + +Now that we have our documents in `priv/static/posts` in HTML format, we need a +way to render them. + +We start with a struct that will hold the values for each post. + +::: filename-for-code-block +`lib/hello/documents/post.ex` +::: + +```elixir +defmodule Hello.Documents.Post do + defstruct [:id, :path, :body] +end +``` + +Next, we add our Documents context that will be responsible for fetching a list +of all posts as well as each individual post. + +We have chosen to name our documents using hyphens (`-`) to separate words. Our +post ids, then, will be the document filename minus the file extension suffix. +So, a file `documents/this-is-the-first-post.md` will have an id of +`this-is-the-first-post` and a URI that looks like +`https://example.org/posts/this-is-the-first-post`. + +::: filename-for-code-block +`lib/hello/documents.ex` +::: + +```elixir +defmodule Hello.Documents do + @moduledoc """ + The Documents context. + """ + + alias Hello.Documents.Post + + def list_posts do + "documents/*" + |> Path.wildcard() + |> Enum.map(fn path -> + %Post{ + id: path |> Path.rootname() |> Path.basename(), + path: path + } + end) + end + + def get_post!(id) do + post = Enum.find(list_posts(), fn post -> post.id == id end) + + body = + :hello + |> :code.priv_dir() + |> Path.join("static/posts/#{id}.html") + |> File.read!() + + %{post | body: body} + end +end +``` + +Our posts controller and view are pretty standard. + +::: filename-for-code-block +`lib/hello_web/controllers/post_controller.ex` +::: + +```elixir +defmodule HelloWeb.PostController do + use HelloWeb, :controller + + alias Hello.Documents + + def index(conn, _params) do + posts = Documents.list_posts() + render(conn, :index, posts: posts) + end + + def show(conn, %{"id" => id}) do + post = Documents.get_post!(id) + render(conn, :show, post: post) + end +end +``` + +::: filename-for-code-block +`lib/hello_web/controllers/post_html.ex` +::: + +```elixir +defmodule HelloWeb.PostHTML do + use HelloWeb, :html + + embed_templates "post_html/*" +end +``` + +Our `index` and `show` templates are pretty similar to what the Phoenix +`phx.gen.html` HTML generator spits out. We take full advantage of the UI core +components that Phoenix provides. + +::: filename-for-code-block +`lib/hello_web/controllers/post_html/index.html.heex` +::: + +```heex +<.header> + Listing Posts + + +<.table id="posts" rows={@posts} row_click={&JS.navigate(~p"/posts/#{&1}")}> + <:col :let={post} label="id"><%= post.id %> + <:action :let={post}> +
+ <.link navigate={~p"/posts/#{post}"}>Show +
+ + +``` + +::: filename-for-code-block +`lib/hello_web/controllers/post_html/show.html.heex` +::: + +```heex +<.header> + <%= @post.id %> + <:subtitle>This is a post from your markdown documents. + + +<%= raw(@post.body) %> + +<.back navigate={~p"/posts"}>Back to posts +``` + +Finally, we add the routes we will need, with only the `index` and `show` +actions being necessary. + +::: filename-for-code-block +`lib/hello_web/router.ex` +::: + +```elixir +scope "/", HelloWeb do + pipe_through :browser + + resources "/posts", PostController, only: [:index, :show] + + get "/", PageController, :home +end +``` + +Let's create a couple of documents to see how our app renders them. + +```bash +$ touch documents/{hello-there.md,welcome-to-our-demo-app.md} +``` + +And now we visit `localhost:4000/posts`. + +![A web browser showing the posts index page, listing all the post IDs](/images/post-pandoc-index.png) + +Let's add some markdown content to `documents/hello-there.md` so we can see how +the `show` template looks. + +::: filename-for-code-block +`documents/hello-there.md` +::: + +```markdown +# hello + +This is a paragraph. + + This is a code block. + +> This is a block quote. + +## this is a heading + +- this is +- a list +- of items +``` + +When we visit `localhost:4000/posts/hello-there`, it looks like this: + +![A web browser showing the posts show page, displaying the unstyled contents +of the document `hello-there.md` as HTML](/images/post-pandoc-show.png) + +Our document's content is visible, but it's missing any styling. We will fix +this by adding the `typography` plugin to Tailwind's config file. + +## 5. Style the markdown content + +Let's add `@tailwindcss/typography` to the plugins list in +`tailwind.config.js`. + +::: filename-for-code-block +`assets/tailwind.config.js` +::: + +```javascript +plugins: [ + // ... + require("@tailwindcss/typography") +] +``` + +Then we'll add Tailwind's `prose` class to our post's HTML content. + +::: filename-for-code-block +`lib/hello_web/controllers/post_html/show.html.heex` +::: + +```heex +
+ <%= raw(@post.body) %> +
+``` + +After restarting our app, we can visit the post `show` page again, and this +time our content is styled appropriately. + +![A web browser showing the posts show page, displaying the contents of the +document `hello-there.md` as HTML, styled appropriately](/images/post-pandoc-show-styled.png) + +## 6. Auto-reload the browser when markdown content changes + +Since our `pandoc` file-watcher converts documents to HTML whenever changes are +detected, all we need to do to have the changes update in our browser in +real-time is add the static files path to the `live_reload` config in +`config/dev.exs`. + +::: warning +**Caveat: Long Documents** + +If the documents we are editing become long, there are two issues that may +arise. + +1. If the page reloads, but our document content is missing, that means the + live-reloader reloaded the page before the document was finished being + converted. To address this, we can use the `interval` option to set a lengh + of time greater than the `100` ms default value. + +2. If our terminal is being flooded with too many `[debug] Live reload...` + messages, we can use the `debounce` option to set a delay before a reload + event is sent to the browser. +::: + +::: filename-for-code-block +`config/dev.exs` +::: + +```elixir +config :hello, HelloWeb.Endpoint, + live_reload: [ + interval: 1000, + debounce: 200, + patterns: [ + # ... + ~r"priv/static/posts/.*(html)$" + ] + ] +``` + +Now we can edit the markdown in our documents and, as soon as we save our +changes, the new content should appear in our browser. + +## 7. Handle draft documents + +It would be very helpful if we had a place to keep draft documents that are +visible to us during development but absent in a production release. With a few +changes to `lib/hello/documents.ex` we can do just that. + +First, let's make a directory for draft documents. + +```bash +$ mkdir documents/_drafts +``` + +So our published documents will live in the top-level `documents` directory, +and our draft documents will live in the subdirectory `documents/_drafts`. + +The first change to the `list_posts/1` function in `documents.ex` adds a +conditional depending on `Mix.env()` for what directories to scan. In +production it will only scan `documents/*` for files, while in development it +will scan all subdirectories with `documents/**/*`. + +The second change required to `list_posts/1` is to filter out directories, +since `Path.wildcard/1` will include the directory `_drafts` in with the list +with files. + +::: filename-for-code-block +`lib/hello/documents.ex` +::: + +```elixir +def list_posts do + "documents" + |> Path.join(if(Mix.env() != :prod, do: "**/*", else: "*")) + |> Path.wildcard() + |> Enum.filter(&(File.stat!(&1).type != :directory)) + |> Enum.map(fn path -> + # ... +``` + +Now we just need to update the `get_post!/1` function. In order to make sure +that we never accidentally publish draft documents, we will convert them on the +fly as necessary and will store their content in memory rather that writing to +disk. Since the `pandoc` profile `default` outputs conversion results +to `stdout`, we can use that profile instead of `hello` when running the +conversion and simply capture the results with `ExUnit.CaptureIO.capture_io/1`. + +::: filename-for-code-block +`lib/hello/documents.ex` +::: + +```elixir +def get_post!(id) do + # ... + body = + if "_drafts" in Path.split(post.path) do + ExUnit.CaptureIO.capture_io(fn -> + Mix.Task.rerun("pandoc", ["default", post.path]) + end) + else + :hello + |> :code.priv_dir() + |> Path.join("static/posts/#{id}.html") + |> File.read!() + end + # ... +end +``` + +Lastly, since we are not writing draft documents to disk, we need to add +another pattern for Phoenix's LiveReloader to reload the browser when draft +content changes. + +::: filename-for-code-block +`config/dev.exs` +::: + +```elixir +config :hello, HelloWeb.Endpoint, + live_reload: [ + patterns: [ + # ... + ~r"priv/static/posts/.*(html)$", + ~r"documents/_drafts/.*(md)$" + ] + ] +``` + +## 8. Release + +When we next release our application, we simply need to run `mix +statics.deploy` to build assets and documents first, and then run the release +command. + +```bash +$ mix statics.deploy +$ MIX_ENV=prod mix release +``` + +## Conclusion + +That's it! This solution allows us to write our posts in simple markdown and +see them converted to HTML automatically with Phoenix and Pandoc. Furthermore, +we are able to use Phoenix's powerful template language HEEx to make writing +HTML faster and easier, so we can focus more on content and less on +development. + +We hope this post was as useful to others as it has been to us. We are always +appreciative of any +[feedback](mailto:webdevcat@proton.me?subject=re:%20post%20publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix) +readers would like to share with us. Thanks for reading and happy coding!