---
title: "Publish Markdown Documents As Static Web Pages with Pandoc and Phoenix"
blurb: "We thought we wanted a static website generator. It turns out what we
really wanted was Phoenix, with an option to convert markdown to HTML. Here is
our implementation of a solution, using our very own, recently-released,
Pandoc Hex package!"
...
## Introduction
A short while ago, we published our latest version of a [`pandoc` installer Hex
package](https://hex.pm/packages/pandoc), modeled after the existing
[`tailwind`](https://hex.pm/packages/tailwind) and
[`esbuild`](https://hex.pm/packages/esbuild) packages. In this post, we will
show how we used our `pandoc` package to add our current markdown publishing
solution to the Phoenix Framework.
## 1. Generate a new Phoenix project and add the `pandoc` dependency
Let's start with a new project. We will use the `--no-ecto` option because we
don't need a database for this project.
$ mix phx.new hello --no-ecto
Next we add `pandoc` as a dependency to our `mix.exs` file.
::: filename-for-code-block
`mix.exs`
:::
```elixir
{:pandoc, "~> 0.3", only: :dev}
```
Then we fetch our dependencies.
$ mix deps.get
## 2. Configure `pandoc`
Because the goal is to have multiple documents, the name of the output file
will depend on the name of the input file. For this reason, we cannot simply
use a static value for the `--output` option. To deal with this problem,
`pandoc` accepts a function for the `args` config key that allows us to set the
output filename dynamically for each document.
::: filename-for-code-block
`config/config.exs`
:::
```elixir
if config_env() != :prod do
# Configure pandoc (the version is required)
config :pandoc,
version: "3.6.1",
hello: [
args: fn extra_args ->
{_, [input_file], _} = OptionParser.parse(extra_args, switches: [])
~w(--output=../priv/static/posts/#{Path.rootname(input_file)}.html)
end,
cd: Path.expand("../documents", __DIR__)
]
end
```
Because [anonymous functions are not supported in
releases](https://elixirforum.com/t/mix-do-compile-release-could-not-read-configuration-file-config-runtime-exs-not-found/37800/3),
we can wrap our config in a `if config_env() != :prod` conditional, since we'll
only convert markdown to HTML at build time.
Next, we create the directory where our converted HTML documents will live.
$ mkdir priv/static/posts
And we add our new directory to `.gitignore` so that Git doesn't save the HTML
output of our documents.
::: filename-for-code-block
`.gitignore`
:::
```
# Ignore documents that are produced by pandoc.
/priv/static/posts/
```
Lastly, we add our `pandoc` watcher to the endpoint so that, in development,
any changes to our documents' markdown will reflect in the browser in
real-time.
::: filename-for-code-block
`config/dev.exs`
:::
```elixir
config :hello, HelloWeb.Endpoint,
# ...
watchers: [
# ...
pandoc: {Pandoc, :watch, [:hello]}
]
```
To make sure everything is working, we can give it a quick test:
```bash
$ mkdir documents
$ echo "# hello" > documents/hello.md
$ mix pandoc hello hello.md
... [debug] Downloading pandoc from ...
$ cat priv/static/posts/hello.html
hello
```
## 3. Add new document aliases to `mix.exs`
Now that we have Pandoc installed, and a Mix task to convert a document from
markdown to HTML, we need a way to call it on all the documents in the
`documents` directory. We can do this by adding a new alias in `mix.exs` that
will scan the directory for files and call the Pandoc Mix task on each.
::: filename-for-code-block
`mix.exs`
:::
```elixir
defp aliases do
[
setup: [
"deps.get",
"assets.setup",
"assets.build",
"documents.setup",
"documents.build"
],
# ...
"documents.setup": ["pandoc.install --if-missing"],
"documents.build": &pandoc/1,
"statics.deploy": ["assets.deploy", "documents.build"]
]
end
defp pandoc(_) do
config = Application.get_env(:pandoc, :hello)
cd = config[:cd] || File.cwd!()
cd
|> File.cd!(fn ->
Enum.filter(File.ls!(), &(File.stat!(&1).type != :directory))
end)
|> Enum.each(&Mix.Task.rerun("pandoc", ["hello", &1]))
end
```
Now when we run `mix setup`, Pandoc will convert all the files in our documents
directory to markup and place the output in `priv/static/posts`.
We also added a `statics.deploy` alias so we'll only have to run a single task
before we build a release.
## 4. Add context, controller, templates, and routes
Now that we have our documents in `priv/static/posts` in HTML format, we need a
way to render them.
We start with a struct that will hold the values for each post.
::: filename-for-code-block
`lib/hello/documents/post.ex`
:::
```elixir
defmodule Hello.Documents.Post do
defstruct [:id, :path, :body]
end
```
Next, we add our Documents context that will be responsible for fetching a list
of all posts as well as each individual post.
We have chosen to name our documents using hyphens (`-`) to separate words. Our
post ids, then, will be the document filename minus the file extension suffix.
So, a file `documents/this-is-the-first-post.md` will have an id of
`this-is-the-first-post` and a URI that looks like
`https://example.org/posts/this-is-the-first-post`.
::: filename-for-code-block
`lib/hello/documents.ex`
:::
```elixir
defmodule Hello.Documents do
@moduledoc """
The Documents context.
"""
alias Hello.Documents.Post
def list_posts do
"documents/*"
|> Path.wildcard()
|> Enum.map(fn path ->
%Post{
id: path |> Path.rootname() |> Path.basename(),
path: path
}
end)
end
def get_post!(id) do
post = Enum.find(list_posts(), fn post -> post.id == id end)
body =
:hello
|> :code.priv_dir()
|> Path.join("static/posts/#{id}.html")
|> File.read!()
%{post | body: body}
end
end
```
Our posts controller and view are pretty standard.
::: filename-for-code-block
`lib/hello_web/controllers/post_controller.ex`
:::
```elixir
defmodule HelloWeb.PostController do
use HelloWeb, :controller
alias Hello.Documents
def index(conn, _params) do
posts = Documents.list_posts()
render(conn, :index, posts: posts)
end
def show(conn, %{"id" => id}) do
post = Documents.get_post!(id)
render(conn, :show, post: post)
end
end
```
::: filename-for-code-block
`lib/hello_web/controllers/post_html.ex`
:::
```elixir
defmodule HelloWeb.PostHTML do
use HelloWeb, :html
embed_templates "post_html/*"
end
```
Our `index` and `show` templates are pretty similar to what the Phoenix
`phx.gen.html` HTML generator spits out. We take full advantage of the UI core
components that Phoenix provides.
::: filename-for-code-block
`lib/hello_web/controllers/post_html/index.html.heex`
:::
```heex
<.header>
Listing Posts
<.table id="posts" rows={@posts} row_click={&JS.navigate(~p"/posts/#{&1}")}>
<:col :let={post} label="id"><%= post.id %>
<:action :let={post}>
<.link navigate={~p"/posts/#{post}"}>Show
```
::: filename-for-code-block
`lib/hello_web/controllers/post_html/show.html.heex`
:::
```heex
<.header>
<%= @post.id %>
<:subtitle>This is a post from your markdown documents.
<%= raw(@post.body) %>
<.back navigate={~p"/posts"}>Back to posts
```
Finally, we add the routes we will need, with only the `index` and `show`
actions being necessary.
::: filename-for-code-block
`lib/hello_web/router.ex`
:::
```elixir
scope "/", HelloWeb do
pipe_through :browser
resources "/posts", PostController, only: [:index, :show]
get "/", PageController, :home
end
```
Let's create a couple of documents to see how our app renders them.
```bash
$ touch documents/{hello-there.md,welcome-to-our-demo-app.md}
```
And now we visit `localhost:4000/posts`.
Let's add some markdown content to `documents/hello-there.md` so we can see how
the `show` template looks.
::: filename-for-code-block
`documents/hello-there.md`
:::
```markdown
# hello
This is a paragraph.
This is a code block.
> This is a block quote.
## this is a heading
- this is
- a list
- of items
```
When we visit `localhost:4000/posts/hello-there`, it looks like this:
Our document's content is visible, but it's missing any styling. We will fix
this by adding the `typography` plugin to Tailwind's config file.
## 5. Style the markdown content
Let's add `@tailwindcss/typography` to the plugins list in
`tailwind.config.js`.
::: filename-for-code-block
`assets/tailwind.config.js`
:::
```javascript
plugins: [
// ...
require("@tailwindcss/typography")
]
```
Then we'll add Tailwind's `prose` class to our post's HTML content.
::: filename-for-code-block
`lib/hello_web/controllers/post_html/show.html.heex`
:::
```heex
<%= raw(@post.body) %>
```
After restarting our app, we can visit the post `show` page again, and this
time our content is styled appropriately.
## 6. Auto-reload the browser when markdown content changes
Since our `pandoc` file-watcher converts documents to HTML whenever changes are
detected, all we need to do to have the changes update in our browser in
real-time is add the static files path to the `live_reload` config in
`config/dev.exs`.
::: warning
**Caveat: Long Documents**
If the documents we are editing become long, there are two issues that may
arise.
1. If the page reloads, but our document content is missing, that means the
live-reloader reloaded the page before the document was finished being
converted. To address this, we can use the `interval` option to set a lengh
of time greater than the `100` ms default value.
2. If our terminal is being flooded with too many `[debug] Live reload...`
messages, we can use the `debounce` option to set a delay before a reload
event is sent to the browser.
:::
::: filename-for-code-block
`config/dev.exs`
:::
```elixir
config :hello, HelloWeb.Endpoint,
live_reload: [
interval: 1000,
debounce: 200,
patterns: [
# ...
~r"priv/static/posts/.*(html)$"
]
]
```
Now we can edit the markdown in our documents and, as soon as we save our
changes, the new content should appear in our browser.
## 7. Handle draft documents
It would be very helpful if we had a place to keep draft documents that are
visible to us during development but absent in a production release. With a few
changes to `lib/hello/documents.ex` we can do just that.
First, let's make a directory for draft documents.
```bash
$ mkdir documents/_drafts
```
So our published documents will live in the top-level `documents` directory,
and our draft documents will live in the subdirectory `documents/_drafts`.
The first change to the `list_posts/1` function in `documents.ex` adds a
conditional depending on `Mix.env()` for what directories to scan. In
production it will only scan `documents/*` for files, while in development it
will scan all subdirectories with `documents/**/*`.
The second change required to `list_posts/1` is to filter out directories,
since `Path.wildcard/1` will include the directory `_drafts` in with the list
with files.
::: filename-for-code-block
`lib/hello/documents.ex`
:::
```elixir
def list_posts do
"documents"
|> Path.join(if(Mix.env() != :prod, do: "**/*", else: "*"))
|> Path.wildcard()
|> Enum.filter(&(File.stat!(&1).type != :directory))
|> Enum.map(fn path ->
# ...
```
Now we just need to update the `get_post!/1` function. In order to make sure
that we never accidentally publish draft documents, we will convert them on the
fly as necessary and will store their content in memory rather that writing to
disk. Since the `pandoc` profile `default` outputs conversion results
to `stdout`, we can use that profile instead of `hello` when running the
conversion and simply capture the results with `ExUnit.CaptureIO.capture_io/1`.
::: filename-for-code-block
`lib/hello/documents.ex`
:::
```elixir
def get_post!(id) do
# ...
body =
if "_drafts" in Path.split(post.path) do
ExUnit.CaptureIO.capture_io(fn ->
Mix.Task.rerun("pandoc", ["default", post.path])
end)
else
:hello
|> :code.priv_dir()
|> Path.join("static/posts/#{id}.html")
|> File.read!()
end
# ...
end
```
Lastly, since we are not writing draft documents to disk, we need to add
another pattern for Phoenix's LiveReloader to reload the browser when draft
content changes.
::: filename-for-code-block
`config/dev.exs`
:::
```elixir
config :hello, HelloWeb.Endpoint,
live_reload: [
patterns: [
# ...
~r"priv/static/posts/.*(html)$",
~r"documents/_drafts/.*(md)$"
]
]
```
## 8. Release
When we next release our application, we simply need to run `mix
statics.deploy` to build assets and documents first, and then run the release
command.
```bash
$ mix statics.deploy
$ MIX_ENV=prod mix release
```
## Conclusion
That's it! This solution allows us to write our posts in simple markdown and
see them converted to HTML automatically with Phoenix and Pandoc. Furthermore,
we are able to use Phoenix's powerful template language HEEx to make writing
HTML faster and easier, so we can focus more on content and less on
development.
We hope this post was as useful to others as it has been to us. We are always
appreciative of any
[feedback](mailto:webdevcat@proton.me?subject=re:%20post%20publish-markdown-documents-as-static-web-pages-with-pandoc-and-phoenix)
readers would like to share with us. Thanks for reading and happy coding!