ccm/miti.sh
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Publish post 'Test nginx Configuration Directives' (#3)

Browse Source 
Sitegen had some problems rendering `$host$request_uri` and `$$ct` so I had to make some changes to the Markdown renderer.

Reviewed-on: #3
This commit is contained in:
Catalin Constantin Mititiuc 2025-06-30 22:49:28 +00:00
parent 95e9be6e60
commit c0b54ad843
9 changed files with 657 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -56,9 +56,13 @@
$ make build $ make build
### build a single file
$ make build file=index.html
### start dev server ### start dev server
$ make $ make serve
Visit `localhost:8080` in web browser Visit `localhost:8080` in web browser

1
html/.gitignore vendored
View File

@ -15,4 +15,5 @@ posts/resize-a-qemu-disk-image.html
posts/set-up-a-gitweb-server.html posts/set-up-a-gitweb-server.html
posts/start-erlangs-dialyzer-with-gui-from-a-docker-container.html posts/start-erlangs-dialyzer-with-gui-from-a-docker-container.html
posts/test-mix-task-file-modify.html posts/test-mix-task-file-modify.html
posts/test-nginx-conf-directives.html
pygments.css pygments.css

BIN
html/images/openresty-default-index-page.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 49 KiB

566
posts/2025-06-30-test-nginx-conf-directives.md Normal file
View File

@ -0,0 +1,566 @@
{
title: "Test nginx Configuration Directives"
blurb: "We use MoonScript and some Lua packages to write tests for the
directives in our `nginx` configuration files."
}
$index
## Introduction
[`nginx`](https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#rewrite-uris-in-requests)
configuration can contain any number of important directives (redirects and
rewrites, for example) that need to be verified for correctness. We can write
tests for directives and run them against a test server to ensure they are
correct.
To do this, we'll use...
- [MoonScript](https://moonscript.org) and (by extension) [Lua](https://www.lua.org/) programming languages
- `nginx` we'll get from [OpenResty](https://openresty.org/en/), a web platform
created by Chinese developer, [Yichun Zhang](https://agentzh.org/)
- the [Busted testing framework](https://lunarmodules.github.io/busted/)
- the Lua package manager, [LuaRocks](https://luarocks.org/)
- a fantastic little library, [`luajit-curl`](https://bitbucket.org/senanetworksinc/luajit-curl/src/master/),
from Japanese developer [SENA Networks, Inc](https://www.sena-networks.co.jp)
- another great library, written by volunteers, [LuaSocket](https://github.com/lunarmodules/luasocket)
- our favorite container manager, [Docker Engine](https://docs.docker.com/engine/)
## Setup
Since we require LuaRocks, we'll use a Buildpack tag, which comes with it
already installed.
```console
$ docker pull openresty/openresty:bookworm-buildpack
```
Start a server on `localhost`:
```console
$ docker run --rm -it -p 80:80 openresty/openresty:bookworm-buildpack
```
We can visit `localhost` in our browser and we should see the OpenResty splash
page.
![OpenResty default nginx index page](/images/openresty-default-index-page.png)
## Get `nginx` running
First, let's [prepare the directory layout](https://openresty.org/en/getting-started.html#prepare-directory-layout).
```console
$ mkdir -p logs/ conf/conf.d/ html/
```
Next, we copy over [the default `nginx` config file](https://github.com/openresty/docker-openresty?tab=readme-ov-file#nginx-config-files).
```console
$ docker run --rm -it -w /opt -v $PWD:/opt openresty/openresty:bookworm-buildpack \
cp /etc/nginx/conf.d/default.conf /opt/conf.d/
```
Then, we update the root directive in `default.conf`:
::: filename-for-code-block
`conf/conf.d/default.conf`
:::
```diff
location / {
- root /usr/local/openresty/nginx/html;
+ root /var/www;
index index.html index.htm;
```
Now, let's add an index file.
::: filename-for-code-block
`html/index.html`
:::
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title></title>
</head>
<body>
hello world!
</body>
</html>
```
Last, we start `nginx`:
```console
$ docker run --rm -it -p 80:80 \
-v $PWD/conf/conf.d:/etc/nginx/conf.d -v $PWD/html:/var/www \
openresty/openresty:bookworm-buildpack
```
Then, in another console, this should output our index file.
```console
$ curl -v localhost
* Trying 127.0.0.1:80...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: openresty/1.27.1.2
< ...
<
<!DOCTYPE html>
<html lang="en">
...
<body>
hello world!
</body>
</html>
```
## Test an HTTP request
If we want to write a test for that request, we need some packages from
LuaRocks. Let's add a `Dockerfile` to build an image with those packages
installed.
### Add a `Dockerfile`
```Dockerfile
FROM openresty/openresty:bookworm-buildpack
WORKDIR /opt/app
RUN luarocks install moonscript
RUN luarocks install busted
RUN luarocks install luajit-curl
RUN luarocks install luasocket
```
Now let's build our image:
```console
$ docker build -t test-nginx .
```
### Write the test
Let's first make a new directory where our tests will live.
```console
$ mkdir spec
```
Our test makes a cURL request against our test server:
::: filename-for-code-block
`spec/nginx_spec.moon`
:::
```moonscript
http = require "luajit-curl-helper.http"
req = (url) ->
request = http.init url
st = request\perform!
error request\lastError! if not st
request
describe "http://localhost", ->
it "sends /index.html", ->
request = req "http://localhost"
assert.same request\statusCode!, 200
assert.same request\statusMessage!, "OK"
assert.same request\body!\match("<body>%s+(.-)%s+</body>"), "hello world!"
```
### Run the test suite
Start the test server. We're going to use `text-nginx`, the image we just
built.
```console
$ ct=$(docker run --rm -d \
-v $PWD/conf/conf.d:/etc/nginx/conf.d \
-v $PWD/html:/var/www \
-v $PWD:/opt/app \
test-nginx)
```
Start the test run:
```console
$ docker exec -t $ct busted
●
1 success / 0 failures / 0 errors / 0 pending : 0.008246 seconds
```
Stop the test server.
```console
$ docker exec $ct openresty -s stop
```
## Create a `Makefile`
We now have a number of long `docker` commands, let's create a `Makefile`
to make running them easier.
::: filename-for-code-block
`Makefile`
:::
```Makefile
image = test-nginx
image-build:
docker build -t $(image) .
image-rm:
docker image rm $(image)
test:
@ct=$(shell docker run --rm -d \
-v $(PWD)/conf/conf.d:/etc/nginx/conf.d \
-v $(PWD)/html:/var/www \
-v $(PWD):/opt/app \
$(image)); \
docker exec -t $$ct busted; \
docker exec $$ct openresty -s stop
```
Now we can run the test suite with the command `make test`.
## Configure the domain name
Instead of `localhost` we'd like to use an actual domain name. We can do this
with the `--add-host` option. But before we do that, we want to make sure our
container does not have access to the internet, otherwise we might
unintentionally get a response from a domain's server on the internet rather
than from our test server.
### Ensure the test container is offline
We need to create a network that has no external access.
```console
$ docker network create --internal no-internet
```
Now we need to update our `Makefile` to add the test container to our
internal-only network:
```diff
test:
@ct=$(shell docker run --rm -d \
-v $(PWD)/conf/conf.d:/etc/nginx/conf.d \
-v $(PWD)/html:/var/www \
-v $(PWD):/opt/app \
+ --network no-internet \
$(image)); \
```
And now let's add a test in `spec/nginx_spec.moon` to make sure our test
environment is offline:
```moonscript
describe "test environment", ->
it "can't connect to the internet", ->
assert.has_error (-> req "http://example.org"),
"Couldn't resolve host name"
```
Let's run our tests:
```console
$ make test
●●
2 successes / 0 failures / 0 errors / 0 pending : 0.020207 seconds
```
### Replace `localhost` with a custom domain
To use a custom domain name instead of `localhost`, we will need to use the
`--add-host` option for the `docker run` command. Again, we edit `Makefile`:
```diff
test:
@ct=$(shell docker run --rm -d \
-v $(PWD)/conf/conf.d:/etc/nginx/conf.d \
-v $(PWD)/html:/var/www \
-v $(PWD):/opt/app \
--network no-internet \
+ --add-host=domain.abc=127.0.0.1 \
$(image)); \
```
Let's update our test to use the custom domain name:
::: filename-for-code-block
`spec/nginx_spec.moon`
:::
```diff
-describe "http://localhost", ->
+describe "http://domain.abc", ->
it "sends /index.html", ->
- request = req "http://localhost"
+ request = req "http://domain.abc"
assert.same request\statusCode!, 200
```
Verify our tests still pass.
```console
$ make test
●●
2 successes / 0 failures / 0 errors / 0 pending : 0.0224 seconds
```
## Test an HTTP redirect
We want our server to redirect all `http` requests to `https`.
### Write the test
Let's practice a bit of test-driven development and write our test first.
```moonscript
describe "http://domain.abc", ->
it "redirects to https", ->
request = req "http://domain.abc"
assert.same request\statusCode!, 301
assert.same request\statusMessage!, "Moved Permanently"
assert.same request\header!.Location, "https://domain.abc/"
```
We should now have one failing test.
```console
$ make test
●●◼
2 successes / 1 failure / 0 errors / 0 pending : 0.010449 seconds
Failure → .../luajit/lib/luarocks/rocks-5.1/busted/2.2.0-1/bin/busted @ 3
http://domain.abc redirects to https
spec/nginx_spec.moon:24: Expected objects to be the same.
Passed in:
(number) 301
Expected:
(number) 200
```
### Configure `nginx`
We're going to add the redirect directives, as well as a server name for our
domain and the directives for the SSL certificates we will generate.
```diff
+server {
+ listen 80;
+ return 301 https://$host$request_uri;
+}
server {
- listen 80;
+ listen 443 ssl;
+ server_name domain.abc;
+ ssl_certificate /etc/ssl/certs/domain.abc.pem;
+ ssl_certificate_key /etc/ssl/private/domain.abc.pem;
location / {
root /var/www;
index index.html index.htm;
}
```
### Generate self-signed SSL/TLS certs for testing
Add a command to our `Dockerfile` to generate self-signed certificates:
```Dockerfile
RUN openssl req -x509 -newkey rsa:4096 -nodes \
-keyout /etc/ssl/private/domain.abc.pem \
-out /etc/ssl/certs/domain.abc.pem \
-sha256 -days 365 -subj '/CN=domain.abc' \
-addext "subjectAltName=DNS:domain.abc"
```
Rebuild the image:
```console
$ make image-rm image-build
```
We need to update our previous test to use HTTPS instead of HTTP.
::: filename-for-code-block
`spec/nginx_spec.moon`
:::
```diff
-describe "http://domain.abc", ->
+describe "https://domain.abc", ->
it "sends /index.html", ->
- request = req "http://domain.abc"
+ request = req "https://domain.abc"
```
Run tests:
```console
$ make test
●●●
3 successes / 0 failures / 0 errors / 0 pending : 0.017065 seconds
```
## Test reverse proxy a subdomain request to a Unix socket
Let's say we have a running service that connects to a Unix socket. We want to
proxy the requests through `nginx` so that our service can respond to `https`
requests but can leave handling SSL/TLS to `nginx`.
### Configure `nginx`
We'll add another server block to `conf/conf.d/default.conf` for our subdomain,
`git.domain.abc`, with the proxy directives:
```nginx
server {
listen 443 ssl;
server_name git.domain.abc;
location / {
client_max_body_size 1024M;
proxy_pass http://unix:/run/gitea/gitea.socket;
proxy_set_header Connection $http_connection;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
### Add subdomain to SSL/TLS certs
Next, we need to add our subdomain to the generated SSL certs in the
`Dockerfile`:
```diff
RUN openssl req -x509 -newkey rsa:4096 -nodes \
-keyout /etc/ssl/private/domain.abc.pem \
-out /etc/ssl/certs/domain.abc.pem \
-sha256 -days 365 -subj '/CN=domain.abc' \
- -addext "subjectAltName=DNS:domain.abc"
+ -addext "subjectAltName=DNS:domain.abc,DNS:git.domain.abc"
```
### Add subdomain as a host
Let's assign the loopback address to a variable and then add our subdomain as a
host in our `Makefile`:
```diff
+loopback = 127.0.0.1
test:
@ct=$(shell docker run --rm -d \
-v $(PWD)/conf/conf.d:/etc/nginx/conf.d \
-v $(PWD)/html:/var/www \
-v $(PWD):/opt/app \
--network no-internet \
- --add-host=domain.abc=127.0.0.1 \
+ --add-host=domain.abc=$(loopback) \
+ --add-host=git.domain.abc=$(loopback) \
$(image)); \
```
### Add a test socket server
We need to start up a mock socket server for our test to ensure our request is
being proxied correctly. This is why we needed the LuaSocket library.
Copied and modified from [here](https://github.com/lunarmodules/luasocket/blob/4844a48fbf76b0400fd7b7e4d15d244484019df1/test/unixstreamsrvr.lua),
this should suit our purposes:
::: filename-for-code-block
`spec/unixstreamsrvr.moon`
:::
```moonscript
socket = require "socket"
socket.unix = require "socket.unix"
u = assert socket.unix.stream!
assert u\bind "/run/gitea/gitea.socket"
assert u\listen!
assert u\settimeout 1
c = assert u\accept!
while true
m = assert c\receive!
break if m == ""
print m
```
### Write the test
And now we can add our test:
::: filename-for-code-block
`spec/nginx_spec.moon`
:::
```moonscript
describe "https://git.domain.abc", ->
it "reverse-proxy's a subdomain request to a unix socket", ->
socket = fname: "unixstreamsrvr.moon", dir: "/run/gitea", owner: "nobody"
basepath = debug.getinfo(1).short_src\match"^(.*)/[^/]*$" or "."
seconds = 0.1
os.execute "install -o #{socket.owner} -d #{socket.dir}"
cmd = "su -s /bin/bash -c 'moon %s' %s"
server = io.popen cmd\format "#{basepath}/#{socket.fname}", socket.owner
os.execute "sleep #{seconds}" -- wait for server to start
f = io.popen "find #{socket.dir} -type s -ls", "r"
result = with f\read "*a"
f\close!
assert.truthy result\match "nobody%s+nogroup.+#{socket.dir}/gitea.socket"
req "https://git.domain.abc"
reqheader = with server\read "*a"
server\close!
assert.truthy reqheader\match "Host: git.domain.abc"
```
Because we modified the `Dockerfile`, we need to rebuild our image:
```console
$ make image-rm image-build
```
And if all went well, our test should pass.
```console
$ make test
●●●●
4 successes / 0 failures / 0 errors / 0 pending : 0.131619 seconds
```
## Conclusion
These are just a few examples of how to test `nginx` directives. Using these
tools, we can verify that changes to our server configuration are working the
way we intended.

4
pygments.css
View File

@ -23,11 +23,11 @@ span.linenos.special { color: #50fa7b; background-color: #6272a4; padding-left:
.highlight .py-cpf { color: #6272a4 } /* Comment.PreprocFile */ .highlight .py-cpf { color: #6272a4 } /* Comment.PreprocFile */
.highlight .py-c1 { color: #6272a4 } /* Comment.Single */ .highlight .py-c1 { color: #6272a4 } /* Comment.Single */
.highlight .py-cs { color: #6272a4 } /* Comment.Special */ .highlight .py-cs { color: #6272a4 } /* Comment.Special */
.highlight .py-gd { color: #8b080b } /* Generic.Deleted */ .highlight .py-gd { color: #ff5555 } /* Generic.Deleted */
.highlight .py-ge { color: #f8f8f2; text-decoration: underline } /* Generic.Emph */ .highlight .py-ge { color: #f8f8f2; text-decoration: underline } /* Generic.Emph */
.highlight .py-gr { color: #f8f8f2 } /* Generic.Error */ .highlight .py-gr { color: #f8f8f2 } /* Generic.Error */
.highlight .py-gh { color: #f8f8f2; font-weight: bold } /* Generic.Heading */ .highlight .py-gh { color: #f8f8f2; font-weight: bold } /* Generic.Heading */
.highlight .py-gi { color: #f8f8f2; font-weight: bold } /* Generic.Inserted */ .highlight .py-gi { color: #50fa7b } /* Generic.Inserted */
.highlight .py-go { color: #f8f8f2 } /* Generic.Output */ .highlight .py-go { color: #f8f8f2 } /* Generic.Output */
.highlight .py-gp { color: #50fa7b } /* Generic.Prompt */ .highlight .py-gp { color: #50fa7b } /* Generic.Prompt */
.highlight .py-gs { color: #f8f8f2 } /* Generic.Strong */ .highlight .py-gs { color: #f8f8f2 } /* Generic.Strong */

76
renderers/markdown.moon
View File

@ -1,5 +1,70 @@
Path = require "sitegen.path" Path = require "sitegen.path"
dollar_temp = "z000sitegen_markdown00dollar0000"
-- a constructor for quote delimited strings
simple_string = (delim) ->
import P from require "lpeg"
inner = P("\\#{delim}") + "\\\\" + (1 - P delim)
inner = inner^0
P(delim) * inner * P(delim)
lua_string = ->
import P, C, Cmt, Cb, Cg from require "lpeg"
check_lua_string = (str, pos, right, left) ->
#left == #right
string_open = P"[" * P"="^0 * "["
string_close = P"]" * P"="^0 * "]"
valid_close = Cmt C(string_close) * Cb"string_open", check_lua_string
Cg(string_open, "string_open") *
(1 - valid_close)^0 * string_close
-- returns a pattern that parses a cosmo template. Can be used to have
-- pre-processors ignore text that would be handled by cosmo
parse_cosmo = ->
import P, R, Cmt, Cs, V from require "lpeg"
curly = P {
P"{" * (
simple_string("'") +
simple_string('"') +
lua_string! +
V(1) +
(P(1) - "}")
)^0 * P"}"
}
alphanum = R "az", "AZ", "09", "__"
P"$" * alphanum^1 * (curly)^-1
escape_cosmo = (str) ->
escapes = {}
import P, R, Cmt, Cs, V from require "lpeg"
counter = 0
cosmo = parse_cosmo! / (tpl) ->
counter += 1
key = "#{dollar_temp}_#{counter}"
escapes[key] = tpl
key
patt = Cs (cosmo + P(1))^0 * P(-1)
str = patt\match(str) or str, escapes
str, escapes
unescape_cosmo = (str, escapes) ->
import P, R, Cmt, Cs from require "lpeg"
escape_patt = P(dollar_temp) * P("_") * R("09")^1 / (key) ->
escapes[key] or error "bad key for unescape_cosmo"
patt = Cs (escape_patt + P(1))^0 * P(-1)
assert patt\match(str)
needs_shell_escape = (str) -> not not str\match "[^%w_-]" needs_shell_escape = (str) -> not not str\match "[^%w_-]"
shell_escape = (str) -> str\gsub "'", "''" shell_escape = (str) -> str\gsub "'", "''"
@ -19,12 +84,13 @@ write_exec = (cmd, content) ->
fname fname
-- config command like this in site.moon: class PandocRenderer extends require "sitegen.renderers.html"
-- require("renderers.markdown").cmd = "pandoc --mathjax >" @escape_cosmo: escape_cosmo
class PandocRenderer extends require "sitegen.renderers.markdown" @unescape_cosmo: unescape_cosmo
unescape_cosmo = @unescape_cosmo @parse_cosmo: parse_cosmo
escape_cosmo = @escape_cosmo
source_ext: "md"
ext: "html"
cmd: "pandoc --mathjax --lua-filter pygments.lua >" cmd: "pandoc --mathjax --lua-filter pygments.lua >"
pandoc: (content) => Path.read_file write_exec @@cmd, content pandoc: (content) => Path.read_file write_exec @@cmd, content

1
site.moon
View File

@ -29,7 +29,6 @@ get_files = (path, prefix=path) ->
files = for file in *files files = for file in *files
file\gsub "^#{escape_patt prefix}/?", "" file\gsub "^#{escape_patt prefix}/?", ""
table.sort files
files files
-- strip file extension from filename -- strip file extension from filename

4
spec/nginx_spec.moon
View File

@ -104,6 +104,10 @@ describe "https://miti.sh/posts/", ->
describe "https://miti.sh/posts", -> describe "https://miti.sh/posts", ->
it "sends /posts/index.html", -> it "sends /posts/index.html", ->
with require "sitegen.path"
assert .exists("html/posts/index.html"),
"missing html/posts/index.html (try `make build file=blog.html`)"
request = req "https://miti.sh/posts" request = req "https://miti.sh/posts"
assert.same request\statusCode!, 200 assert.same request\statusCode!, 200
assert.same request\statusMessage!, "OK" assert.same request\statusMessage!, "OK"

8
spec/renderers_spec.moon
View File

@ -86,3 +86,11 @@ this code block has no label
assert.same [[<div class="sourceCode" id="cb1"><pre assert.same [[<div class="sourceCode" id="cb1"><pre
class="sourceCode heex"><code class="sourceCode elixir"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="op">&lt;.</span>greet name<span class="op">=</span><span class="st">&quot;Jane&quot;</span><span class="op">/&gt;</span></span></code></pre></div>]], out class="sourceCode heex"><code class="sourceCode elixir"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="op">&lt;.</span>greet name<span class="op">=</span><span class="st">&quot;Jane&quot;</span><span class="op">/&gt;</span></span></code></pre></div>]], out
it "escapes and unescapes double dollar signs", ->
out = flatten_html render [[
```Makefile
$$name
```]]
assert.same [[<div class="highlight"><pre><span></span><code><span class="py-w"></span><span class="py-nv">$$name</span></code></pre></div>]], out