From 5d7896e9e576a68da667f134fd460f32b47f5e53 Mon Sep 17 00:00:00 2001 From: Catalin Constantin Mititiuc Date: Sun, 29 Jun 2025 20:28:07 -0700 Subject: [PATCH] Update post --- .../2025-06-22-test-nginx-conf-directives.md | 275 ++++++++++++------ pygments.css | 4 +- 2 files changed, 183 insertions(+), 96 deletions(-) diff --git a/posts/2025-06-22-test-nginx-conf-directives.md b/posts/2025-06-22-test-nginx-conf-directives.md index aeb214a..bfdae27 100644 --- a/posts/2025-06-22-test-nginx-conf-directives.md +++ b/posts/2025-06-22-test-nginx-conf-directives.md @@ -1,12 +1,36 @@ { - title: "Test `nginx.conf` Directives (subtitle: With MoonScript, OpenResty, - and Busted)" + title: "Test nginx Configuration Directives" + blurb: "Write tests for `nginx.conf` directives and run them against a test + server." } $index ## Introduction -We'll need nginx and luarocks. Buildpack has luarocks installed. +[`nginx`](https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#rewrite-uris-in-requests) +config file `nginx.conf` can contain any number of important directives +(redirects and rewrites, for example) that need to be verified for correctness. +We can write `specs` for directives and run them against a running test server +to ensure they are correct. + +We'll use... + +- [MoonScript](https://moonscript.org) and (by extension) +[Lua](https://www.lua.org/) programming languages +- `nginx` we'll get from [OpenResty](https://openresty.org/en/), a web platform +created by Chinese developer, [Yichun Zhang](https://agentzh.org/) +- the [Busted testing framework](https://lunarmodules.github.io/busted/) +- the Lua package manager, [LuaRocks](https://luarocks.org/) +- a fantastic little library, +[`luajit-curl`](https://bitbucket.org/senanetworksinc/luajit-curl/src/master/), +from Japanese developer [SENA Networks, Inc](https://www.sena-networks.co.jp) +- another great library, written by volunteers, [LuaSocket](https://github.com/lunarmodules/luasocket) +- our favorite container manager, [Docker Engine](https://docs.docker.com/engine/) + +## Setup + +Since we require LuaRocks, we'll use a Buildpack tag, which comes with it +already installed. ```console $ docker pull openresty/openresty:bookworm-buildpack @@ -18,29 +42,44 @@ Start a server on `localhost`: $ docker run --rm -it -p 80:80 openresty/openresty:bookworm-buildpack ``` -Visit `localhost` in browser. Should see OpenResty splash page. +Visit `localhost` in browser. We should see the OpenResty splash page. ![OpenResty default nginx index page](/images/openresty-default-index-page.png) -[Prepare directory layout](ehttps://openresty.org/en/getting-started.html#prepare-directory-layout) +## Get `nginx` running + +First, let's [prepare the directory layout](https://openresty.org/en/getting-started.html#prepare-directory-layout). ```console $ mkdir -p logs/ conf/conf.d/ html/ ``` -[Copy config file](https://github.com/openresty/docker-openresty?tab=readme-ov-file#nginx-config-files) +Next, we copy over [the default `nginx` config file](https://github.com/openresty/docker-openresty?tab=readme-ov-file#nginx-config-files). ```console $ docker run --rm -it -w /opt -v $PWD:/opt openresty/openresty:bookworm-buildpack \ cp /etc/nginx/conf.d/default.conf /opt/conf.d/ ``` -edit default.conf -change `root /usr/local/openresty/nginx/html;` to: +Then, we edit `default.conf` to change `root /usr/local/openresty/nginx/html;` +to `root /var/www;`: - root /var/www; +::: filename-for-code-block +`conf/conf.d/default.conf` +::: +```diff + location / { +- root /usr/local/openresty/nginx/html; ++ root /var/www; + index index.html index.htm; +``` + +Now, let's add an index file. + +::: filename-for-code-block `html/index.html` +::: ```html @@ -56,7 +95,7 @@ change `root /usr/local/openresty/nginx/html;` to: ``` -Start nginx: +Last, we start `nginx`: ```console $ docker run --rm -it -p 80:80 \ @@ -64,6 +103,8 @@ $ docker run --rm -it -p 80:80 \ openresty/openresty:bookworm-buildpack ``` +## Test an HTTP request + Then, in another console: ```console @@ -88,27 +129,40 @@ $ curl -v localhost ``` -If we wanted to write a test for that, we need some packages from `luarocks`. +If we want to write a test for that, we need some packages from LuaRocks. Let's +add a Dockerfile. -`Dockerfile` +### Add a `Dockerfile` ```Dockerfile FROM openresty/openresty:bookworm-buildpack WORKDIR /opt/app -# needed for testing RUN luarocks install busted RUN luarocks install luajit-curl -RUN luarocks install luasocket # needed for testing nginx reverse proxy +RUN luarocks install luasocket ``` +Now let's build our image: + ```console $ docker build -t test-nginx . +``` + +### Write the test + +Let's first make a new directory for our 'specs'. + +```console $ mkdir spec ``` +Our test makes a cURL request against our test server: + +::: filename-for-code-block `spec/nginx_spec.moon` +::: ```moonscript http = require "luajit-curl-helper.http" @@ -127,6 +181,8 @@ describe "http://localhost", -> assert.same request\body!\match("%s+(.-)%s+"), "hello world!" ``` +### Run the test suite + Start the test server: ```console @@ -137,7 +193,7 @@ $ ct=$(docker run --rm -d \ test-nginx) ``` -Run the tests. +Start the test run: ```console $ docker exec -t $ct busted @@ -148,67 +204,13 @@ $ docker exec -t $ct busted Stop the test server. ```console -$ docker exec -t $ct openresty -s stop +$ docker exec $ct openresty -s stop ``` -## Edit hosts +## Create a `Makefile` -Instead of `localhost` we'd like to use an actual domain name. We can do this -with the `--add-host` option. But before we do that, we want to make sure our -container does not have access to the internet, otherwise we might -unintentionally get a response from a domain's server on the internet rather -than from our test server. - -```console -$ docker network create --internal no-internet -``` - -Now we can start the test server with our host: - -```console -$ ct=$(docker run --rm -d \ --v $PWD/conf/conf.d:/etc/nginx/conf.d \ --v $PWD/html:/var/www \ --v $PWD:/opt/app \ ---network no-internet \ ---add-host=domain.abc=127.0.0.1 \ -test-nginx) -``` - -Update our test - - request = req "http://localhost" - -to - - request = req "http://domain.abc" - -Run the tests. - -```console -$ docker exec -t $ct busted -● -1 success / 0 failures / 0 errors / 0 pending : 0.008246 seconds -``` - -Stop the test server. - -```console -$ docker exec -t $ct openresty -s stop -``` - -## Add a test to make sure the test server is offline - -```moonscript -describe "test environment", -> - it "can't connect to the internet", -> - assert.has_error (-> req "http://example.org"), - "Couldn't resolve host name" -``` - -## Create a Makefile - -Let's create a `Makefile` to make running all these commands easier. +Ok, we now have a number of long `docker` commands, let's create a `Makefile` +to make running them easier. `Makefile` @@ -242,10 +244,74 @@ $ make test 2 successes / 0 failures / 0 errors / 0 pending : 0.008812 seconds ``` -## SSL +## Configure the domain name + +Instead of `localhost` we'd like to use an actual domain name. We can do this +with the `--add-host` option. But before we do that, we want to make sure our +container does not have access to the internet, otherwise we might +unintentionally get a response from a domain's server on the internet rather +than from our test server. + +```console +$ docker network create --internal no-internet +``` + +Now we can start the test server with our host: + +```console +$ ct=$(docker run --rm -d \ +-v $PWD/conf/conf.d:/etc/nginx/conf.d \ +-v $PWD/html:/var/www \ +-v $PWD:/opt/app \ +--network no-internet \ +--add-host=domain.abc=127.0.0.1 \ +test-nginx) +``` + +Update our test: + +::: filename-for-code-block +`spec/nginx_spec.moon` +::: + +```diff +-describe "http://localhost", -> ++describe "http://domain.abc", -> + it "sends /index.html", -> +- request = req "http://localhost" ++ request = req "http://domain.abc" + assert.same request\statusCode!, 200 +``` + +Run the tests. + +```console +$ docker exec -t $ct busted +● +1 success / 0 failures / 0 errors / 0 pending : 0.008246 seconds +``` + +Stop the test server. + +```console +$ docker exec -t $ct openresty -s stop +``` + +### Ensure the test container is offline + +```moonscript +describe "test environment", -> + it "can't connect to the internet", -> + assert.has_error (-> req "http://example.org"), + "Couldn't resolve host name" +``` + +## Test an HTTP redirect We want our server to redirect all `http` requests to `https`. +### Write the test + Our test: ```moonscript @@ -271,17 +337,7 @@ Expected: (number) 200 ``` -Make self-signed certs in Dockerfile: - -```Dockerfile -RUN openssl req -x509 -newkey rsa:4096 -nodes \ - -keyout /etc/ssl/private/domain.abc.pem \ - -out /etc/ssl/certs/domain.abc.pem \ - -sha256 -days 365 -subj '/CN=domain.abc' \ - -addext "subjectAltName=DNS:domain.abc" -``` - -Edit `default.conf`: +### Configure `nginx` ``` server { @@ -296,6 +352,18 @@ server { ssl_certificate_key /etc/ssl/private/domain.abc.pem; ``` +### Generate self-signed SSL/TLS certs for testing + +Make self-signed certs in Dockerfile: + +```Dockerfile +RUN openssl req -x509 -newkey rsa:4096 -nodes \ + -keyout /etc/ssl/private/domain.abc.pem \ + -out /etc/ssl/certs/domain.abc.pem \ + -sha256 -days 365 -subj '/CN=domain.abc' \ + -addext "subjectAltName=DNS:domain.abc" +``` + Rebuild the image: ```console @@ -318,7 +386,7 @@ Expected: (number) 301 ``` -Fix test: +It's our other test breaking, now. Fix spec: ```moonscript describe "https://domain.abc", -> @@ -334,9 +402,17 @@ $ make test 3 successes / 0 failures / 0 errors / 0 pending : 0.017065 seconds ``` -## Reverse proxy a subdomain to a unix socket +👍 -Add to `default.conf`: +## Test subdomain reverse proxy to a unix socket + +Let's say we have a running service that connects to a unix socket. We want to +proxy the requests through `nginx` so that our service can respond to `https` +requests but can leave handling SSL/TLS to `nginx`. + +### Configure `nginx` + +Our `nginx` config file might look something like this: ``` server { @@ -356,13 +432,15 @@ server { } ``` -Add subdomain to certs in Dockerfile: +### Add subdomain to SSL/TLS certs -``` +```Dockerfile -addext "subjectAltName=DNS:domain.abc,DNS:git.domain.abc" ``` -Add a test socket server (copied and modified from [here](https://github.com/lunarmodules/luasocket/blob/4844a48fbf76b0400fd7b7e4d15d244484019df1/test/unixstreamsrvr.lua)): +### Add a test socket server + +Copied and modified from [here](https://github.com/lunarmodules/luasocket/blob/4844a48fbf76b0400fd7b7e4d15d244484019df1/test/unixstreamsrvr.lua)): ::: filename-for-code-block `spec/unixstreamsrvr.moon` @@ -383,7 +461,11 @@ while true print m ``` -Add a spec: +### Write the test + +::: filename-for-code-block +`spec/nginx_spec.moon` +::: ```moonscript describe "https://git.domain.abc", -> @@ -411,6 +493,10 @@ describe "https://git.domain.abc", -> Edit Makefile: +::: filename-for-code-block +`Makefile` +::: + --add-host=git.domain.abc=$(loopback) \ Rebuild image: @@ -427,11 +513,12 @@ $ make test 4 successes / 0 failures / 0 errors / 0 pending : 0.131619 seconds ``` - ## Conclusion +Using these tools, we can verify that our `nginx` configuration is working the +way we intend. -## Bonus!: Issues We Ran Into Just Trying To Make This Post +## Bonus!: Issues Ran Into Just Making This Post ### `$host$request_uri` diff --git a/pygments.css b/pygments.css index db6cb84..bc781fd 100644 --- a/pygments.css +++ b/pygments.css @@ -23,11 +23,11 @@ span.linenos.special { color: #50fa7b; background-color: #6272a4; padding-left: .highlight .py-cpf { color: #6272a4 } /* Comment.PreprocFile */ .highlight .py-c1 { color: #6272a4 } /* Comment.Single */ .highlight .py-cs { color: #6272a4 } /* Comment.Special */ -.highlight .py-gd { color: #8b080b } /* Generic.Deleted */ +.highlight .py-gd { color: #ff5555 } /* Generic.Deleted */ .highlight .py-ge { color: #f8f8f2; text-decoration: underline } /* Generic.Emph */ .highlight .py-gr { color: #f8f8f2 } /* Generic.Error */ .highlight .py-gh { color: #f8f8f2; font-weight: bold } /* Generic.Heading */ -.highlight .py-gi { color: #f8f8f2; font-weight: bold } /* Generic.Inserted */ +.highlight .py-gi { color: #50fa7b } /* Generic.Inserted */ .highlight .py-go { color: #f8f8f2 } /* Generic.Output */ .highlight .py-gp { color: #50fa7b } /* Generic.Prompt */ .highlight .py-gs { color: #f8f8f2 } /* Generic.Strong */