From f9ef223cf12c34949dcdc7acca14c884859e9d71 Mon Sep 17 00:00:00 2001 From: Catalin Constantin Mititiuc Date: Mon, 30 Jun 2025 15:25:14 -0700 Subject: [PATCH] Edit post --- .../2025-06-22-test-nginx-conf-directives.md | 278 ++++++++++-------- 1 file changed, 163 insertions(+), 115 deletions(-) diff --git a/posts/2025-06-22-test-nginx-conf-directives.md b/posts/2025-06-22-test-nginx-conf-directives.md index bfdae27..b89dc39 100644 --- a/posts/2025-06-22-test-nginx-conf-directives.md +++ b/posts/2025-06-22-test-nginx-conf-directives.md @@ -8,12 +8,12 @@ $index ## Introduction [`nginx`](https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#rewrite-uris-in-requests) -config file `nginx.conf` can contain any number of important directives -(redirects and rewrites, for example) that need to be verified for correctness. -We can write `specs` for directives and run them against a running test server -to ensure they are correct. +configuration can contain any number of important directives (redirects and +rewrites, for example) that need to be verified for correctness. We can write +tests for directives and run them against a test server to ensure they are +correct. -We'll use... +To do this, we'll use... - [MoonScript](https://moonscript.org) and (by extension) [Lua](https://www.lua.org/) programming languages @@ -42,7 +42,8 @@ Start a server on `localhost`: $ docker run --rm -it -p 80:80 openresty/openresty:bookworm-buildpack ``` -Visit `localhost` in browser. We should see the OpenResty splash page. +We can visit `localhost` in our browser and we should see the OpenResty splash +page. ![OpenResty default nginx index page](/images/openresty-default-index-page.png) @@ -61,8 +62,7 @@ $ docker run --rm -it -w /opt -v $PWD:/opt openresty/openresty:bookworm-buildpac cp /etc/nginx/conf.d/default.conf /opt/conf.d/ ``` -Then, we edit `default.conf` to change `root /usr/local/openresty/nginx/html;` -to `root /var/www;`: +Then, we update the root directive in `default.conf`: ::: filename-for-code-block `conf/conf.d/default.conf` @@ -103,9 +103,7 @@ $ docker run --rm -it -p 80:80 \ openresty/openresty:bookworm-buildpack ``` -## Test an HTTP request - -Then, in another console: +Then, in another console, this should output our index file. ```console $ curl -v localhost @@ -129,8 +127,11 @@ $ curl -v localhost ``` -If we want to write a test for that, we need some packages from LuaRocks. Let's -add a Dockerfile. +## Test an HTTP request + +If we want to write a test for that request, we need some packages from +LuaRocks. Let's add a Dockerfile to build an image with those packages +installed. ### Add a `Dockerfile` @@ -139,6 +140,7 @@ FROM openresty/openresty:bookworm-buildpack WORKDIR /opt/app +RUN luarocks install moonscript RUN luarocks install busted RUN luarocks install luajit-curl RUN luarocks install luasocket @@ -152,7 +154,7 @@ $ docker build -t test-nginx . ### Write the test -Let's first make a new directory for our 'specs'. +Let's first make a new directory where our tests will live. ```console $ mkdir spec @@ -183,7 +185,8 @@ describe "http://localhost", -> ### Run the test suite -Start the test server: +Start the test server. We're going to use `text-nginx`, the image we just +built. ```console $ ct=$(docker run --rm -d \ @@ -212,37 +215,30 @@ $ docker exec $ct openresty -s stop Ok, we now have a number of long `docker` commands, let's create a `Makefile` to make running them easier. +::: filename-for-code-block `Makefile` +::: ```Makefile image = test-nginx -loopback = 127.0.0.1 image-build: - docker build -t $(image) . + docker build -t $(image) . image-rm: - docker image rm $(image) + docker image rm $(image) test: - @ct=$(shell docker run --rm -d \ - -v $(PWD)/conf/conf.d:/etc/nginx/conf.d \ - -v $(PWD)/html:/var/www \ - -v $(PWD):/opt/app \ - --network no-internet \ - --add-host=domain.abc=$(loopback) \ - $(image)); \ - docker exec -t $$ct busted; \ - docker exec $$ct openresty -s stop + @ct=$(shell docker run --rm -d \ + -v $(PWD)/conf/conf.d:/etc/nginx/conf.d \ + -v $(PWD)/html:/var/www \ + -v $(PWD):/opt/app \ + $(image)); \ + docker exec -t $$ct busted; \ + docker exec $$ct openresty -s stop ``` -Now we can run tests by running `make test`. - -```console -$ make test -●● -2 successes / 0 failures / 0 errors / 0 pending : 0.008812 seconds -``` +Now we can run the test suite with the command `make test`. ## Configure the domain name @@ -252,23 +248,62 @@ container does not have access to the internet, otherwise we might unintentionally get a response from a domain's server on the internet rather than from our test server. +### Ensure the test container is offline + +We need to create a network that has no external access. + ```console $ docker network create --internal no-internet ``` -Now we can start the test server with our host: +Now we need to update our `Makefile` to add the test container to our +internal-only network: -```console -$ ct=$(docker run --rm -d \ --v $PWD/conf/conf.d:/etc/nginx/conf.d \ --v $PWD/html:/var/www \ --v $PWD:/opt/app \ ---network no-internet \ ---add-host=domain.abc=127.0.0.1 \ -test-nginx) +```diff + test: + @ct=$(shell docker run --rm -d \ + -v $(PWD)/conf/conf.d:/etc/nginx/conf.d \ + -v $(PWD)/html:/var/www \ + -v $(PWD):/opt/app \ ++ --network no-internet \ + $(image)); \ ``` -Update our test: +And now let's add a test in `spec/nginx_spec.moon` to make sure our test +environment is offline: + +```moonscript +describe "test environment", -> + it "can't connect to the internet", -> + assert.has_error (-> req "http://example.org"), + "Couldn't resolve host name" +``` + +Let's run our tests: + +```console +$ make test +●● +2 successes / 0 failures / 0 errors / 0 pending : 0.020207 seconds +``` + +### Replace `localhost` with a custom domain + +To use a custom domain name instead of `localhost`, we will need to use the +`--add-host` option for the `docker run` command. Again, we edit `Makefile`: + +```diff + test: + @ct=$(shell docker run --rm -d \ + -v $(PWD)/conf/conf.d:/etc/nginx/conf.d \ + -v $(PWD)/html:/var/www \ + -v $(PWD):/opt/app \ + --network no-internet \ ++ --add-host=domain.abc=127.0.0.1 \ + $(image)); \ +``` + +Let's update our test to use the custom domain name: ::: filename-for-code-block `spec/nginx_spec.moon` @@ -283,27 +318,12 @@ Update our test: assert.same request\statusCode!, 200 ``` -Run the tests. +Verify our tests still pass. ```console -$ docker exec -t $ct busted -● -1 success / 0 failures / 0 errors / 0 pending : 0.008246 seconds -``` - -Stop the test server. - -```console -$ docker exec -t $ct openresty -s stop -``` - -### Ensure the test container is offline - -```moonscript -describe "test environment", -> - it "can't connect to the internet", -> - assert.has_error (-> req "http://example.org"), - "Couldn't resolve host name" +$ make test +●● +2 successes / 0 failures / 0 errors / 0 pending : 0.0224 seconds ``` ## Test an HTTP redirect @@ -312,7 +332,7 @@ We want our server to redirect all `http` requests to `https`. ### Write the test -Our test: +Let's practice a bit of test-driven development and write our test first. ```moonscript describe "http://domain.abc", -> @@ -323,6 +343,8 @@ describe "http://domain.abc", -> assert.same request\header!.Location, "https://domain.abc/" ``` +We should now have one failing test. + ```console $ make test ●●◼ @@ -339,22 +361,31 @@ Expected: ### Configure `nginx` -``` -server { - listen 80; - return 301 https://$host$request_uri; -} +We're going to add the redirect directives, as well as a server name for our +domain and the directives for the SSL certificates we will generate. -server { - listen 443 ssl; - server_name domain.abc; - ssl_certificate /etc/ssl/certs/domain.abc.pem; - ssl_certificate_key /etc/ssl/private/domain.abc.pem; +```diff ++server { ++ listen 80; ++ return 301 https://$host$request_uri; ++} + + server { +- listen 80; ++ listen 443 ssl; ++ server_name domain.abc; ++ ssl_certificate /etc/ssl/certs/domain.abc.pem; ++ ssl_certificate_key /etc/ssl/private/domain.abc.pem; + + location / { + root /var/www; + index index.html index.htm; + } ``` ### Generate self-signed SSL/TLS certs for testing -Make self-signed certs in Dockerfile: +Add a command to our Dockerfile to generate self-signed certificates: ```Dockerfile RUN openssl req -x509 -newkey rsa:4096 -nodes \ @@ -370,28 +401,18 @@ Rebuild the image: $ make image-rm image-build ``` -Run tests: +We need to update our previous test to use HTTPS instead of HTTP. -```console -$ make test -●◼● -2 successes / 1 failure / 0 errors / 0 pending : 0.009618 seconds +::: filename-for-code-block +`spec/nginx_spec.moon` +::: -Failure → .../luajit/lib/luarocks/rocks-5.1/busted/2.2.0-1/bin/busted @ 3 -http://domain.abc sends /index.html -spec/nginx_spec.moon:17: Expected objects to be the same. -Passed in: -(number) 200 -Expected: -(number) 301 -``` - -It's our other test breaking, now. Fix spec: - -```moonscript -describe "https://domain.abc", -> - it "sends /index.html", -> - request = req "https://domain.abc" +```diff +-describe "http://domain.abc", -> ++describe "https://domain.abc", -> + it "sends /index.html", -> +- request = req "http://domain.abc" ++ request = req "https://domain.abc" ``` Run tests: @@ -402,9 +423,7 @@ $ make test 3 successes / 0 failures / 0 errors / 0 pending : 0.017065 seconds ``` -👍 - -## Test subdomain reverse proxy to a unix socket +## Test reverse proxy a subdomain request to a unix socket Let's say we have a running service that connects to a unix socket. We want to proxy the requests through `nginx` so that our service can respond to `https` @@ -412,9 +431,10 @@ requests but can leave handling SSL/TLS to `nginx`. ### Configure `nginx` -Our `nginx` config file might look something like this: +We'll add another server block to `conf/conf.d/default.conf` for our subdomain, +`git.domain.abc`, with the proxy directives: -``` +```nginx server { listen 443 ssl; server_name git.domain.abc; @@ -434,13 +454,46 @@ server { ### Add subdomain to SSL/TLS certs -```Dockerfile --addext "subjectAltName=DNS:domain.abc,DNS:git.domain.abc" +Next, we need to add our subdomain to the generated SSL certs in the +`Dockerfile`: + +```diff + RUN openssl req -x509 -newkey rsa:4096 -nodes \ + -keyout /etc/ssl/private/domain.abc.pem \ + -out /etc/ssl/certs/domain.abc.pem \ + -sha256 -days 365 -subj '/CN=domain.abc' \ +- -addext "subjectAltName=DNS:domain.abc" ++ -addext "subjectAltName=DNS:domain.abc,DNS:git.domain.abc" ``` +### Add subdomain as a host + +Let's assign the loopback address to a variable and then add our subdomain as a +host in our `Makefile`: + +```diff ++loopback = 127.0.0.1 + + test: + @ct=$(shell docker run --rm -d \ + -v $(PWD)/conf/conf.d:/etc/nginx/conf.d \ + -v $(PWD)/html:/var/www \ + -v $(PWD):/opt/app \ + --network no-internet \ +- --add-host=domain.abc=127.0.0.1 \ ++ --add-host=domain.abc=$(loopback) \ ++ --add-host=git.domain.abc=$(loopback) \ + $(image)); \ +``` + + ### Add a test socket server -Copied and modified from [here](https://github.com/lunarmodules/luasocket/blob/4844a48fbf76b0400fd7b7e4d15d244484019df1/test/unixstreamsrvr.lua)): +We need to start up a mock socket server for our test to ensure our request is +being proxied correctly. This is why we needed the LuaSocket library. + +Copied and modified from [here](https://github.com/lunarmodules/luasocket/blob/4844a48fbf76b0400fd7b7e4d15d244484019df1/test/unixstreamsrvr.lua), +this should suit our purposes: ::: filename-for-code-block `spec/unixstreamsrvr.moon` @@ -463,6 +516,8 @@ while true ### Write the test +And now we can add our test: + ::: filename-for-code-block `spec/nginx_spec.moon` ::: @@ -491,21 +546,13 @@ describe "https://git.domain.abc", -> assert.truthy reqheader\match "Host: git.domain.abc" ``` -Edit Makefile: - -::: filename-for-code-block -`Makefile` -::: - - --add-host=git.domain.abc=$(loopback) \ - -Rebuild image: +Because we modified the `Dockerfile`, we need to rebuild our image: ```console $ make image-rm image-build ``` -Run tests: +And if all went well, our test should pass. ```console $ make test @@ -515,8 +562,9 @@ $ make test ## Conclusion -Using these tools, we can verify that our `nginx` configuration is working the -way we intend. +These are just a few examples of how to test `nginx` directives. Using these +tools, we can verify that changes to our server configuration are working the +way we intended. ## Bonus!: Issues Ran Into Just Making This Post