diff --git a/html/.gitignore b/html/.gitignore index fc8baad..628bfae 100644 --- a/html/.gitignore +++ b/html/.gitignore @@ -2,6 +2,7 @@ app.css code.html index.html pandoc.css +posts/add-a-pygments-lexer-to-chroma.html posts/build-a-neovim-qt-appimage-from-source.html posts/build-static-website-generator-part-1.html posts/deploy-elixir-generated-html-with-docker-on-digitalocean.html diff --git a/posts/2025-06-20-add-a-pygments-lexer-to-chroma.md b/posts/2025-06-20-add-a-pygments-lexer-to-chroma.md new file mode 100644 index 0000000..9049063 --- /dev/null +++ b/posts/2025-06-20-add-a-pygments-lexer-to-chroma.md @@ -0,0 +1,287 @@ +{ + title: "Add a Pygments Lexer to Chroma" + blurb: "[Pygments][4] and [Chroma][5] are syntax highlighting libraries + written in [Python][6] and [Go][7], respecitvely. Chroma is missing a + language we like, which Pygments already supports. We add support for our + language to Chroma by converting the existing lexer from Pygments. + + [4]: https://github.com/pygments/pygments + [5]: https://github.com/alecthomas/chroma + [6]: https://www.python.org/ + [7]: https://go.dev/" +} +$index + +## Introduction + +[Gitea][8] uses [Chroma][9] for syntax highlighting. Chroma is based on the +Python syntax highlighter, [Pygments][10], and includes a [script][11] to help +convert Pygments lexers for use with Chroma. We describe how below. + +[8]: https://github.com/go-gitea/gitea +[9]: https://github.com/alecthomas/chroma +[10]: https://github.com/pygments/pygments +[11]: https://github.com/alecthomas/chroma/blob/484750a96fc430f49d6b69cc2a2a8b7a67691446/_tools/pygments2chroma_xml.py + +## Setup + +We're going to be using the `python` and `golang` [Docker][3] images. Docker +Desktop is _not_ required. + +```console +$ docker pull python +$ docker pull golang +``` + +Let's set up some aliases to make running the commands easier. + +```console +$ alias docker-run='docker run --rm -it -w /opt -v $PWD:/opt' +$ alias docker-run-go='docker-run golang' +$ alias docker-run-py='docker-run python' +``` + +[3]: https://docs.docker.com/engine/ + +## Convert a Pygments lexer to a Chroma lexer with `pygments2chroma_xml.py` + +```console +$ git clone https://github.com/alecthomas/chroma.git +$ cd chroma +``` + +In the Chroma root directory, we run: + +```console +$ docker-run-py bash -c \ + "pip install pystache pygments && \ + python _tools/pygments2chroma_xml.py \ + pygments.lexers.scripting.LuaLexer > lexers/embedded/lua.xml && \ + pip list" +``` + +We should see this in the output: + +``` +Package Version +-------- ------- +pip 25.0.1 +Pygments 2.19.2 +pystache 0.6.8 +``` + +This just helps us know what version of Pygments we generated our lexer from. +The file `lexers/embedded/lua.xml` should now contain all the tokenization +rules for the [Lua](https://www.lua.org) language. + +::: filename-for-code-block +`lexers/embedded/lua.xml` +::: + +```xml + + + Lua + ... +``` + +## Highlight some code with a Chroma lexer + +Chroma provides a [simple example test file][1] we can modify to see what syntax +highlighting with our new lexer looks like. First, though, we need to create a +new Go module by running `go mod init`: + +```console +$ cd .. +$ docker-run-go go mod init main +go: creating new go.mod: module main +go: to add module requirements and sums: + go mod tidy +``` + +We will need required modules, so let's go ahead and run `go mod tidy` as the +output suggests. + +```console +$ docker-run-go go mod tidy +``` + +We should now have 2 additional files, `go.mod` and `go.sum`. `go.sum` has some +package hashes while `go.mod` should look like this: + +::: filename-for-code-block +`go.mod` +::: + +``` +module main + +go 1.25 + +require github.com/alecthomas/chroma/v2 v2.18.0 + +require github.com/dlclark/regexp2 v1.11.5 // indirect +``` + +Now we can create a `main.go` file and copy over the code from Chroma's example +test file, but we update the `code` variable with some Lua, `print("hello")`, +and the lexer we pass into the `Highlight` function is changed to `"lua"`: + +::: filename-for-code-block +`main.go` +::: + +```go +package main + +import ( + "log" + "os" + + "github.com/alecthomas/chroma/v2/quick" +) + +func main() { + code := `print("hello")` + + err := quick.Highlight(os.Stdout, code, "lua", "html", "monokai") + if err != nil { + log.Fatal(err) + } +} +``` + +Now we can try running our `main.go` like this: + +```console +$ docker-run-go go run main.go +go: downloading github.com/alecthomas/chroma/v2 v2.18.0 +go: downloading github.com/dlclark/regexp2 v1.11.5 + +