184 lines
3.6 KiB
Markdown
184 lines
3.6 KiB
Markdown
{
|
|
blurb: "Add a new lexer to chroma"
|
|
}
|
|
|
|
## Intro
|
|
|
|
Chroma doesn't have a MoonScript lexer. It does has a Python script that can
|
|
convert Pygments lexers, though, and Pygments has a MoonScript lexer.
|
|
|
|
## Run MoonScript lexer generation script
|
|
|
|
To create the lexer, in the Chroma root directory run:
|
|
|
|
```console
|
|
$ docker run --rm -it -w /opt -v $PWD:/opt python bash -c \
|
|
"pip install pystache pygments \
|
|
&& python _tools/pygments2chroma_xml.py \
|
|
pygments.lexers.scripting.MoonScriptLexer > lexers/embedded/moonscript.xml \
|
|
&& pip list"
|
|
```
|
|
|
|
## Use the Chroma MoonScript lexer to highlight some code
|
|
|
|
Create a file like this:
|
|
|
|
::: filename-for-code-block
|
|
`main.go`
|
|
:::
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
|
|
"github.com/alecthomas/chroma/v2/quick"
|
|
)
|
|
|
|
func main() {
|
|
code := `package main
|
|
|
|
func main() { }
|
|
`
|
|
|
|
fmt.Println(quick.Highlight(os.Stdout, code, "go", "html", "monokai"))
|
|
}
|
|
```
|
|
|
|
I did one of these:
|
|
|
|
```console
|
|
$ docker run --rm -it -w /opt -v $PWD:/opt golang:tip-bookworm \
|
|
go mod init main
|
|
```
|
|
|
|
Which gave me the `go.mod` file.
|
|
|
|
::: filename-for-code-block
|
|
`go.mod`
|
|
:::
|
|
|
|
```
|
|
module main
|
|
|
|
go 1.25
|
|
|
|
require (
|
|
github.com/alecthomas/chroma/v2 v2.18.0 // indirect
|
|
github.com/dlclark/regexp2 v1.11.5 // indirect
|
|
)
|
|
```
|
|
|
|
Then I did one of these:
|
|
|
|
```console
|
|
$ docker run --rm -it -w /opt -v $PWD:/opt golang:tip-bookworm \
|
|
go run main.go
|
|
```
|
|
|
|
And that should output markup (and styles) for highlighting that block of Go
|
|
code to the console. But if we notice, it's importing the Chroma package from
|
|
the GitHub repo. We want to use our local version of chroma, so we use `go mod
|
|
edit` to [replace the chroma import with our local version](https://go.dev/ref/mod#go-mod-file-replace):
|
|
|
|
```console
|
|
$ docker run --rm -it -w /opt -v $PWD:/opt golang:tip-bookworm \
|
|
go mod edit -replace github.com/alecthomas/chroma/v2@v2.18.0=./chroma
|
|
```
|
|
|
|
Which adds this line to our `go.mod` file:
|
|
|
|
::: filename-for-code-block
|
|
`go.mod`
|
|
:::
|
|
|
|
```
|
|
...
|
|
|
|
replace github.com/alecthomas/chroma/v2 v2.18.0 => ./chroma
|
|
```
|
|
|
|
Now we can put some MoonScript in `main.go`.
|
|
|
|
```go
|
|
code := `print "Hello, #{@name}!"`
|
|
|
|
fmt.Println(quick.Highlight(os.Stdout, code, "moonscript", "html", "monokai"))
|
|
```
|
|
|
|
And we have it:
|
|
|
|
```console
|
|
$ docker run --rm -it -w /opt -v $PWD:/opt golang:tip-bookworm \
|
|
go run main.go
|
|
go: downloading github.com/dlclark/regexp2 v1.11.5
|
|
```
|
|
|
|
That should output syntax highlighting using our local version of chroma.
|
|
|
|
## Create testdata
|
|
|
|
Create a file in `lexers/testdata` called `moonscript.actual`. Add the tokens
|
|
from the language in this file.
|
|
|
|
## Record test output
|
|
|
|
Create another file called `lexers/testdata/moonscript.expected`. This is the
|
|
file we will record to.
|
|
|
|
```console
|
|
$ RECORD=true go test ./lexers
|
|
```
|
|
|
|
Visually inspect and verify that the expected data is correct.
|
|
|
|
## Run tests
|
|
|
|
```console
|
|
$ go test ./lexers
|
|
```
|
|
|
|
## Bonus!: Use local `pygments` with `pygments2chroma_xml.py`
|
|
|
|
These lines in `pygments2chroma_xml.py`:
|
|
|
|
```python
|
|
import pystache
|
|
from pygments import lexer as pygments_lexer
|
|
from pygments.token import _TokenType
|
|
```
|
|
|
|
Import pygments from pip? How do we get it to load a local version of
|
|
`pygments`?
|
|
|
|
In Pygments root directory:
|
|
|
|
```console
|
|
$ docker run --rm -it -w /opt -v $PWD:/opt \
|
|
-v ../gitea-syntax-highlight/chroma/_tools/pygments2chroma_xml.py:/opt/pygments2chroma_xml.py \
|
|
python bash -c "pip install pystache && pip list \
|
|
&& python pygments2chroma_xml.py pygments.lexers.scripting.LuaLexer"
|
|
```
|
|
|
|
Should see.
|
|
|
|
```console
|
|
Package Version
|
|
-------- -------
|
|
pip 25.0.1
|
|
pystache 0.6.8
|
|
```
|
|
|
|
That shows no remote pygments package is installed. After that you will see the
|
|
lexer markup output.
|
|
|
|
```console
|
|
<lexer>
|
|
<config>
|
|
...
|
|
```
|
|
|
|
If you wanted to save
|