Replace ANTLR with new parser #265

Closed
opened 2025-12-30 01:22:58 +01:00 by adam · 3 comments
Owner

Originally created by @bioball on GitHub (Jan 24, 2025).

The current parser is quite slow. On my machine (macOS on M1 Max), it takes almost half a second to parse pkl:base in Java.

In some large projects, about 40% of the wall clock time is taken by ANTLR parsing.

It's time to replace it with a hand-rolled parser.

Originally created by @bioball on GitHub (Jan 24, 2025). The current parser is _quite_ slow. On my machine (macOS on M1 Max), it takes almost half a second to parse `pkl:base` in Java. In some large projects, about 40% of the wall clock time is taken by ANTLR parsing. It's time to replace it with a hand-rolled parser.
adam closed this issue 2025-12-30 01:22:58 +01:00
Author
Owner

@sgammon commented on GitHub (Jan 28, 2025):

This will be big for a few other reasons too:

Sometimes ANTLR is shaded with Truffle builds. The version of ANTLR they provide conflicts with the fork Pkl prefers; in particular, the fork uses ANTLR's gencode v3, and GVM ships a version using v4. In particular, Sulong (GraalVM's LLVM interpreter) depends on the shaded copy.

Additionally, this is one non-modular dependency eliminated on the way toward true JPMS support, which would enable shipping a Pkl jmod and usage in jpackage and jlink targets

@sgammon commented on GitHub (Jan 28, 2025): This will be big for a few other reasons too: Sometimes [ANTLR is shaded](https://central.sonatype.com/artifact/org.graalvm.shadowed/antlr4) with Truffle builds. The version of ANTLR they provide conflicts with the [fork Pkl prefers](https://github.com/tunnelvisionlabs/antlr4); in particular, the fork uses ANTLR's gencode v3, and GVM ships a version using v4. In particular, Sulong (GraalVM's LLVM interpreter) [depends on the shaded copy](https://central.sonatype.com/artifact/org.graalvm.shadowed/antlr4/dependents). Additionally, this is one non-modular dependency eliminated on the way toward true JPMS support, which would enable shipping a Pkl jmod and usage in `jpackage` and `jlink` targets
Author
Owner

@bioball commented on GitHub (Jan 28, 2025):

ANTLR also generates grammars that can only be used with that version of ANTLR (unless this has changed recently).

If the runtime library differs from the version used to generate your grammar, you have a conflict. That's partially why ANTLR gets shaded so often.

@bioball commented on GitHub (Jan 28, 2025): ANTLR also generates grammars that can only be used with that version of ANTLR (unless this has changed recently). If the runtime library differs from the version used to generate your grammar, you have a conflict. That's partially why ANTLR gets shaded so often.
Author
Owner

@driverkt commented on GitHub (Feb 6, 2025):

Just an observer, but would it be worth considering javacc? There isn't a list of requirements here, per se.

Update: Looks like work has already begun on a custom parser. You may disregard this suggestion.

@driverkt commented on GitHub (Feb 6, 2025): Just an observer, but would it be worth considering [javacc](https://javacc.github.io/javacc/)? There isn't a list of requirements here, per se. **Update**: Looks like work has already begun on a custom parser. You may disregard this suggestion.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/pkl#265