Replace ANTLR with new parser #265

New Issue

2025-12-30T01:22:58+01:00

adam commented

2025-12-30 01:22:58 +01:00

Originally created by @bioball on GitHub (Jan 24, 2025).

The current parser is quite slow. On my machine (macOS on M1 Max), it takes almost half a second to parse pkl:base in Java.

In some large projects, about 40% of the wall clock time is taken by ANTLR parsing.

It's time to replace it with a hand-rolled parser.

Originally created by @bioball on GitHub (Jan 24, 2025). The current parser is _quite_ slow. On my machine (macOS on M1 Max), it takes almost half a second to parse `pkl:base` in Java. In some large projects, about 40% of the wall clock time is taken by ANTLR parsing. It's time to replace it with a hand-rolled parser.

adam closed this issue

2025-12-30 01:22:58 +01:00

adam commented

2025-12-30 01:22:59 +01:00

@sgammon commented on GitHub (Jan 28, 2025):

This will be big for a few other reasons too:

Sometimes ANTLR is shaded with Truffle builds. The version of ANTLR they provide conflicts with the fork Pkl prefers; in particular, the fork uses ANTLR's gencode v3, and GVM ships a version using v4. In particular, Sulong (GraalVM's LLVM interpreter) depends on the shaded copy.

Additionally, this is one non-modular dependency eliminated on the way toward true JPMS support, which would enable shipping a Pkl jmod and usage in jpackage and jlink targets

@sgammon commented on GitHub (Jan 28, 2025): This will be big for a few other reasons too: Sometimes [ANTLR is shaded](https://central.sonatype.com/artifact/org.graalvm.shadowed/antlr4) with Truffle builds. The version of ANTLR they provide conflicts with the [fork Pkl prefers](https://github.com/tunnelvisionlabs/antlr4); in particular, the fork uses ANTLR's gencode v3, and GVM ships a version using v4. In particular, Sulong (GraalVM's LLVM interpreter) [depends on the shaded copy](https://central.sonatype.com/artifact/org.graalvm.shadowed/antlr4/dependents). Additionally, this is one non-modular dependency eliminated on the way toward true JPMS support, which would enable shipping a Pkl jmod and usage in `jpackage` and `jlink` targets

adam commented

2025-12-30 01:22:59 +01:00

@bioball commented on GitHub (Jan 28, 2025):

ANTLR also generates grammars that can only be used with that version of ANTLR (unless this has changed recently).

If the runtime library differs from the version used to generate your grammar, you have a conflict. That's partially why ANTLR gets shaded so often.

@bioball commented on GitHub (Jan 28, 2025): ANTLR also generates grammars that can only be used with that version of ANTLR (unless this has changed recently). If the runtime library differs from the version used to generate your grammar, you have a conflict. That's partially why ANTLR gets shaded so often.

adam commented

2025-12-30 01:23:00 +01:00

@driverkt commented on GitHub (Feb 6, 2025):

Just an observer, but would it be worth considering javacc? There isn't a list of requirements here, per se.

Update: Looks like work has already begun on a custom parser. You may disregard this suggestion.

@driverkt commented on GitHub (Feb 6, 2025): Just an observer, but would it be worth considering [javacc](https://javacc.github.io/javacc/)? There isn't a list of requirements here, per se. **Update**: Looks like work has already begun on a custom parser. You may disregard this suggestion.

adam referenced this issue

2025-12-30 01:24:54 +01:00

[PR #265] [MERGED] Fix: add missing "const" and "fixed" modifiers to reflect API #485

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/pkl#265