Unlimited lookahead in an external tokenizer is definitely going to impact how much incremental parsing Lezer is going to be able to do. I’m not really sure what way to point you in at this point—the thing you’re trying to do may not match what Lezer can do very well.