Memory Leak during Syntax Development

I’m currently fine-tuning a language/syntax extension I’ve created for a pretty simple “structured” data file type. It’s just to provide syntax highlighting (for now at least), so no symbolication or completions or anything.

The syntax definition is pretty much fully functional and complete, but I was trying to diagnose one last situation that wasn’t receiving proper syntax highlighting… that’s when odd behavior started occurring.

When I tried to start typing a new indented line (new subscope) it appeared that all syntax parsing after initial file opening had been completely disabled. Saving, closing, and re-opening the file was the only way to get the newly added text to become highlighted again. Notably while typing on this unparsed new line, holding the space bar down resulted in way slower repetition than normal (which I initially assumed to be the result of unintended recursion in my syntax’s regex or scoping definitions).

Even stranger, when trying to use the inspection tool to see what it thought of this new, un-highlighted text… it apparently just did not function anymore. I tried hovering over all different parts of correctly parsed text and could not get anything to pop up.

As I went back to figure out the issue I was greeted with these repeated popup warnings that made it impossible to do anything without force quitting Nova:

This is on a MacBook Pro (M1 Pro, 16GB unified memory)


Now, I’m not well-versed in low-level programming or memory allocation issues… but that said, I’ve never encountered a system alert on macOS repeatedly telling me to force quit an app due to memory concerns (not even on the old 2012 MBP I was using up until earlier this year).

I don’t see what else this could be other than an internal memory leak within Nova… most likely kicked off by some strange edge case involving a specific recursive scenario present in my syntax definition.

Is this a known issue? Anyone else encounter anything similar?

Update:

I’ve been cautiously poking this issue with a stick today and found that entering a certain subscope consistently maxes the CPU and triggers Nova’s used memory to gradually and irreversibly (without deactivating the extension or closing its targeted files) start increasing at around 1-2 GB/s … according Activity Monitor.

Convinced that this was the result of an infinite regex parsing loop that I got myself into, I tried all sorts of variations and alternative setups to the relevant parsing/scoping rules and initiated the same trigger to see if it still made the memory balloon.

I’m really at a loss now, literally nothing i’ve tried has been able to break to consistency of this memory leak trigger.

The exact trigger is to enter a certain subscope, which matches a Capitalized word followed some other characters, and then entering just one capital letter kicks off the leak. Bizarrely even testing regexes for the scope which don’t even accept capital letters… entering a capital letter still triggers the memory leak. Typing anything once the memory use has begun increasing seems to do nothing to break it.

However, everything else works fine, even if I paste a full matching string, it seems to bypass this trigger, so only when manually typing out the same matching string does it occur upon the first keystroke.

Not sure what to make of this but, I’m just gonna work around it for now.

I came across this eclecticlight.co article which describes a persistent macOS memory leak bug related to Finder searches… the description sounds very similar to what I’ve already described above:

For example, let’s say you start typing the word syzygy, which should have very few hits. When you’ve just typed the first character s there are likely to be hundreds of thousands of hits. As those are being delivered to the Finder, its memory use grows rapidly. Add the second character y and the rate of memory use slows sharply, as there are far fewer hits, and after the third character z there should be even fewer.

What should happen is that, as the search is refined, the results of earlier partial searches should be cleared from memory, which should reduce the amount of memory used by the Finder. That doesn’t happen, though, and memory use continues to rise, never to be freed, even when you close that window to finish using Find.

I suppose that if Nova uses the same underlying APIs/frameworks for syntax parsing that Finder uses to search files for specific text strings, perhaps this problem originates at the OS level? Just an educated guess though as, like i’ve said, I’m not really qualified to speculate within this realm.


By the way, I eventually ended up finding a scoping setup that seems to avoid the memory leak and haven’t had any trouble with it since then. Not sure what exactly I did to bypass the issue… but I know better than to question an inexplicably functional workaround to an inexplicable problem.