A little bit (ok, a lot) stuck on a language extension

So I’m working on a language and completion extension for a CMS called perch. I’m still trying to get my head around all the markup, I have the completions working, but on generic html items (syntax html, with a selector of html.tag.attribute, etc). When I try and change it to use my /syntaxes/PerchSyntax file, it breaks, and I get a little lost.

Perch tags exist in .html files and look like this: <perch:content id=“test” type=“text” format=“lc” … >
So in my syntax file, I need to create an expression that always selects set strings that start with <perch:

Is that right? I can’t seem to come up the the regex to make it work.

Is there a simple way to see if my syntax file is being used? (Sorry, a little lost… not sure if this is even a clear enough question to respond to :slight_smile:

– Monty

Your syntax file is used when you either manually select its syntax, or when Nova applies it according to the rules in <detectors>. The most basic way to check if the syntax file is active at all is to check if the syntax you defined in it is available in the syntax selector. If it isn’t, you probably have an error in the XML itself, in which case you should check Nova’s console (Nova’s error messages on malformed syntax file XML vary widely in utility, but at the very least, they will tell you if there is an error). If the syntax is available, the next issue is to find out what scopes are applied; the scope inspector (the small crossbeam icon in the status area) is your ticket to that.

Beyond that, things are hard to diagnose without knowing more about your syntax file. Is it uploaded anywhere one could have a look at it?

Oh, thank you! that helps so much! I see it in the syntax selector and selected it from the dropdown and now see that it is active… So if I make a better selector, I can have the editor drop into the ‘perch’ syntax automatically, right? I think it would be a combination of an html extension and a content match detector… I’ll try that.

And thank you for pointing out the scope tool! You really helped me!

– Monty

Neat! so I think that worked by doing a combo detector. But I don’t want to lose all the value of the HTML syntax. How would I tell Nova to also use it’s html syntax while using the one I’m adding?

1 Like

Generally speaking, you can’t: Nova does not allow extending existing syntaxes. However, with some syntaxes, and depending on what you intend to inject, you can achieve this by strategically including the original syntax in your own. This seems to work rather well for HTML based template languages; see here for an example (not my code, just the first example coming to my mind; also note that example does not seem to add custom HTML tags to the language, so YMMV).

1 Like

oh, thank you!

I kind of need both for this to be worthwhile as perch templates are a mixture of html and perch tags.
The SwiftSencil example doesn’t seem to work for me ().

Could the parent meta tag do it?

<parent>html</parent>

The <parent> tag only affects what other (non-syntax) extensions should be active when your syntax is applied. For a more detailed explanation of the relationship of syntax and tooling, see here. If injection à la Swift Stencil does not work for you, you basically have two options left:

  1. Fork the HTML syntax and add the necessary changes. This is likely a maintenance nightmare, as you need to redo this every time Nova updates the original syntax (though you might be able to partly automate this).
  2. Inject your scopes as Template Scopes, which has its own drawbacks.

I have been writing a syntax definition for a similar language that uses special HTML comments, and it works pretty well using the <parent>html</parent> and <type>markup</type> metas, as well as using template scopes as @kopischke mentions.

Edit: Oh, and I’m also using <scopes><include syntax="html" /></scopes>

Thank you, Martin and Christian… I will try the parent and themplate scopes again. Really appreciate the help! – Monty

Thank you so much for your help. It’s allowed me to get it off the ground! …but can’t figure a couple things out… I’ve created a repository at GitHub - montlewis/perch.novaextension: Perch template syntax and completion files where my extension files are.

On the completion side, I can’t figure out how to offer competitions for the beginning of the tag that defines the whole perch syntax… the <perch:[textstring] string…

I have:

    <provider name="perch.applications">
        <syntax>perchCMS</syntax>
            <selector>perch.tag.open.single</selector>
            <expression>(?&lt;=\&lt;)[a-zA-Z0-9_-]*</expression>

            <set>perch.applications</set>
        </provider>

where I’m trying to find that initial string to offer completions. Can’t figure that out on my own.

Also, I’m wondering how if I can add the closing “>” on the tag when the outer completion is selected.

Thank you! Monty

At a quick glance, I would say that selector cannot work. Your syntax definition scopes strings starting with <perch:«string» as perch.tag.open.single, which means you are essentially trying to complete something that is not recognised as a perch tag yet, but have scoped that completion to a recognised perch tag.

ok, that’s what I thought. So does that mean that I can’t have a completion on the set that defines the syntax?

You may be misunderstanding what the scope selector does in completions. What it does not is to indicate for what syntax group you want completions; instead, it tells Nova inside which syntax scopes the completion should be offered to the user. Which means that, to offer perch tag completions in your HTML extension syntax, you just need to scope these to the HTML open tag scope (html.tag.open.paired).

So I’m over-scoping and I should leave some of the syntax to the html syntax.

Thanks to your help, I got the other issue working where my completions show up for the app space (perch.tag.name)! I’ve updated the respoitory. Still over-scoping though. :slight_smile:

If I want the completions to show without the user typing anything, is this the code I would use on the set?

<behavior>
    <arguments shouldCompleteOnConfirm="true" />
</behavior>

That doesn’t seem to work for me.

Thank you so much for your help – Monty

Essentially, yes. The point being that the HTML syntax scopes tags far more generically than your Perch syntax, hence allowing you to escape the catch 22 of needing a <perch: tag to insert one via completion.

According to the shouldCompleteOnConfirm docs, that attribute’s effect is that

When set to true , completion will be attempted immediately after inserting the completion, as if the user had typed additional characters.

which I read as “try another completion immediately when this one has been accepted” (but I’ve not used that feature anywhere, so I might be wrong). I am not aware of a way to display a completion menu without some kind of trigger characters preceding, if that is what you are asking for?

Well, that works; however, that means your user needs to type <perch: before being offered completions. Try my suggestion, then you should get the completions as soon as < is typed (technically; you’ll probably need to narrow down the menu by typing at least <p).

If I am right about how shouldCompleteOnConfirm works, you should even be able to combine both approaches. Just offer perch: as a tag name in the HTML tag scope, then automatically follow it up by the specific variants.

Thank you! I will try that. I’m not sure I fully understand what you are saying (mainly the’ follow up with specific variants’ part). But I will try that when I have time to throw at this again. Really appreciate the help, Martin! – Monty

1 Like

ok. So I’ve gotten rid of almost all of my perch syntaxing… and have corrected my expressions in the completions file to target html elements. I see how that works now, thank you!

In perch, there are some tags that have no closing tags and some that do…

I had been addressing them by using scopes of perch.tag.open.paired and perch.tag.open.single…

whether they are pared or not depends on the text right after the opening <Perch: string… for instance, <perch:content, <perch:blog, <perch:events… (a known list of text values) should be scoped to html.tag.open.single while <perch:if, <perch:blocks… (another known list) should be scoped to html.tag.open.paired.

So do I still need to define those? wouldn’t they conflict with my use of the html syntaxing?

Glad to hear completions work for you now, but you don’t actually have to to do the former to have the latter. This is because what scopes you define in your syntax XML and what scopes you address in your completions XML to insert completions for the former do not need to be linked. I’m going to try to illustrate how that is using your original case, the <perch:«.+» tags, as I am familiar with them already. Please bear with me.

Keep in mind you have HTML syntax scoping in Perch files through your inclusion of the HTML syntax. Assuming your Perch tags are defined as they were, i.e. as starting with <perch:«.+», when will that scope be active in Nova? Answer: only when <perch:«word» is actually present in the code. If only a substring of that is in the document (like <, or <p, or pe … you get the idea), it will not be scoped as a Perch tag (that is why your original completions did not trigger). Instead, the HTML syntax kicks in and scopes that substring as a generic opening tag, until it is specific enough for the Perch tag scope to apply instead:

<!-- “|” indicates cursor while inserting a tag -->
<per|>              <!-- = generic HTML tag -->
<perch:«any char»|> <!-- = Perch tag -->

Now, when it comes to completions, this means that you cannot meaningfully offer perch:«word» completions scoped to Perch tags, because that scope only exists at a point where most, if not all, of what you want to offer as a completion has already been typed. This is the point where this discussion started.

The solution, obviously, is to offer Perch tag completions in the HTML tag scope. This has the advantage that completions kick in at a useful point, i.e right after the user starts typing a tag. However, as long as there are separate Perch tag scopes, completions will stop working as soon as Perch syntax highlighting kicks in, as that changes the syntax scope, as illustrated above.

To remedy this, you have three options:

  1. Saying “bugger this for a lark” and just remove your Perch scope – what you did on your last iteration. This gets rid of the issue I just described, i.e. Perch syntax scoping kicking in and preventing completions. However, that might not be the best way to go. If Perch tags need to do anything differently from generic HTML tags (maybe there are specialised attributes? Unique contents, like nested tags? I don’t know enough about Perch to say), you have lost the opportunity to define that in your syntax.

  2. Addressing both scopes. A <selector> expression in a completion can address several different scopes, using CSS selector syntax. For instance

    <selector>
    	html.tag.open.paired, perch.tag.open.single, perch.tag.open.paired
    </selector>
    

    would mean Perch tag name completions are shown in all three scopes. You keep the ability to define Perch specific scopes and you get completions for them, whether the tag is recognisably a Perch one or not.

  3. A variant of this is to define several different completion providers with different scopes. This is pointless duplication if they offer the same completions, but may be useful if the completions need to differ somehow in the different scopes. This will work just fine because Nova offers the completions of all providers whose scope selector and anchoring apply at the insertion point.

Just to drive the point home, because I think it is the main source of confusion: what <selector> does is to switch a completion provider on when the cursor is inside the selected syntax scope – no more, no less. There need not be any syntactic relationship between the completions themselves and the scopes they activate in. If you want to offer :poop: emojis as a completion for XML tag names, you do not need to define a syntax this is acceptable in first; a completion provider scoped to the right selector will be enough.

For your Perch syntax that means, conversely, that you don’t need to avoid defining Perch specific syntax scopes just to get completion working: HTML and Perch scopes can cohabit nicely,

man, you’re the best. thank you for that.

I think I have my completions working pretty well. But I’m still struggling with some syntax issues. Specifically I need to figure out how to scope the perch tags that are self closing. I can’t figure out how to do that without losing the html scope on the tag…

I would like to have <perch:content (and a list of others) be self-closing (like the html…single scope) but if I try and introduce that I’m back where I started by having to define all the tag and value scopes. Is there a way to append the html.tag.open.single scope’s understanding of what a is self-closing tag is? by including additional strings (perch:content, perch:blog, etc)?

by default, my perch tags are going to html.tag.open.paired which doesn’t really work as the editor will offer the closing tag when the user types ‘/>’