Add a spell checker as "extension".

[GilShoshan94]

Hi,

Now, we have a code formater with rustfmt, we have a linter with cargo check or cargo clippy, why not a spell checker?

A spell checker sounds like an easy addition, it's basically just a linter specialize in spelling.

I know a spell checker sounds a bit out of scope for Rust Analyzer, but is it really ? I mean we do have a formater already, having code well formatted help with consistency and to focus on the logic instead of the aesthetic, I would argue that the same holds for spelling.

Typos is a source code spell checker written in Rust, it supports several interfaces, file, stdin to stdout and --format json.
Its design choice is described here.

Instead of having Typos to make it's own LSP (if it's even relevant, as I heard that editors will not work with multiple LSP servers for one language, but could not confirm), or having several custom plugging for each IDE, it could instead work like a "linter supplier" to existing LSP, such as Rust Analyzer to start with.

What would it requires from Typos to make it work with Rust Analyzer and is it simple enough/possible for Rust Analyzer to add a spell checker source too ?

(Also I hope this is the right place to open this conversation, if needed, I can open an issue "feature request" instead)
(Mention to crate-ci/typos#263 and @epage)

[Veykril]

A spell checker is vastly different from a code formatter. A code formatter needs knowledge about the structure of the language it targets, a spell checker does not. A spell checker can target any source files from any language without requiring any specific support for languages while a code formatter is usually tied to a certain language.

As such this is very much out of scope for rust-analyzer and would be better served as its own extension

[GilShoshan94]

Hi,

Thank you for responding.

The way r-a runs rustfmt is not just by passing the file to it ?

In theory, how hard would it be for r-a to pass the input to the spell checker, and get back a json formatted reponse (same format than the linter) and use that to add more underlines, warnings, fix suggestion, ... back in the IDE ?

Otherwise, can you advise if the right way for the spell checker is to implement lsp or to make a normal extension ?

[Veykril]

Whether r-a formats code by passing it to rustfmt or not isn't relevant here. rustfmt is a rust specific tool (that is also shipped with the rust toolchain) and as such support for it has a place in a rust IDE. Note that for running cargo check from within r-a, we are actually already contemplating whether it would make more sense to split that out from the server or not as well.

Using a language server (that is LSP) for a spell checker makes little sense, a spell checker is not a tool that provides intellisense or similar functionality for a given language. A simple extension whose sole purposes is to spell check files makes more sense.

[GilShoshan94]

Thank you for the insight !

I understand, an LSP for a spell checker is overkill, the only reason it was mention was for the standardization between all the IDE implementing the lsp protocol.

Are you aware of another solution than writing a custom extension for each IDE, that would work for all ?

[Veykril]

No I am not

[GilShoshan94]

Too bad... it would be cool if there was a standard such as LSP but for tools such as linters, formatter and spell checker, that only need to pass on the file and give a feedback to IDE, without all the complex knowledge about the structure of the language of the code.

Thank you for your time an for sharing your knowledge.
I am closing this discussion now.

Have a nice day.

[GilShoshan94]

Hi @Veykril,
Sorry to reopen...

I researched about the subject a bit more, and I am still not totally conviced that this is really out of scope...

@matklad seems to think that an ideal spell checker for code should be aware of the language (comment here and here).

Also looking more into Typos, it seems to be language aware at some extend (see typos --type-list). @epage can talk more about this.

They already discuss the idea to integrate LSP and spellcheckers.

Typos is a mature tool, past 1.0, has many features, differenciates between identifiers and words, and support several outputs formats, including json.

I really feel that it can improve the devlopper experience and it really seems that it would not even be that hard to plug this into r-a, all what the spell checker needs, is to be fed, which r-a does already for rustfmt, cargo-check and cargo-clippy, and the outputs needs to be understood by r-a to know what to underlines in the IDE and provide the fix suggestions in the IDE, exactly like with the linter. So if r-a has already a specific format that it expect from the linter, why not simply use the same for Typos ?

I would like to hear what @matklad think of this.

Please think of it for a couple of days. I really think it would be a great feature that cost near to nothing to r-a and would improve dev experience.

After I will close this I promess.

[Veykril]

No need to apologize for re-opening. I haven't looked through all the things you've linked but there are some valid arguments as to why a spell checker may be language aware in them that I didn't think of before. I'll come back to this with some more thoughts later on then, but maybe matklad will also have left some comments by then (as he does seem to have some ideas around the topic).

[GilShoshan94]

@matklad, can you comment on this please (whenever you can of course) ?

Also @epage, do you see any features you can get by beeing more integrated into an LSP and have more meta data about the code beeing check ?

Thinking more about this, if you know when a token is a function or a method call, you can skip the spell check as it cannot be changed there by the user, only spell check declarations and definitions.

Also in doc comments, we can also know if this is sample code with ```rust .... ``` blocks and with tokens that are surounded by ``.

It can reduce a lot of noise from the spell checker and be more performant I think.

[epage]

Thinking more about this, if you know when a token is a function or a method call, you can skip the spell check as it cannot be changed there by the user, only spell check declarations and definitions.

This would be the biggest advantage. typos does a lot of work to try to avoid spell checking non-words like hashes. Granted, a lot of programs could be helped by spell checking user-facing text, so maybe those heuristics would still be needed.

rust-lang/rust-clippy#2508 would be an alternative to rust-analyzer as it too can look at definitions rather than uses, there is the --fix flag for corrections, and clippy has a setting to not lint exported APIs to avoid breaking changes.

[matklad]

Some thoughts:

• I'd be against using typos as it is:
  • not ubiquitous enough to justify it's inclusion on the grounds of practicality
  • doesn't have the right design for the problem at hand, to to justify inclusion on the grounds of "this is the beginning of a perfect solution"

That's the general design principle for features in rust-analyzer:

• either the feature is implemented with technically optimal design (it doesn't matter if its incomplete or buggy, if the foundation is right. That's basically the whole rust-analyzer vs rustc thing)
• or we begrudgingly accept something which has wrong design, but ubiquitous (eg, we shell out cargo check or rustfmt for the time being while we haven't implemented the right thing ourselves yet)

For spell checker, a significant chunk of the logic indeed belongs to rust-analyzer. Specifically, rust-analyzer should:

• separate keywords from identifiers
• separate reference identifiers from definition identifiers. Typos should be checked only for definitions. Its compiler's job to point typos in the references.
• separate declarations from definitions. Function names should be checked in traits, but not in trait impls.
• find comments
• unesape strings (values of string literals should be checked for typos, not the source syntax)

I think adding the following API would make sense:

struct SpellWord {
    text: String,
    range: TextRange
}

impl Analysis {
  /// Returns the list of words for spell-checking.
  /// Note that this returns strings and not just ranges, because the value of
  /// the word to spell-check might differ from the text at range due to escaping. 
  pub fn spell_words(&self, file_id: FileId) -> Vec<SpellWord>

  /// Correct spelling (this might trigger rename refactor).
  pub fn correct_spelling(&self, file_id: FileId, word: SpellWord, correction: String) -> SourceChange
}

Implementaiton wise, I think to be pedantically correct we need to run this on macro-expanded code, but I think it'd be more robust to restrict this feature to be based strictly on the syntax.

[GilShoshan94]

And IMO spell checking comments is the most useful thing about spell checking.

100% agree

EDIT: A quick google search revealed cargo-spellcheck, which currently just runs on doc comments IIUC.

cargo-spellcheck is cool too, thank you for sharing, I looked under and it uses nlprule, which is described as:

A fast, low-resource Natural Language Processing and Error Correction library written in Rust. nlprule implements a rule- and lookup-based approach to NLP using resources from LanguageTool.

And looking more into languagetool, it's very cool, especially the rephrase suggestion, feels like chatgpt for writting better comment.

I'm defining this as out of scope for now for a spell checker.

But I have to admit that having something like this integrated in Rust-Analyzer would be an amazing DX, your code get all the existing LSP features r-a is already providing, and in addition you get spell check without all the noise since it would be only on the relevant places (on the definitions and comments) AND you would get some NLP AI to rephrase comments, all of this in one tool, one interface!

That's the future one day (I know Git Copilot and ChatGPT exist already but I don't trust it so much to not collect info...)

[GilShoshan94]

For me, its the opposite. My original motivation for typos was because I had to deal with a typo in a public API and wanted to prevent that from happening again. Its in all of the outward facing areas (APIs, doc comments, strings) that I most care about fixing typos.

100% agree too, it's important in both public API and in documentation, comments.

[GilShoshan94]

@flip1995 @matklad @Veykril @epage

Hi, back from vacation.
Can we add this on the roadmap for r-a (I know it's not high priority)?
Can we create an issue from this discussion ?

[Veykril]

Sure, an issue to track this seems fine

[GilShoshan94]

Done, I opened #14597