Improve QA bots with GH Actions

[evgenydmitriev]

Our crowdsourced Crypto Attack Wiki has a number of bots in charge of automated quality checks. Things have changed since we first implemented the quality checks, and now most of the functionality is covered by GPT-like services. It should be possible to refactor existing quality checks to fit the whole pull request (instead of splitting it into chunks) into one API request. Anything that goes beyond LLM's context window, can safely be cut out and produce a warning to submit smaller PRs. Although OpenAI API still lacks web browsing functionality, there are alternative solutions offered by Goggle or Azure. Reach out if you need test API keys.

To participate in the challenge, submit a pull request that refactors existing bots into one GPT-like API request and request a review from this issue assignees. Below are some improvement ideas, but feel free to suggest other functionality.

Tasks

Preview Give feedback

1. Fact-checking. Azure OpenAI Service and reverse-engineered Bard libraries have that functionality, but there are plenty of alternative solutions.
2. Spellcheck can safely be moved to GPT with a carefully crafted prompt.
3. Hugo SSG formatting checker. The fact that an article is a Markdown document with headers for Hugo SSG would probably need to be specified in the prompt anyway for adding the spellcheck functionality. Asking the LLM to check formatting should be an easy addition.
4. Existing articles' check. Getting the directory listing is the easiest way to do it, and you already implemented it.
5. Article submission guidelines checks. A good idea might be adding a markdown file with guidelines to load guidelines from there instead of hard-coding them in the prompt.
6. Plagiarism. Probably requires additional testing of existing LLM services, but there's nothing fundamental that would prevent models with internet access to spot copy-paste work.

All submissions will be reviewed by the wiki maintainers, and additional changes to your pull request may be asked of you to bring your submission to improve code quality, security, and/or efficiency.

How to Claim Bounty

Email your bitcoin or stablecoin payment address to challenge.bounties@dn.institute. Expect all payouts to be completed by the end of the month.

---

🎉 @Kseymur manged to figure out most of the needed improvements using Claude and Brave API.

[Kseymur]

Hello! Could you provide more details about Readability? What specific functionality do you need? I found a solution like this one - https://github.com/alan-turing-institute/ReadabiliPy. Do you need something similar?

[evgenydmitriev]

Yes, that is one of the libraries that implements needed functionality if you want to do fact-checking yourself. That said, latest LLM APIs have internet browsing functionality that might make readability redundant. I guess more research is needed to find the right approach here.

[Kseymur]

Thanks, i've got it!

[Serstefs]

Hi! Could be used here - language detection of the comment - with the "langdetect" library (https://pypi.org/project/langdetect/).
I'm a beginner in DS, maybe I misunderstood something."

[evgenydmitriev]

Issue description has been updated with clarifications and ideas on how to distill all needed functionality to one GPT prompt. The total bounty is raised to $1000 to encourage more challenge submissions!

[evgenydmitriev]

@Kseymur manged to figure out most of the needed improvements using Claude and Brave API. Congrats, the $1000 is yours! @albina-at-inca will process the payment at the end of the month.