Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📝 docs: improve askAI plugin documentation #630

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

azigler
Copy link

@azigler azigler commented Jan 8, 2025

Updates documentation for the askAI plugin to provide clearer information about data privacy, API usage, and getting started with recommended use cases. Includes better formatting, more precise descriptions of the plugin's functionality, and improved examples of how to use the askAI integration with OpenAI's latest model.

The documentation and FAQ now better explains the data privacy of using AI features with third-party providers. The FAQ section has been streamlined for better readability and understanding of gitStream's capabilities in regards to AI.___

PR Description by askLB('Describe PR')

  • Updated documentation to clarify gitStream's access to code and AI plugin usage.
  • Revised AI plugin integration to require OpenAI's gpt-4o-2024-08-06 model and an API token.
  • Refined examples and parameters in askAI plugin reference to match new implementation.

The purpose of these changes is to enhance clarity and functionality of gitStream's AI integration documentation and plugin usage, ensuring users understand configuration requirements and potential impacts.
Please rate this askLB description:

  • Excellent
  • Good Enough
  • Needs Improvement

Updates documentation for the askAI plugin and FAQ sections to provide clearer
information about data privacy, API usage, and supported features. Includes better
formatting, more precise descriptions of the plugin's functionality, and improved
examples of how to use the askAI integration with OpenAI's latest model.

The documentation now better explains the data privacy implications of using AI
features and provides more accurate technical details about the OpenAI
integration. The FAQ section has been streamlined for better readability and
understanding of gitStream's capabilities.
Copy link

jit-ci bot commented Jan 8, 2025

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

Copy link

@orca-security-us orca-security-us bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

plugins/filters/askAI/index.js Outdated Show resolved Hide resolved
docs/faq.md Outdated Show resolved Hide resolved
plugins/filters/askAI/index.js Outdated Show resolved Hide resolved
plugins/filters/askAI/reference.md Outdated Show resolved Hide resolved
PavelLinearB and others added 3 commits January 8, 2025 13:44
…ntax

Documentation has been updated to be more precise about data sharing in gitStream and to fix syntax in the askAI plugin example. The parameter type for the token argument has been corrected from Object to string in the JSDoc comments to better reflect its actual type.
Improves documentation for askAI features by standardizing titles, descriptions,
and formatting across all automation files. Updates prompts to be more specific
and adds consistent footers to AI responses. Makes labels formatting consistent
using backticks and enhances readability of configuration descriptions.
@azigler azigler marked this pull request as ready for review January 8, 2025 20:20
@azigler azigler requested a review from PavelLinearB January 8, 2025 20:23
@gitstream-cm gitstream-cm bot requested a review from a team January 8, 2025 20:23
Copy link
Collaborator

@BenLloydPearson BenLloydPearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few comments, but nothing that should block publishing these changes.

docs/automations/integrations/askAI/add-tests/README.md Outdated Show resolved Hide resolved
## gitStream askAI: recommended documentation for this PR 📑
{{ source | askAI("experienced technical writer focused on documentation",
"Review the PR code diff.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about putting the prompt info into the CM file, rather than incorporating it into the plugin code because this has a risk of becoming messy if we build out additional use cases this way.

Is it possible to incorporate the prompt into the plugin? I'm imagining it as a JSON file that's pulled into the code for easy updates.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BenLloydPearson One way we could do this is using the gitStream readFile filter in a .cm config to import *.txt files from a askAI/prompts/ folder as strings. Then we could maintain the prompts there or even use multiple readFile function calls to "build" a prompt for each use case. Something like this:

automations:
  ask_ai_summarize:
    if:
      - true
    run:
      - action: add-comment@v1
        args:
          comment: |
            {{ TITLE }}
            {{ source | askAI(ROLE, PROMPT, env.OPEN_AI_TOKEN) }}

TITLE: ## gitStream askAI: summary for this PR 📜
ROLE: {{ "./.cm/plugins/filters/askAI/prompts/PR_summary_role.txt" | readFile() }}
PROMPT: {{ "./.cm/plugins/filters/askAI/prompts/PR_summary_prompt.txt" | readFile() }}

Using variables in this approach makes it so each AI automation uses the same base configuration, and you simply adjust behavior by swapping in and out prompts. I think a small folder of combinable prompts is the good way to approach this for longevity while keeping it flat and intuitive, but open to more brainstorming here.

Alternatively, we could change the plugin's .js file to load prompts this way, but feels less composable for dev adoption in the long run. @PavelLinearB probably has an opinion on a good approach.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this solution.

I think your naming convention for the txt files should work, but just be wary that we'll want to make sure we follow something that's consistent, descriptive, and sorts properly alphabetically (i.e. the prompt and role should appear next to each other when viewed inside the directory.) I believe your convention should check all these boxes, and we can adjust in the future if we find too many edge cases.

docs/integrations/askAI.md Outdated Show resolved Hide resolved
The documentation titles and descriptions are now more clear and consistent.
The changes make the purpose of each AI integration more explicit and
actionable. For example, "Ask AI to Review a PR" becomes "Ask AI for a PR
Review Checklist" which better describes the actual functionality.
@MishaKav MishaKav removed the LB CR label Jan 14, 2025
@MishaKav MishaKav added the LB CR label Jan 14, 2025
Copy link
Contributor

gitstream-cm bot commented Jan 14, 2025

PR Review by LinearB_AI('Review PR')

Documentation improvements - docs/automations/integrations/askAI/ (multiple files)

The documentation changes improve consistency, clarity and readability across the askAI integration documentation:

- title: Automation - Ask AI for a Code Review
+ title: Automation - Ask AI for a PR Review Checklist

- Use AI to suggests additional test cases for uncovered or modified functions in the PR, including edge cases.
+ Use AI to recommend unit tests for the changes in a PR.

- Add a comment with suggested tests generated by an AI model 
+ Add a comment with AI-generated tests recommendations for the PR.

The changes:

  • Make titles and descriptions more specific and action-oriented
  • Use consistent terminology (e.g. "recommend", "generate")
  • Fix grammar and punctuation issues
  • Add note about AI plugin credentials and costs
  • Format code blocks consistently
AI Integration refactoring - plugins/filters/askAI/ (index.js)

The askAI plugin code documentation has been improved for clarity:

- * @description A gitStream plugin to interact with AI models. Currently works with `ChatGPR-4o-mini`.
+ * @description A gitStream plugin to facilitate AI workflows with OpenAI's `gpt-4o-2024-08-06` model.

- * @param {Object} context - The context that will be attached to the prompt .
+ * @param {Object} context - The context to be sent to the AI model with the prompt.

- * @param {Object} token - The token to the AI model.
+ * @param {string} token - Your OpenAI API token.

The changes:

  • Clarify plugin purpose and model compatibility
  • Improve parameter descriptions
  • Fix example code snippet
  • Use consistent parameter types
FAQ updates - docs/faq.md

The FAQ content has been improved for clarity:

- ## Does gitStream services have access to my code?
+ ## Does gitStream have access to my code?

- Like any other CI/CD automation, the source code is being scanned in the repo and is not shared with any external services. Only metadata related to and affecting the workflow is shared to allow rule-based automation on the repo.
+ Like any other CI/CD automation, the source code is being scanned in the repo and is not shared with any external services. By default, only metadata related to and affecting the workflow is shared to allow rule-based automation on the repo. Your own gitStream plugins that may connect to other services, such as using the [`askAI`](/filter-function-plugins/#askai) plugin which will provide context to the configured model provider.

The changes:

  • Improve question clarity
  • Add important context about plugin data sharing
  • Fix formatting consistency

Please rate this LinearB_AI review:

  • Excellent
  • Good Enough
  • Needs Improvement

Copy link
Contributor

gitstream-cm bot commented Jan 14, 2025

A screenshot of the relevant part of docs after the changes is a life saver 🛟

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants