merging main into docs-branch (#405)

* fix: Add style attribute to all relevant components (#384) * chore(main): release 0.5.1 (#385) * fix: Restore deleted publish code (#387) * chore(main): release 0.5.2 (#388) * fix: Free versions and proper environment installs (#389) * fix: Free versions and proper environment installs * fix: Missed condition * chore(main): release 0.5.3 (#390) * fix: updated docs with higher res screenshots and fixed typos/errors in code (#382) * updated docs with higher res screenshots and to fix typos/errors in code * minor docs updates and saving/reading as python file * made slight change to handle multine cell definitions in parser * restored backend and moved saving to python file to new branch * fix: Restore yarn lock and remove unwanted changes * fix: space --------- Co-authored-by: Red Giuliano <red.giuliano@zero-true.com> Co-authored-by: Carson-Shaar <carson.shaar@zero-true.com> * fix(solve-parser-issue): Solved parsing issue with list of variables define globally and in function together (#379) * fix(solve-bug-in-linting): Updated the linting command to check only stable errors according to ruff and ignore others (#383) * fix: Actions artifacts v3 is closing down and other upgrades (#391) * chore(updated-cell-traversal): Updated cell traversal to move between notebook cells using up and down arrow (#381) * chore(updated-cell-traversal): Updated cell traversal to move between notebook cells using up and down arrow * fix(solve-text-component-issue): Fix traversal in text component * chore(optimize-ui): Improved UI for file explorer and copilot components (#380) * chore(optimize-ui): Improved UI for file explorer and copilot components * Updated bug found in copilot unauthorized state * fix(UI-file-explorer): Maintaining proper ui with file item * chore(file-explorer-UI): Optimize UI to open and clsoe folders when clicked on title * chore(optimize-ui-file-explorer):OPtimized file explorer by providing features like right click menu,subfolder item creation, crag drop items, and click open titles in explorer * fix(drag-drop-ui): imporove drag drop funcitonalities * fix(text-cell-traversal): Debugged the issue with text cell traversal and handled all the edge cases for traversal with different decorations and elements (#395) fix(text-cell-traversal):Debugged the issue with text cell traversal and handled all the edge cases for traversal with different decorations and elements * chore(update-copilot-reference): Updated copilot added reference for zero true components to suggest (#396) * Added document context for zero_true components to provide reference for copilot * chore(update-copilot-reference): Updated copilot added reference for zero true components to suggest * fix: Include docs files for copilot in package (#397) * chore(deps): bump nanoid from 3.3.7 to 3.3.8 in /zt_frontend (#398) Bumps [nanoid](https://github.com/ai/nanoid) from 3.3.7 to 3.3.8. - [Release notes](https://github.com/ai/nanoid/releases) - [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md) - [Commits](ai/nanoid@3.3.7...3.3.8) --- updated-dependencies: - dependency-name: nanoid dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(logic-optimizations): Added copy button on copilot code,update search logic in file explorer, optimized copilot suggesstions (#402) * fix(logic-optimizations): Added copy button on copilot code,update search logic in file explorer, optimized copilot suggesstions * fix(copilot-suggesstions): improved some context and increased debounce --------- Co-authored-by: Carson-Shaar <120226019+Carson-Shaar@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Carson-Shaar <120226019+Carson-Shaar@users.noreply.github.com> Co-authored-by: Jonathan Meyers <jmymy@users.noreply.github.com> Co-authored-by: Red Giuliano <red.giuliano@zero-true.com> Co-authored-by: Carson-Shaar <carson.shaar@zero-true.com> Co-authored-by: Priya Kanabar <119392691+priyakanabar-crest@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Zero-True · Jan 3, 2025 · 4993559 · 4993559
1 parent fa94ea8
commit 4993559
Show file tree

Hide file tree

Showing 80 changed files with 5,624 additions and 7,284 deletions.
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
@@ -11,7 +11,7 @@ jobs:
     outputs:
       any_changed: ${{ steps.check-changes.outputs.any_changed }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 2
       - name: Check for changes in zt_frontend/
@@ -35,7 +35,7 @@ jobs:
           cp -r dist/ ../zt_backend/dist_app/
       - name: Upload Frontend Artifacts
         if: steps.check-changes.outputs.any_changed == 'true'
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: frontend-files
           path: |
@@ -52,9 +52,9 @@ jobs:
         python-version: ['3.9', '3.10', '3.11','3.12']
     steps:
       - name: Checkout Code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       - name: Setup Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       - name: Remove Existing Frontend Files (Windows)
@@ -72,7 +72,7 @@ jobs:
           rm -rf zt_backend/dist_app
       - name: Download Frontend Files
         if: needs.build-frontend.outputs.any_changed == 'true'
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: frontend-files
           path: zt_backend/

diff --git a/.github/workflows/publish_package.yml b/.github/workflows/publish_package.yml
@@ -14,7 +14,7 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: 3.9
 
@@ -39,7 +39,7 @@ jobs:
           python -m build
 
       - name: Store the distribution packages
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: python-package-distributions
           path: dist/
@@ -56,7 +56,7 @@ jobs:
 
     steps:
       - name: Download all the dists
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: python-package-distributions
           path: dist/

diff --git a/.github/workflows/release-please.yml b/.github/workflows/release-please.yml
@@ -13,7 +13,7 @@ jobs:
         runs-on: ubuntu-latest
         steps:
             - name: release-please-action
-              uses: google-github-actions/release-please-action@v4.1.0
+              uses: google-github-actions/release-please-action@v4.1.1
               with:
                   token: ${{ secrets.RELEASE_PLEASE_TOKEN }}
                   release-type: python
diff --git a/.github/workflows/update-docs.yml b/.github/workflows/update-docs.yml
@@ -12,7 +12,7 @@ jobs:
       uses: actions/checkout@v4
 
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: '3.x'
 

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,26 @@
 # Changelog
 
+## [0.5.3](https://github.com/Zero-True/zero-true/compare/v0.5.2...v0.5.3) (2024-12-10)
+
+
+### Bug Fixes
+
+* Free versions and proper environment installs ([#389](https://github.com/Zero-True/zero-true/issues/389)) ([6729dd2](https://github.com/Zero-True/zero-true/commit/6729dd2a6e2109bd40d3ac8464622f1c95b44889))
+
+## [0.5.2](https://github.com/Zero-True/zero-true/compare/v0.5.1...v0.5.2) (2024-12-09)
+
+
+### Bug Fixes
+
+* Restore deleted publish code ([#387](https://github.com/Zero-True/zero-true/issues/387)) ([24f371d](https://github.com/Zero-True/zero-true/commit/24f371de547b5866352f22725682888138e6d63c))
+
+## [0.5.1](https://github.com/Zero-True/zero-true/compare/v0.5.0...v0.5.1) (2024-12-09)
+
+
+### Bug Fixes
+
+* Add style attribute to all relevant components ([#384](https://github.com/Zero-True/zero-true/issues/384)) ([68f42f7](https://github.com/Zero-True/zero-true/commit/68f42f7eb76ef58b9e1263e793b2490a58fd4f9e))
+
 ## [0.5.0](https://github.com/Zero-True/zero-true/compare/v0.4.9...v0.5.0) (2024-11-25)
 
 

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,6 +1,7 @@
 recursive-include zt_backend/dist_dev *
 recursive-include zt_backend/dist_app *
 recursive-include copilot/node_modules *
+recursive-include mintlify-docs/Components *
 include copilot/client.js
 include zt_cli/log_config.yaml
 include zt_dev_cli/log_config.yaml
diff --git a/copilot/context_extractor.py b/copilot/context_extractor.py
@@ -0,0 +1,121 @@
+from dataclasses import dataclass
+from typing import Dict, List, Optional
+import os
+import re
+
+@dataclass
+class ComponentProperty:
+    name: str
+    type: str
+    description: str
+
+@dataclass
+class ComponentMetadata:
+    name: str
+    properties: List[ComponentProperty]
+    example: str
+
+class MdxComponentParser:
+    def __init__(self, mdx_directory: str):
+        self.mdx_directory = mdx_directory
+        self._component_cache: Dict[str, ComponentMetadata] = {}
+
+    def parse_mdx_file(self, filename: str) -> Optional[ComponentMetadata]:
+        if filename in self._component_cache:
+            return self._component_cache[filename]
+
+        filepath = os.path.join(self.mdx_directory, filename)
+        if not os.path.exists(filepath):
+            return None
+
+        with open(filepath, 'r', encoding='utf-8') as file:
+            content = file.read()
+
+            component_match = re.search(r'<ResponseField name="zero_true\.(\w+)"', content)
+            if not component_match:
+                return None
+            component_name = component_match.group(1)
+
+            properties = self._extract_properties(content)
+            example = self._extract_complete_example(content, component_name)
+
+            metadata = ComponentMetadata(
+                name=component_name,
+                properties=properties,
+                example=example
+            )
+            self._component_cache[filename] = metadata
+            return metadata
+
+    def _extract_properties(self, content: str) -> List[ComponentProperty]:
+        properties = []
+        property_pattern = r'<Accordion title="([^"]+)">\s+\*\*([^:]+):\s*([^)]+)\*\*:\s*([^<]+)'
+        matches = re.finditer(property_pattern, content, re.MULTILINE | re.DOTALL)
+
+        for match in matches:
+            name, prop_name, type_str, description = match.groups()
+            properties.append(ComponentProperty(
+                name=name.strip(),
+                type=type_str.strip(),
+                description=description.strip()
+            ))
+        return properties
+
+    def _extract_complete_example(self, content: str, component_name: str) -> str:
+        """Extract complete example with all parameters but without comments."""
+        example_match = re.search(r'```python(.*?)```', content, re.DOTALL)
+        if not example_match:
+            return ""
+
+        example = example_match.group(1)
+
+        component_pattern = rf'(\w+)\s*=\s*zt\.{component_name}\((.*?)\)'
+        component_match = re.search(component_pattern, example, re.DOTALL)
+
+        if not component_match:
+            return ""
+
+        var_name = component_match.group(1)
+        params_block = component_match.group(2)
+
+        params_list = []
+        lines = params_block.split('\n')
+        for line in lines:
+            param_line = re.sub(r'#.*$', '', line).strip()
+            if param_line:
+                param_match = re.match(r'(\w+)\s*=\s*([^,]+)(?:,|$)', param_line)
+                if param_match:
+                    param_name, param_value = param_match.groups()
+                    param_value = param_value.strip()
+                    params_list.append(f"{param_name}={param_value}")
+
+        clean_params = ','.join(params_list)
+        clean_example = f"{var_name}=zt.{component_name}({clean_params})"
+
+        return clean_example
+
+    def generate_completion_context(self) -> str:
+        """Generate minimal context with no extra spacing."""
+        context_parts = [
+        "/* COPILOT RULES",
+        "- Only use components listed below - no custom components",
+        "- No empty lines in suggestions",
+        "- No indentation",
+        "- No alignment formatting",
+        "- Minimal spaces between parameters",
+        "- Don't Create own parameters for component only show if present in usage"
+        "*/",
+        ]
+
+        # Component definitions
+        context_parts.append("\n/* Component definitions and their common usage:")
+        for filename in os.listdir(self.mdx_directory):
+            if filename.endswith('.mdx'):
+                metadata = self.parse_mdx_file(filename)
+                if metadata:
+                    example = metadata.example if metadata.example else f"sample_{metadata.name.lower()}=zt.{metadata.name}(id='sample_{metadata.name.lower()}')"
+                    context_parts.append(f"Component: {metadata.name}")
+                    context_parts.append(f"Usage: {example}")
+
+        context_parts.append("*/")
+        return '\n'.join(context_parts)
diff --git a/copilot/copilot.py b/copilot/copilot.py
@@ -12,10 +12,15 @@
 from pathlib import Path
 import asyncio
 import traceback
+from copilot.context_extractor import MdxComponentParser
+from zt_backend.config import settings
+import importlib.resources
+
+
 
 copilot_app = FastAPI()
 
-copilot_app.add_middleware(
+copilot_app.add_middleware( 
     CORSMiddleware,
     allow_origins=["*"],
     allow_credentials=False,
@@ -27,6 +32,8 @@
 copilot_enabled = False
 copilot_doc_open = False
 version = 0
+MDX_DIRECTORY = importlib.resources.files("mintlify-docs") / "Components" 
+mdx_parser = MdxComponentParser(MDX_DIRECTORY)
 
 def is_docker():
     cgroup = Path('/proc/self/cgroup')
@@ -60,7 +67,8 @@ async def start_node_server_route():
         await start_node_server()
         return {"message": 'Node server started successfully'}
     except Exception as e:
-        raise HTTPException(status_code=500, detail=str(e))
+        print(f"An error occurred while starting server: {traceback.format_exc()}")
+        raise HTTPException(status_code=500, detail=str(traceback.format_exc()))
 
 @copilot_app.post("/check_status",response_model=Union[CopilotPayloadSignInConfirm,CopilotPayloadSignOut])
 async def check_status(req: BlankRequest):
@@ -111,27 +119,42 @@ async def sign_in_confirm(req: BlankRequest):
         return response.json()
     except requests.RequestException as e:
         raise HTTPException(status_code=500, detail=str(e))
-    
+
 async def text_document_did_open(params: DidOpenTextDocumentParams):
     global copilot_enabled
     if copilot_enabled:
         try:
-            response = requests.post(f"{NODE_SERVER_URL}/sendNotification", json={"method": "textDocument/didOpen", "params": params.dict()})
+            component_context = mdx_parser.generate_completion_context()
+
+            # Combine all context
+            full_context = f"/*\n{component_context}/\n"
+
+            if component_context not in params.textDocument.text:
+                params.textDocument.text = full_context + params.textDocument.text
+            payload = {
+                "method": "textDocument/didOpen",
+                "params": params.dict()
+            }
+            response = requests.post(
+                f"{NODE_SERVER_URL}/sendNotification",
+                json=payload
+            )
             response.raise_for_status()
             return {"message": "Notification sent successfully"}
         except requests.RequestException as e:
             raise HTTPException(status_code=500, detail=str(e))
 
-@async_debounce(0.2)
+@async_debounce(0.65)
 async def text_document_did_change(params):
     global copilot_enabled
     global copilot_doc_open
     global version
     if copilot_enabled:
         try:
             params = DidChangeTextDocumentParams(**params)
+            content_change = params.contentChanges[0]
             if not copilot_doc_open:
-                open_document = TextDocumentItem(uri=params.textDocument.uri, languageId="python", version=version, text=params.contentChanges[0].text)
+                open_document = TextDocumentItem(uri=params.textDocument.uri, languageId="python", version=version, text=content_change.text)
                 open_request = DidOpenTextDocumentParams(textDocument=open_document)
                 response = await text_document_did_open(open_request)
                 copilot_doc_open = True
@@ -150,15 +173,25 @@ async def get_completions(params: GetCompletionsParams):
     global copilot_enabled
     global copilot_doc_open
     global version
-    if copilot_enabled and copilot_doc_open:
-        try:
-            params.doc.version = version
-            response = requests.post(f"{NODE_SERVER_URL}/sendRequest", json={"method": "getCompletions", "params": params.dict()})
-            response.raise_for_status()
-            return response.json()
-        except requests.RequestException as e:
-            raise HTTPException(status_code=500, detail=str(e))
+    if not copilot_enabled or not copilot_doc_open:
+        return CopilotGetCompletionsResult(completions=[])  # Return empty result instead of None
+
+    try:
+        params.doc.version = version
+        response = requests.post(f"{NODE_SERVER_URL}/sendRequest", json={"method": "getCompletions", "params": params.dict()})
+        response.raise_for_status()
+
+        result = response.json()
+        if result is None:
+            print("Warning: Node server returned None response")
+            return CopilotGetCompletionsResult(completions=[])
+
+        return result
 
+    except requests.RequestException as e:
+        print(f"Request error: {str(e)}")
+        raise HTTPException(status_code=500, detail=str(e))
+
 @copilot_app.post("/accept_completion")
 async def check_status(req: AcceptRequest):
     try:

diff --git a/copilot/copilot_models.py b/copilot/copilot_models.py
@@ -74,7 +74,7 @@ class Completion(BaseModel):
     displayText: str
 
 class CopilotGetCompletionsResult(BaseModel):
-    completions: List[Completion]
+    completions: List[Completion] = Field(default_factory=list)
 
 class AcceptRequest(BaseModel):
     uuid: str

diff --git a/mintlify-docs/__init__.py b/mintlify-docs/__init__.py