Updating Enrichment status #1544

shreyaspimpalgaonkar · 2024-10-31T19:23:35Z

Important

Update enrichment and extraction status handling across workflows, services, and database interactions, including new status types and endpoint in management_router.py.

Behavior:
- Update handling of kg_enrichment_status and kg_extraction_status in kg_workflow.py, ingestion_workflow.py, and simple/kg_workflow.py.
- Modify on_failure methods to update status to FAILED for both enrichment and extraction workflows.
- Add new endpoint /r2r_project_name in management_router.py to return project name from environment variable.
Database:
- Add kg_enrichment_status to get_collections_overview() in collection.py.
- Update set_workflow_status() and get_workflow_status() in document.py to handle new status types.
Models:
- Add kg_enrichment_status to CollectionOverviewResponse in responses.py.
- Add methods table_name() and id_column() to IngestionStatus, KGExtractionStatus, and KGEnrichmentStatus in document.py.
Misc:
- Minor formatting and import changes in ingestion.py and database.py.

^{This description was created by}^{for 24abc6d. It will automatically update as commits are pushed.}

* Feature/tweak actions (#1507) * up * tweak actions * adding bin sup and making it default * tested and vetted

* up * set verification to default false

* cleanup * cleanup prompt mgmt * up * cleanup printout * cleanup new parser logic, set vlm as default for all providers * allow user to re-override

ellipsis-dev

❌ Changes requested. Reviewed everything up to ce73f2e in 1 minute and 55 seconds

More details

Looked at 623 lines of code in 11 files
Skipped 1 files when reviewing.
Skipped posting 2 drafted comments based on config settings.

1. py/core/main/orchestration/hatchet/kg_workflow.py:295

Draft comment:
Add a check to ensure input_data['kg_enrichment_settings'] is not None before accessing its attributes to avoid potential AttributeError.
Reason this comment was not posted:
Marked as duplicate.

2. py/core/main/services/ingestion_service.py:541

Draft comment:
Add a check to ensure chunk_enrichment_settings is not None before accessing its attributes to avoid potential AttributeError.
Reason this comment was not posted:
Marked as duplicate.

Workflow ID: wflow_dCuSIltlcbcl6xag

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ellipsis-dev · 2024-10-31T19:25:37Z

py/core/main/orchestration/hatchet/ingestion_workflow.py

+                        or {},
+                    )
+
+                if chunk_enrichment_settings.enable_chunk_enrichment:


Add a check to ensure chunk_enrichment_settings is not None before accessing chunk_enrichment_settings.enable_chunk_enrichment to avoid potential AttributeError.

Suggested change

if chunk_enrichment_settings.enable_chunk_enrichment:

if chunk_enrichment_settings and chunk_enrichment_settings.enable_chunk_enrichment:

ellipsis-dev · 2024-10-31T19:25:37Z

py/core/main/orchestration/simple/kg_workflow.py

@@ -61,32 +68,50 @@ async def enrich_graph(input_data):



Add a check to ensure input_data['kg_enrichment_settings'] is not None before accessing its attributes to avoid potential AttributeError.

ellipsis-dev · 2024-10-31T19:25:37Z

py/shared/abstractions/document.py

+        return self.value
+
+    @classmethod
+    def table_name(cls) -> str:


Duplicate methods table_name and id_column found in the same file. Consider reusing the existing methods at lines 149-155.

document.py

NolanTrem · 2024-10-31T20:34:46Z

py/core/main/api/management_router.py

@@ -887,3 +887,10 @@ async def delete_conversation(
        ) -> WrappedDeleteResponse:
            await self.service.delete_conversation(conversation_id)
            return None  # type: ignore
+
+        @self.router.get("/r2r_project_name")


Can we add this to the server settings?

vercel · 2024-10-31T20:43:56Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
yc_demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 31, 2024 8:54pm
yc-demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 31, 2024 8:54pm

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
recommendation_platform	⬜️ Ignored (Inspect)			Oct 31, 2024 8:54pm

ellipsis-dev

👍 Looks good to me! Incremental review on ef87b47 in 19 seconds

More details

Looked at 47 lines of code in 2 files
Skipped 0 files when reviewing.
Skipped posting 1 drafted comments based on config settings.

1. py/core/main/api/management_router.py:887

Draft comment:
The PR description mentions adding a new endpoint /r2r_project_name, but this code shows it was removed. Please update the PR description to reflect the actual changes.
Reason this comment was not posted:
Comment did not seem useful.

Workflow ID: wflow_HNBDHVF8EsKc6wKY

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ellipsis-dev

👍 Looks good to me! Incremental review on 24abc6d in 19 seconds

More details

Looked at 29 lines of code in 2 files
Skipped 0 files when reviewing.
Skipped posting 2 drafted comments based on config settings.

1. py/shared/api/models/management/responses.py:59

Draft comment:
Remove the commented-out r2r_version line if it's not needed, or uncomment it if it is needed.
Reason this comment was not posted:
Confidence changes required: 50%
The commented-out line for r2r_version in AppSettingsResponse should be removed if it's not needed, or uncommented if it is needed. Leaving commented-out code can lead to confusion.

2. py/core/main/services/management_service.py:204

Draft comment:
Remove the commented-out r2r_version line if it's not needed, or uncomment it if it is needed.
Reason this comment was not posted:
Confidence changes required: 50%
The commented-out line for r2r_version in app_settings method should be removed if it's not needed, or uncommented if it is needed. Leaving commented-out code can lead to confusion.

Workflow ID: wflow_fBAuJj5LdYg0gITx

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

shreyaspimpalgaonkar and others added 14 commits October 23, 2024 16:42

minor fixes

590ad92

up

4c271e5

adding bin support and make it default (#1508)

610c1db

* Feature/tweak actions (#1507) * up * tweak actions * adding bin sup and making it default * tested and vetted

up (#1510)

2f674dd

* up * set verification to default false

up

255bf73

Merge remote-tracking branch 'origin/minor-fixes' into enrichment_status

41e0f33

cleanup (#1512)

080d8cb

* cleanup * cleanup prompt mgmt * up * cleanup printout * cleanup new parser logic, set vlm as default for all providers * allow user to re-override

Merge branch 'dev-minor' into enrichment_status

7acb9b9

Merge remote-tracking branch 'origin/main' into enrichment_status

ca6761a

cleanup; pre-commit

2b71f74

rm launch json

a57dbbb

checkin work

34b9158

Merge remote-tracking branch 'origin/main' into enrichment_status

232a148

up

ce73f2e

ellipsis-dev bot reviewed Oct 31, 2024

View reviewed changes

NolanTrem reviewed Oct 31, 2024

View reviewed changes

modify endpoint

ef87b47

ellipsis-dev bot reviewed Oct 31, 2024

View reviewed changes

vercel bot deployed to Preview – yc_demo October 31, 2024 20:45 View deployment

vercel bot deployed to Preview – yc-demo October 31, 2024 20:46 View deployment

up

24abc6d

ellipsis-dev bot reviewed Oct 31, 2024

View reviewed changes

vercel bot deployed to Preview – yc_demo October 31, 2024 20:53 View deployment

vercel bot deployed to Preview – yc-demo October 31, 2024 20:54 View deployment

NolanTrem merged commit 6fe18bd into main Oct 31, 2024
14 of 15 checks passed

NolanTrem deleted the enrichment_status branch October 31, 2024 21:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating Enrichment status #1544

Updating Enrichment status #1544

shreyaspimpalgaonkar commented Oct 31, 2024 •

edited by ellipsis-dev bot

Loading

ellipsis-dev bot left a comment

ellipsis-dev bot Oct 31, 2024

ellipsis-dev bot Oct 31, 2024

ellipsis-dev bot Oct 31, 2024

NolanTrem Oct 31, 2024

vercel bot commented Oct 31, 2024 •

edited

Loading

ellipsis-dev bot left a comment

ellipsis-dev bot left a comment

	if chunk_enrichment_settings.enable_chunk_enrichment:
	if chunk_enrichment_settings and chunk_enrichment_settings.enable_chunk_enrichment:

Updating Enrichment status #1544

Updating Enrichment status #1544

Conversation

shreyaspimpalgaonkar commented Oct 31, 2024 • edited by ellipsis-dev bot Loading

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

ellipsis-dev bot Oct 31, 2024

Choose a reason for hiding this comment

ellipsis-dev bot Oct 31, 2024

Choose a reason for hiding this comment

ellipsis-dev bot Oct 31, 2024

Choose a reason for hiding this comment

NolanTrem Oct 31, 2024

Choose a reason for hiding this comment

vercel bot commented Oct 31, 2024 • edited Loading

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

ellipsis-dev bot left a comment

Choose a reason for hiding this comment

shreyaspimpalgaonkar commented Oct 31, 2024 •

edited by ellipsis-dev bot

Loading

vercel bot commented Oct 31, 2024 •

edited

Loading