The command /ask fails on large contexts

[gentlementlegen]

! No answer from OpenAI

<!--
{
  "caller": "issueCommentCreatedCallback"
}
-->

Originally posted by @ubiquity-os-beta[bot] in ubiquity-os-marketplace/text-vector-embeddings#67 (comment)

What happened

The /ask command failed to run, due to a context being too large, thus failing with

{
       "res": {
         "error": {
           "message": "Provider returned error",
           "code": 400,
           "metadata": {
             "raw": "{\n  \"error\": {\n    \"message\": \"This model's maximum context length has been exceeded. Please reduce the length of the messages.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"messages\",\n    \"code\": \"context_length_exceeded\"\n  }\n}",
             "provider_name": "OpenAI"
           }
         },
         "user_id": "user_2njZil2BmAzm3iDZvVsx69KWg4A"
       },
       "caller": "Completions.createCompletion"
     }

Multiple problems here:

• it should use postComment so we get the run url and the proper error stack
• there should not be two comments for the failure
• the context should be truncated in some way to allow /ask to run properly

What was expected

/ask should truncate the context so it runs properly and still gives an accurate answer.

How to reproduce

The easiest would probably to run /ask against the aforementioned issue, or create an issue with a very long content.

[ubiquity-os-beta]

Note

The following contributors may be suitable for this task:

Keyrxng

81% Match ubiquity-os-marketplace/command-ask#26

[0x4007]

@gentlementlegen you should always suggest a time estimate when filing new issues. @shiv810 please feel free to add the time label if you think you can handle this.

[gentlementlegen]

I must say I didn't work on that plugin and wouldn't be able to give an estimate on this one so I left it empty.

[ubiquity-os-beta]

 [ 600 WXDAI ] 

@shiv810

Contributions Overview

View	Contribution	Count	Reward
Issue	Task	1	600
Review	Comment	15	0

Conversation Incentives

Comment	Formatting	Relevance	Priority	Reward
Resolves #48 - [X] Control the depth, with which you want to e…	1 content: content: p: score: 0 elementCount: 3 ul: score: 0 elementCount: 1 li: score: 0.5 elementCount: 2 result: 1 regex: wordCount: 24 wordValue: 0 result: 0	0.8	3	0
Added a description	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 3 wordValue: 0 result: 0	0.6	3	0
Fixed	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 1 wordValue: 0 result: 0	0.5	3	0
Updated this with pagination	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 4 wordValue: 0 result: 0	0.7	3	0
Isn't this all expected, since we're trying to extract as many r…	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 62 wordValue: 0 result: 0	0.4	3	0
Updated with the pagination, handles the PRRC comments and the …	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 13 wordValue: 0 result: 0	0.4	3	0
[PR Review Comment](https://github.com/ShivTestOrg/test-public/p…	5 content: content: p: score: 0 elementCount: 1 a: score: 5 elementCount: 1 result: 5 regex: wordCount: 3 wordValue: 0 result: 0	0.6	3	0
Fixed them, has to set them to const value for testing.	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 11 wordValue: 0 result: 0	0.5	3	0
No, re-ranking and embedding search, is different. Re-Ranking, u…	0 content: content: p: score: 0 elementCount: 3 result: 0 regex: wordCount: 87 wordValue: 0 result: 0	0.4	3	0
The Claude SDK has never worked for me with Openrouter. The only…	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 25 wordValue: 0 result: 0	0.2	3	0
I can manage these cases on a case-by-case basis; the alternativ…	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 31 wordValue: 0 result: 0	0.3	3	0
QA: [Without the structured representation](https://github.com…	10 content: content: p: score: 0 elementCount: 3 a: score: 5 elementCount: 2 result: 10 regex: wordCount: 7 wordValue: 0 result: 0	0.1	3	0
QA:1. Normal Discussion Reference [Issue](https://github.com/S…	16.5 content: content: p: score: 0 elementCount: 4 ol: score: 0 elementCount: 1 li: score: 0.5 elementCount: 3 a: score: 5 elementCount: 3 result: 16.5 regex: wordCount: 15 wordValue: 0 result: 0	0.4	3	0
The Openrouter account has run out of credits and needs to be to…	0 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 14 wordValue: 0 result: 0	0.3	3	0
Earlier, we retrieved all the contexts and tried to include ever…	0 content: content: p: score: 0 elementCount: 3 result: 0 regex: wordCount: 149 wordValue: 0 result: 0	0.9	3	0

 [ 17.46 WXDAI ] 

@0x4007

Contributions Overview

View	Contribution	Count	Reward
Issue	Comment	1	2.55
Review	Comment	9	14.91

Conversation Incentives

Comment	Formatting	Relevance	Priority	Reward
@gentlementlegen you should always suggest a time estimate when …	1.7 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 28 wordValue: 0.1 result: 1.7	0.5	3	2.55
QA seems fine	0.25 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 3 wordValue: 0.1 result: 0.25	0.2	3	0.15
Why string and number? Seems wrong	0.46 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 6 wordValue: 0.1 result: 0.46	0.6	3	0.828
```suggestionexport type GraphQlIssueSearchResult =…	0.77 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 11 wordValue: 0.1 result: 0.77	0.5	3	1.155
So you're reranking based on embeddings. And the embeddings I as…	2.2 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 38 wordValue: 0.1 result: 2.2	0.7	3	4.62
Do you have QA for this	0.46 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 6 wordValue: 0.1 result: 0.46	0.4	3	0.552
Shouldn't we look this up from a collection based on what model …	1 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 15 wordValue: 0.1 result: 1	0.6	3	1.8
Shouldn't we be using open router? I wonder if we should use Cla…	1.28 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 20 wordValue: 0.1 result: 1.28	0.5	3	1.92
It's a bit difficult for me to wrap my head around how much cont…	3.15 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 58 wordValue: 0.1 result: 3.15	0.3	3	2.835
Can we coordinate in Telegram to fix this? As I understand it's …	1.75 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 29 wordValue: 0.1 result: 1.75	0.2	3	1.05

 [ 62.475 WXDAI ] 

@gentlementlegen

Contributions Overview

View	Contribution	Count	Reward
Issue	Specification	1	43.92
Issue	Comment	1	0
Review	Comment	8	18.555

Conversation Incentives

Comment	Formatting	Relevance	Priority	Reward
```diff! No answer from OpenAI```_…	14.64 content: content: p: score: 0 elementCount: 8 em: score: 0 elementCount: 1 h2: score: 1 elementCount: 3 ul: score: 0 elementCount: 1 li: score: 0.5 elementCount: 3 a: score: 5 elementCount: 1 result: 9.5 regex: wordCount: 103 wordValue: 0.1 result: 5.14	1	3	43.92
I must say I didn't work on that plugin and wouldn't be able to …	1.65 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 27 wordValue: 0.1 result: 1.65	0	3	0
Wouldn't these logs be `debug` instead of `info`?	0.52 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 7 wordValue: 0.1 result: 0.52	0.2	3	0.312
Would be nice to have a description here	0.59 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 8 wordValue: 0.1 result: 0.59	0.5	3	0.885
This seems dangerous and lead to issues before, like in the foll…	4.69 content: content: p: score: 0 elementCount: 1 ul: score: 0 elementCount: 1 li: score: 0.5 elementCount: 4 result: 2 regex: wordCount: 48 wordValue: 0.1 result: 2.69	0.7	3	9.849
Seems strange to return either or Record or `void`, `…	0.83 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 12 wordValue: 0.1 result: 0.83	0.6	3	1.494
Might be safer to go for `100` (which I believe is the l…	0.83 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 12 wordValue: 0.1 result: 0.83	0.4	3	0.996
Don't you need to use pagination for all of these, or you don't …	1.28 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 20 wordValue: 0.1 result: 1.28	0.7	3	2.688
To be deleted	0.25 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 3 wordValue: 0.1 result: 0.25	0	3	0
If you think this has no side effect then that is fine. I mentio…	2.59 content: content: p: score: 0 elementCount: 1 result: 0 regex: wordCount: 46 wordValue: 0.1 result: 2.59	0.3	3	2.331