feat: fix parser #95

anfibiacreativa · 2023-10-27T10:20:11Z

Purpose

Fix the broken parsing for the follow up questions in current version. This version does not decouple the components or introduces code optimizations.
It minimally varies color scheme and the rendering format of follow up questions (as shown in GIF)

Does this introduce a breaking change?

[ ] Yes
[ x] No

Pull Request Type

What kind of change does this Pull Request introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ x] Other... Please describe: cosmetic changes

How to Test

Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

Test the code
No test was added

What to Check

Follow up questions are properly rendered, as displayed in the GIF

packages/chat-component/src/core/parser/index.ts

sinedied

Does it work when it's not streaming?
As I recall there is a separate path for non-streaming mode

anfibiacreativa · 2023-10-27T18:37:18Z

Does it work when it's not streaming? As I recall there is a separate path for non-streaming mode

Yes! Uploading gif to teams since it's too large.

manekinekko · 2023-10-30T08:32:20Z

packages/chat-component/src/core/parser/index.ts

@@ -52,9 +52,10 @@ export async function parseStreamedMessages({
    // followup questions are marked either with the word 'Next Questions:' or '<<text>>' or both at the same time
    // these markers may be split across multiple chunks, so we need to buffer them!
    // TODO: remove all this logic from the frontend and implement a solution on the backend or with TypeChat
-
+    // we start by creating a buffer when we match the first marker
    const matchedFollowupQuestionMarker =
      (!isFollowupQuestion && chunkValue.includes('Next')) || chunkValue.includes('<<');


btw should we still parse Next markers? Is the new model still using Next to format followup questions?

If the updated model uses only << and >> to format followup questions, then I'd suggest changing the condition to:

const matchedFollowupQuestionMarker = !isFollowupQuestion && chunkValue.includes('<<');

Good question. @sinedied can you clarify?

manekinekko · 2023-10-30T08:39:32Z

packages/chat-component/src/core/parser/index.ts

    } else if (chunkValue.includes('<<') && isFollowupQuestion) {
      isFollowupQuestion = true;
      continue;
+      // this updates the index, so we add each question to a different array entry
+      // to simplify styling
    } else if (chunkValue.includes('?>') || chunkValue.includes('>')) {


why are we checking for ?> && > and not ?>> && >>?

Because in all tests I ran, ?> and the last > were returned as part of 2 different chunks, and there was never a match.

manekinekko · 2023-10-30T08:39:59Z

Love the comments, thank you!

* added initial test support for the playwright tests * add snapshot and thought process test along with default network response * playwright config update * ux update * test screenshot updates * update tests * no stream har files * embed the body in the har files * stream har file updated * update the screenshot for response formatting * add screenshots for firefox and webkit tests * fix the testid attribute * remove playwright workflow for now * add the github action for running local tests and fix a broken test * only upload artifacts on failure * ensure playwright browsers are installed in devcontainer * move the tests to a e2e * update types * keep playwright tests separate * cleanup tests

* setup env for k6 * test: load tests to package.json * test: add tests * test: add load test * test: granualize metrics * test: have a standard workload * test: load 50 users 1 request/sec/user * test: log the response failures * test: randomize the iteration interval * test: adjust the load scenario * test: doc update * test: move globals to shared esconfig

* chore: update followup questions preliminary * chore: update styles * chore: restyle followup questions * chore: fix parser for follow up questions (before TC) * chore: fix bug and adapt styles * chore: add margin to citations * chore: remove commented regex * fix: remove unnecessary regex and replace * chore: update global config * chore: add question icon * chore: update styles * chore: update comments to reflect decisions * chore: refine cosmetic styles * chore: replace icons with iconcloud set * fix: fix minor bug with index replacement * test: add E2E playwright tests for web app (#89) * added initial test support for the playwright tests * add snapshot and thought process test along with default network response * playwright config update * ux update * test screenshot updates * update tests * no stream har files * embed the body in the har files * stream har file updated * update the screenshot for response formatting * add screenshots for firefox and webkit tests * fix the testid attribute * remove playwright workflow for now * add the github action for running local tests and fix a broken test * only upload artifacts on failure * ensure playwright browsers are installed in devcontainer * move the tests to a e2e * update types * keep playwright tests separate * cleanup tests * docs: update readme with auth info * test: add load tests (#93) * setup env for k6 * test: load tests to package.json * test: add tests * test: add load test * test: granualize metrics * test: have a standard workload * test: load 50 users 1 request/sec/user * test: log the response failures * test: randomize the iteration interval * test: adjust the load scenario * test: doc update * test: move globals to shared esconfig * test: add playwright test for ask interaction (#94) * ci: disable playwright for demo * ci: revert previous commit and disable from UI --------- Co-authored-by: shibbas <26466942+shibbas@users.noreply.github.com>

anfibiacreativa added 7 commits October 26, 2023 21:08

chore: update followup questions preliminary

d26e185

chore: update styles

e9075c6

chore: restyle followup questions

4868a48

chore: fix parser for follow up questions (before TC)

fb45e89

chore: fix bug and adapt styles

8b04c0b

chore: add margin to citations

5a9e5aa

chore: remove commented regex

80436e3

anfibiacreativa requested review from sinedied and manekinekko October 27, 2023 10:20

manekinekko reviewed Oct 27, 2023

View reviewed changes

packages/chat-component/src/core/parser/index.ts Outdated Show resolved Hide resolved

manekinekko reviewed Oct 27, 2023

View reviewed changes

packages/chat-component/src/core/parser/index.ts Show resolved Hide resolved

manekinekko reviewed Oct 27, 2023

View reviewed changes

packages/chat-component/src/core/parser/index.ts Outdated Show resolved Hide resolved

sinedied reviewed Oct 27, 2023

View reviewed changes

anfibiacreativa added 4 commits October 27, 2023 14:52

fix: remove unnecessary regex and replace

6449f69

chore: update global config

e0de7af

chore: add question icon

1362575

chore: update styles

8fe2d31

chore: update comments to reflect decisions

f79735c

manekinekko reviewed Oct 30, 2023

View reviewed changes

anfibiacreativa and others added 8 commits October 30, 2023 10:28

chore: refine cosmetic styles

65d3533

chore: replace icons with iconcloud set

bb0baf3

fix: fix minor bug with index replacement

3ea28cd

docs: update readme with auth info

0ff12b6

test: add playwright test for ask interaction (#94)

f6719f1

ci: disable playwright for demo

444eeef

ci: revert previous commit and disable from UI

7b1abaf

anfibiacreativa merged commit 42f29cd into main Oct 31, 2023
5 checks passed

anfibiacreativa changed the title ~~Feat/fix parser~~ feat: fix parser Oct 31, 2023

sinedied mentioned this pull request Nov 8, 2023

[chat-component] fix follow-up questions parsing #81

Closed

anfibiacreativa deleted the feat/fix-parser branch January 19, 2024 14:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: fix parser #95

feat: fix parser #95

anfibiacreativa commented Oct 27, 2023

sinedied left a comment

anfibiacreativa commented Oct 27, 2023

manekinekko Oct 30, 2023

manekinekko Oct 30, 2023

anfibiacreativa Oct 30, 2023

manekinekko Oct 30, 2023

anfibiacreativa Oct 30, 2023 •

edited

Loading

manekinekko commented Oct 30, 2023

feat: fix parser #95

feat: fix parser #95

Conversation

anfibiacreativa commented Oct 27, 2023

Purpose

Does this introduce a breaking change?

Pull Request Type

How to Test

What to Check

sinedied left a comment

Choose a reason for hiding this comment

anfibiacreativa commented Oct 27, 2023

manekinekko Oct 30, 2023

Choose a reason for hiding this comment

manekinekko Oct 30, 2023

Choose a reason for hiding this comment

anfibiacreativa Oct 30, 2023

Choose a reason for hiding this comment

manekinekko Oct 30, 2023

Choose a reason for hiding this comment

anfibiacreativa Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

manekinekko commented Oct 30, 2023

anfibiacreativa Oct 30, 2023 •

edited

Loading