Research across multiple sources

[ausangshukla]

Is there a way to load data from the web, docs and a custom API, and use all the data across all the sources to generate a report. FRom the docs at https://docs.gptr.dev/docs/gpt-researcher/tailored-research I can see that we can provide a single report source. Is it possible to change that to provide an array of sources?

My use case

1. We have to write research reports on companies
2. The companies have give us a lot of thier documents
3. We also want to get news from the web about these companies
4. Further we have access to APIs that give us financial data about the companies

Now we want to use all this data to generate detailed analyst reports. Any help is appreciated.

Thanks

[smjure]

That would be helpful in my cases too. The web news resources you have are already good in my case, but I also have some paid subscriptions, which I download and then do the research twice, i.e. (1) 'Web' and (2) 'Docs' options. But this is not the same as GPT would interrelate the informations and produce the overall result. Please do add the option (3) 'Web & My Docs'.

[assafelovic]

@ausangshukla the feature request is on point! We'd definitely like to add a hybrid search experience later on. For now, the best would be to do two calls in parallel of GPTR once for web and once for docs. After you get both reports back, you can call an LLM with a prompt to create a final report from both initial reports. This practice is actually optimal by many users since it gives you more control over the various sources and outputs.

[ElishaKay]

Update: a hybrid search option is now available when instantiating the GPTResearcher class 🥳

Pass report_type: "hybrid" - this will leverage both docs & web

[ElishaKay]

There's a lot of interesting directions we can take Custom Docs & Custom Sources.

For example, users can create separate processes for ingesting data & query their custom vector stores

Alternatively, if we index each source with custom metadata, we should also be able to filter vector store search results

We can followup on improvements in the Discord's #managing-context channel