Error in generate_report: 'content'

[hherb]

Describe the bug
Since updating to version 0.10.9 (same with 0.10.10) I get an error "Error in generate_report: 'content'" after successful web scraping. This only happens when I use a custom log handler - the moment I comment out the custom logs handler the report gets generated as usual

This happens irrespective of LLM used (OpenAI default, ollama various ...) and regardless of type of research.

How can the log handler afect report generation that way?

To Reproduce
this is the relevant part of my python app:

class CustomLogsHandler:
    """A custom Logs handler class to handle JSON data."""
    def __init__(self):
        self.logs = []  # Initialize logs to store data

    def set_widget(self, widget):
        self.widget = widget

    async def send_json(self, data: Dict[str, Any]) -> None:
        """Send JSON data and log it."""
        
        self.logs.append(data)  # Append data to logs
        output = data['output']
        if starts_with_emoji(output):
            output = f"\n{output}"
        self.widget.stream(output)
        if data['metadata']:
            for item in data['metadata']:
                self.widget.stream(item)


async def do_research(query: str, report_type: str = "research_report", widget=None) -> str:
    research_settings()
    custom_logs_handler = CustomLogsHandler()
    researcher = GPTResearcher(query=query, 
                                report_type=report_type, 
                                config_path=None, 
                                websocket=custom_logs_handler,
                                )
    researcher.set_verbose(True)

    # Initialize the researcher
    with widget.add_step(status="running", running_title="Researching ...", success_title="job done!") as chatstep:    
        custom_logs_handler.set_widget(chatstep)
        await researcher.conduct_research()
    report = await researcher.write_report()
    return(report)

Log excerpt:
INFO: [07:42:51] ✍️ Writing report for 'aspartame consumption and diabetes risk'...
{'content': 'writing_report',
'metadata': None,
'output': "✍️ Writing report for 'aspartame consumption and diabetes risk'...",
'type': 'logs'}
{'output': '# Aspartame Consumption and Diabetes Risk: A Comprehensive '
'Analysis\n'
'\n',
'type': 'report'}
Error in generate_report: 'content'
INFO: [07:42:53] 📝 Report written for 'aspartame consumption and diabetes risk'
{'content': 'report_written',
'metadata': None,
'output': "📝 Report written for 'aspartame consumption and diabetes risk'",
'type': 'logs'}

Expected behavior
Previously, the report would be generated without problems

[assafelovic]

@kga245 any chance you can help with this?

[hherb]

I am trying already… hopefully will have found the problem in the next couple of days Horst

…

On Mon, 30 Dec 2024 at 17:03, Assaf Elovic ***@***.***> wrote: @kga245 <https://github.com/kga245> any chance you can help with this? — Reply to this email directly, view it on GitHub <#1052 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZYZTXIDY7ZDGDBQDFPXGD2IDV4XAVCNFSM6AAAAABULFJUZSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRVGEYDANRYGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[kga245]

Hi, @hherb . @assafelovic brought me into the loop because logging is an area I have recently worked on improving.

I have stabilized logging more recently, but I haven't experimented with custom logging. I'd love to know what you're trying to get. So let's start there if you you don't mind.

Here's what I can tell from the code you supplied:

The "Something went wrong!" error you are seeing is likely because:

• The custom handler isn't properly integrated with the frontend websocket system
• The backend isn't receiving or processing the websocket messages correctly
• There's a version mismatch between their frontend dependencies and what's expected

Alternatively, if you package up a branch and I am happy to see if I can diagnose locally.

[hherb]

Hi, will try. I am a MD working in remote hospitals, and only have time for coding when I get some time off - next 10 days I am working 12+ hours a day + on call during the night, so I won't be able to squeeze any debugging/coding in, but after that I will once I had some sleep. I have attached a minimal example of runnable code that will reproduce the error if you uncomment lines 30-35. As long as those lines stay commented out, it runs without error. Don't understand why. Thanks, Horst

…

On Wed, Jan 8, 2025 at 3:14 PM Kelly Abbott ***@***.***> wrote: Hi, @hherb <https://github.com/hherb> . @assafelovic <https://github.com/assafelovic> brought me into the loop because logging is an area I have recently worked on improving. I have stabilized logging more recently, but I haven't experimented with custom logging. I'd love to know what you're trying to get. So let's start there if you you don't mind. Here's what I can tell from the code you supplied: The "Something went wrong!" error you are seeing is likely because: - The custom handler isn't properly integrated with the frontend websocket system - The backend isn't receiving or processing the websocket messages correctly - There's a version mismatch between their frontend dependencies and what's expected Alternatively, if you package up a branch and I am happy to see if I can diagnose locally. — Reply to this email directly, view it on GitHub <#1052 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZYZTWOJHFBFJHY3HGZOKT2JSUJNAVCNFSM6AAAAABULFJUZSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZWG42DEMJUG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kga245]

I'll have a go at this. You save lives.