Skip to content

Commit

Permalink
fix(server): Drastically reduce memory usage in proxy (#3172)
Browse files Browse the repository at this point in the history
<!-- Describe the problem and your solution --> 

This does a couple of things:

- Logs when we have a response body > 50MB just for our knowledge in the
logs. Will be interesting to see if it comes up around our OOM issues.
- Switches to building an array of buffers rather than doing tons of
string conversions. Should drastically reduce GC churn in there,
especially depending on how often our data callback was happening.

Benchmarking locally against a 52MB json file:

Before the update:
Pre-request used heap: 179977928
Post-request used heap: 425877152
Memory from request(ish): 245899224

After the update:
Pre-request used heap: 186414032
Post-request used heap: 339980184
Memory from request(ish): 153566152

<!-- Issue ticket number and link (if applicable) -->

<!-- Testing instructions (skip if just adding/editing providers) -->

## How I tested it:

So, maybe this is janky, maybe not, but:

In `providers.ts` I added a fake provider pointing to `localhost:4000`:

providers.ts, line 19
```
if (providers) {
  providers['__local'] = {
    display_name: 'Local Dev',
    auth_mode: 'NONE',
    proxy: {
      base_url: 'http://localhost:4000'
    },
    docs: 'https://google.com'
  };
}
```

Then I ran an http server on port 4000 with a 52MB json file under
`biggie.json`, set up the integration and a connection, and made proxy
requests against the dev server with that:

```
curl 'http://localhost:3003/proxy/biggie.json' --header 'Authorization: Bearer 07d4a5e1-cf98-49a2-bf86-7481aa258cab' --header 'Connection-Id: 90b41599-37aa-4ec0-95ec-88c6b546a21e' --header 'Provider-Config-Key: __local'
```
  • Loading branch information
nalanj authored Dec 16, 2024
1 parent 465caae commit e557dda
Showing 1 changed file with 10 additions and 4 deletions.
14 changes: 10 additions & 4 deletions packages/server/lib/controllers/proxy.controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -286,13 +286,19 @@ class ProxyController {
return;
}

let responseData = '';
const responseData: Buffer[] = [];
let responseLen = 0;

responseStream.data.on('data', (chunk: Buffer) => {
responseData += chunk.toString();
responseData.push(chunk);
responseLen += chunk.length;
});

responseStream.data.on('end', async () => {
if (responseLen > 5_000_000) {
logger.info(`Response > 5MB: ${responseLen} bytes`);
}

if (responseStream.status === 204) {
res.status(204).end();
metrics.increment(metrics.Types.PROXY_SUCCESS);
Expand All @@ -301,14 +307,14 @@ class ProxyController {
}

if (!isJsonResponse) {
res.send(responseData);
res.send(Buffer.concat(responseData));
await logCtx.success();
metrics.increment(metrics.Types.PROXY_SUCCESS);
return;
}

try {
const parsedResponse = JSON.parse(responseData);
const parsedResponse = JSON.parse(Buffer.concat(responseData).toString());

res.json(parsedResponse);
metrics.increment(metrics.Types.PROXY_SUCCESS);
Expand Down

0 comments on commit e557dda

Please sign in to comment.