Node.js Proxies with custom Http (a) agents and Connect.js Middleware

Question

I put together a proxy server in Node that needs a https request tunnel over tls, and it all works. Using the following two packages, it was very easy to configure: proxy , https-proxy-agent . My problem is that I am trying to capture HAR files with connect as an intermediate layer and I get the following error:

_http_outgoing.js:357 throw new Error('Can\'t set headers after they are sent.'); ^ Error: Can't set headers after they are sent. at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:357:11) at ServerResponse.writeHead (_http_server.js:180:21) at ClientRequest.<anonymous> (/.../node_modules/proxy/proxy.js:233:11) at emitOne (events.js:96:13) at ClientRequest.emit (events.js:188:7) at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:473:21) at HTTPParser.parserOnHeadersComplete (_http_common.js:99:23) at Socket.socketOnData (_http_client.js:362:20) at emitOne (events.js:96:13) at Socket.emit (events.js:188:7) 

It is as simple as the following, and it only happens when I use connect and proxying through my local browser (this proxy is actually used with BrowserStackLocal ). When I go through a proxy server from something other than my browser with a local machine, it doesn't seem to even know if middleware exists.

Basically, I just need to get connected to work in this scenario, and I'm not sure that I need to pause and resume something, or that ... any ideas will be highly appreciated. Base code below:

 const path = require('path'); const http = require('http'); const proxy = require('proxy'); const Agent = require('https-proxy-agent'); const connect = require('connect'); const har = require('./har'); const middleware = connect(); middleware.use(har({ harOutputDir: path.resolve(process.cwd(), 'har/') })); const server = proxy(http.createServer(middleware)); server.agent = new Agent('http://localhost:8081'); server.listen(8081) 

Thanks!

EDIT: just a note: hard middleware doesn't change the headers at all.

Answer 1

proxy not supported for some time. The assembly fails, the last commit fails the tests. The source of the stack trace from here on line 233 is the library code with the error.

Writing a similar proxy file is trivial. The following code illustrates how to create it.

 const http = require('http'); const urlP = require('url'); const proxiedServer = 'http://localhost:8888'; // Proxy server http.createServer((req, res) => { console.log(`Proxy: Got ${req.url}`); const _r = http.request( Object.assign( {}, urlP.parse(proxiedServer), { method: req.method, path: req.url } ), _res => { res.writeHead(_res.statusCode, _res.headers); _res.pipe(res); } ); req.pipe(_r); }).listen(3124, () => { console.log("Listening on 3124"); }); // Regular server. Could be Express http.createServer((req, res) => { console.log('Proxied server: ', req.url); let b = ''; req.on('data', c => { b += c; }); req.on('end', () => { console.log('Proxied server: ', b); }); res.writeHead(200); res.end('ok'); }).listen(8888, () => { console.log('proxied server listening on 8888'); }); 

Your code using your own proxy will look like this:

 const urlP = require('url'); const path = require('path'); const http = require('http'); const connect = require('connect'); const har = require('./har'); const proxiedServer = 'http://localhost:8888'; // Proxy server http.createServer((req, res) => { console.log(`Proxy: Got ${req.url}`); const _r = http.request( Object.assign( {}, urlP.parse(proxiedServer), { method: req.method, path: req.url } ), _res => { res.writeHead(_res.statusCode, _res.headers); _res.pipe(res); } ); req.pipe(_r); }).listen(3124, () => { console.log("Listening on 3124"); }); const middleware = connect(); middleware.use(har({ harOutputDir: path.resolve(process.cwd(), 'har/') })); middleware.use((req, res) => { console.log('Proxied server: ', req.url); let b = ''; req.on('data', c => { b += c; }); req.on('end', () => { console.log('Proxied server: ', b); }); res.writeHead(200); res.end('ok'); }); http.createServer(middleware).listen(8888, () => { console.log('proxied server listening on 8888'); });