Read remote file with node.js (http.get)

node.js

node.js Problem Overview


Whats the best way to read a remote file? I want to get the whole file (not chunks).

I started with the following example

var get = http.get(options).on('response', function (response) {
    response.on('data', function (chunk) {
        console.log('BODY: ' + chunk);
    });
});

I want to parse the file as csv, however for this I need the whole file rather than chunked data.

node.js Solutions


Solution 1 - node.js

I'd use request for this:

request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png'))

Or if you don't need to save to a file first, and you just need to read the CSV into memory, you can do the following:

var request = require('request');
request.get('http://www.whatever.com/my.csv', function (error, response, body) {
    if (!error && response.statusCode == 200) {
        var csv = body;
        // Continue with your processing here.
    }
});

etc.

Solution 2 - node.js

You can do something like this, without using any external libraries.

const fs = require("fs");
const https = require("https");

const file = fs.createWriteStream("data.txt");

https.get("https://www.w3.org/TR/PNG/iso_8859-1.txt", response => {
  var stream = response.pipe(file);

  stream.on("finish", function() {
    console.log("done");
  });
});

Solution 3 - node.js

http.get(options).on('response', function (response) {
    var body = '';
    var i = 0;
    response.on('data', function (chunk) {
        i++;
        body += chunk;
        console.log('BODY Part: ' + i);
    });
    response.on('end', function () {

        console.log(body);
        console.log('Finished');
    });
});

Changes to this, which works. Any comments?

Solution 4 - node.js

function(url,callback){
	request(url).on('data',(data) => {
		try{
			var json = JSON.parse(data);	
		}
		catch(error){
			callback("");
		}
		callback(json);
	})
}

You can also use this. This is to async flow. The error comes when the response is not a JSON. Also in 404 status code .

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questionuser140291View Question on Stackoverflow
Solution 1 - node.js7zark7View Answer on Stackoverflow
Solution 2 - node.jsFreddyView Answer on Stackoverflow
Solution 3 - node.jsuser140291View Answer on Stackoverflow
Solution 4 - node.jsSarath Kumar RajendranView Answer on Stackoverflow