If you are programming, you should use curl. It has a nice api and is available for most languages. Shelling out to the os to run wget is a kludge and shouldn&#39;t be done if you have an API interface!

There is some overlap in functionality. While [GNU wget](http://www.gnu.org/software/wget/) is a package for retrieving files using HTTP/FTP, [curl](http://curl.haxx.se/) transfers data with single URL. As noted in the link shared by [MarkusQ](https://stackoverflow.com/users/62970/markusq), wget can download recursively - see [this comparison article](http://daniel.haxx.se/docs/curl-vs-wget.html) for more details by the curl author.

`cURL` is intended for data transfer in both directions while `wget` is for non-interactive downloading file(s) from a particular source. There are [some][q1] overlaps in functionality, but they are [not meant][q2] to do exactly [the same things][q3].

It really depends on what you are trying to do; for simpler tasks like downloading files `wget` and `cURL` are comparable, but this really only scratches the surface of either tool.

[q1]: /q/9409214
[q2]: /q/6371719
[q3]: /q/12846933

They both offer endless options, most of which I&#39;ve never used. However, I tend to like wget more as it by default saves the output from the URL you give — perfect for downloading. Curl goes the other way, and displays the output directly to the terminal (by default).

I tried to use java.io.FileReader to read some text files and convert them into a string, but I found the result is wrongly encoded and not readable at all.

Here&#39;s my environment:

 - Windows 2003, OS encoding: CP1252
   
 - Java 5.0

My files are UTF-8 encoded or CP1252 encoded, and some of them (UTF-8 encoded files)  may contain Chinese (non-Latin) characters.

I use the following code to do my work:

       private static String readFileAsString(String filePath)
        throws java.io.IOException{
            StringBuffer fileData = new StringBuffer(1000);
            FileReader reader = new FileReader(filePath);
            //System.out.println(reader.getEncoding());
            BufferedReader reader = new BufferedReader(reader);
            char[] buf = new char[1024];
            int numRead=0;
            while((numRead=reader.read(buf)) != -1){
                String readData = String.valueOf(buf, 0, numRead);
                fileData.append(readData);
                buf = new char[1024];
            }
            reader.close();
            return fileData.toString();
        }

The above code doesn&#39;t work. I found the FileReader&#39;s encoding is CP1252 even if the text is UTF-8 encoded. But the JavaDoc of java.io.FileReader says that:

&gt; The constructors of this class assume
&gt; that the default character encoding
&gt; and the default byte-buffer size are
&gt; appropriate.

Does this mean that I am not required to set character encoding by myself if I am using FileReader? But I did get wrongly encoded data currently, what&#39;s the correct way to deal with my situtaion? Thanks.

Java FileReader encoding issue

I need to automatically generate a PDF file from an exisiting (X)HTML-document. The input files (reports) use a rather simple, table-based layout, so support for really fancy JavaScript/CSS stuff is probably not needed.

As I am used to working in Java, a solution that can easily be used in a java-project is preferable. It only needs to work on windows systems, though.

One way to do it that is feasable, but does not produce good quality output (at least out of the box) is using [CSS2XSLFO][1], and Apache FOP to create the PDF files. The problem I encountered was that while CSS-attributes are converted nicely, the table-layout is pretty messed up, with text flowing out of the table cell.

I also took a quick look at Jrex, a Java-API for using the Gecko rendering engine. 

Is there maybe a way to grab the rendered page from the internet explorer rendering engine and send it to a PDF-Printer tool automatically? I have no experience in OLE programming in windows, so I have no clue what&#39;s possible and what is not.

Do you have an idea?

  [1]: http://re.be/css2xslfo/index.xhtml

Converting HTML files to PDF

Are they the same or not? Can certain things be done with one but not the other? What are those? Or is it, at the end of the day, a matter of familiarity?

What is better, curl or wget?

<p>Are they the same or not? Can certain things be done with one but not the other? What are those? Or is it, at the end of the day, a matter of familiarity?</p>


Is there any way to find out if the current session user is running an Xserver (under Linux) ?

I&#39;ve started off with things like:

    ps -e | grep X 

but this doesn&#39;t work always

and one more thing I tried is checking the `$DISPLAY` variable

Are there any other ways to check this?

EDIT:

Some people suggested using the `$DISPLAY` variables but what if the user fiddles with this variable ? what if he tries to do something and changes this variable and then when I check it, it no longer reflects an accurate state of the system. 
Is there no specific way to do this that will always return a correct answer ?

I found that it can be done programmatically thus:

    #include &lt;X11/Xlib.h&gt; 
    int main()
        { exit(XOpenDisplay(NULL) ? 0 : 1);  } 
    
    $ gcc -o xprobe xprobe.c -L/usr/X11R6/lib -lX11 

But I am looking for a script way.

How to check if X server is running?

Is there a way to prevent a command from being added to the bash shell&#39;s command history?

I would like to be able to prevent certain &quot;dangerous&quot; commands from being added to the history, such as &quot;`rm -rf ~/some/dir`&quot;, so that it is not accessible to me by pressing the up-arrow to reach previous commands.  In this way, it would not be possible to accidentally repeat one of these commands.

The reason I ask is that I use the up arrow a lot to access previous commands in the shell history, and have often caught myself about to hit enter on what I **thought** was the correct command, only to realise that I was about to do something stupid/annoying/dangerous.  I don&#39;t like the idea of `rm`s etc floating around in my shell history, waiting for me to step on them!

(Note: I am aware that one can set up patterns in HISTIGNORE, but what would be nice is something one can apply on a per-command basis, which would become good a habit.  Unless there is a clever way to achieve this using HISTIGNORE that I have missed?)

Can you prevent a command from going into the bash shell command history?

Bash scripts are very useful and can save a lot of programming time. So how do you start a bash script in a C++ program? Also if you know how to make user become the super-user that would be nice also. Thanks!

How to run a bash script from C++ program

&gt; **Possible Duplicate:**  
&gt; [How can I run an external program from C and parse its output?](https://stackoverflow.com/questions/43116/how-can-i-run-an-external-program-from-c-and-parse-its-output)  

&lt;!-- End of automatically inserted text --&gt;

I want to run a command in linux and get the text returned of what it outputs, but I **do not** want this text printed to screen. Is there a more elegant way than making a temporary file?



C: Run a System Command and Get Output?

Has anyone an idea how many tcp-socket connections are possible on a modern standard Linux server?

(There is in general less traffic on each connection, but all the connections have to be up all the time.)

How many socket connections possible?

How do I make curl ignore the proxy?
Setting $NO_PROXY doesn&#39;t seem to work for me.


How do I make curl ignore the proxy?

I want to see the request headers made by `curl` when I am sending a request to the server. How can I check that?

How can I see the request headers made by curl when sending a request to the server?

How can I do a RAW POST in PHP using cURL?

Raw post as in without any encoding, and my data is stored in a string. The data should be formatted like this:

    ... usual HTTP header ...
    Content-Length: 1039
    Content-Type: text/plain

    89c5fdataasdhf kajshfd akjshfksa hfdkjsa falkjshfsa
    ajshd fkjsahfd lkjsahflksahfdlkashfhsadkjfsalhfd
    ajshdfhsafiahfiuwhflsf this is just data from a string
    more data kjahfdhsakjfhsalkjfdhalksfd

One option is to manually write the entire HTTP header being sent, but that seems less optimal.

Anyway, can I just pass options to curl_setopt() that say use POST, use text/plain, and send the raw data from a `$variable`?


RAW POST using cURL in PHP

So some guy at some other company thought it would be awesome if instead of using soap or xml-rpc or rest or any other reasonable communication protocol he just embedded all of his response as cookies in the header.

I need to pull these cookies out as hopefully an array from this curl response.  If I have to waste a bunch of my life writing a parser for this I will be very unhappy.

Does anyone know how this can simply be done, preferably without writing anything to a file?

I will be very grateful if anyone can help me out with this.

how to get the cookies from a php curl into a variable

I am looking for a simple way to import/copy images from remote server to a local folder using PHP.  I have no FTP access to the server, but all remote images can be accessed via HTTP (i.e. http://www.mydomain.com/myimage.jpg).

Example use: A user wishes to add an image to his profile.  The image already exists on the web and the user provides with a direct URL.  I do not wish to hotlink the image but to import and serve from my domain.

Copy Image from Remote Server Over HTTP

I need files to be downloaded to /tmp/cron_test/. My wget code is

    wget --random-wait -r -p -nd -e robots=off -A&quot;.pdf&quot; -U mozilla http://math.stanford.edu/undergrad/

So is there some parameter to specify the directory?


How to specify the download location with wget?

I am trying to use [Wget][1] to download a page, but I cannot get past the login screen.

How do I send the username/password using post data on the login page and then download the actual page as an authenticated user?

  [1]: http://en.wikipedia.org/wiki/Wget


How to get past the login page with Wget?

I&#39;m running a PHP script via cron using Wget, with the following command:

    wget -O - -q -t 1 http://www.example.com/cron/run

The script will take a maximum of 5-6 minutes to do its processing. Will WGet wait for it and give it all the time it needs, or will it time out?

Does WGET timeout?

How do I prevent *wget* from following redirects?

wget: don&#39;t follow redirects

I&#39;m looking for a way to pseudo-spider a website. The key is that I don&#39;t actually want the content, but rather a simple list of URIs. I can get reasonably close to this idea with [Wget](http://www.gnu.org/software/wget/) using the `--spider` option, but when piping that output through a `grep`, I can&#39;t seem to find the right magic to make it work:

    wget --spider --force-html -r -l1 http://somesite.com | grep &#39;Saving to:&#39;

The `grep` filter seems to have absolutely no affect on the `wget` output. Have I got something wrong or is there another tool I should try that&#39;s more geared towards providing this kind of limited result set?

**UPDATE**

So I just found out offline that, by default, `wget` writes to stderr. I missed that in the man pages (in fact, I still haven&#39;t found it if it&#39;s in there). Once I piped the return to stdout, I got closer to what I need:

    wget --spider --force-html -r -l1 http://somesite.com 2&gt;&amp;1 | grep &#39;Saving to:&#39;

I&#39;d still be interested in other/better means for doing this kind of thing, if any exist.

Content Type	Original Author	Original Content on Stackoverflow
Question	flybywire	View Question on Stackoverflow
Solution 1 - Linux	Byron Whitlock	View Answer on Stackoverflow
Solution 2 - Linux	David Kim	View Answer on Stackoverflow
Solution 3 - Linux	icedwater	View Answer on Stackoverflow
Solution 4 - Linux	Guðmundur H	View Answer on Stackoverflow

What is better, curl or wget?

Linux Problem Overview

Linux Solutions

Solution 1 - Linux

Solution 2 - Linux

Solution 3 - Linux

Solution 4 - Linux

Converting HTML files to PDF

Java FileReader encoding issue

Attributions