Export from sqlite to csv using shell script

I'm making a shell script to export a sqlite query to a csv file, just like this:

 #!/bin/bash
./bin/sqlite3 ./sys/xserve_sqlite.db ".headers on"
./bin/sqlite3 ./sys/xserve_sqlite.db ".mode csv"
./bin/sqlite3 ./sys/xserve_sqlite.db ".output out.csv"
./bin/sqlite3 ./sys/xserve_sqlite.db "select * from eS1100_sensor_results;"
./bin/sqlite3 ./sys/xserve_sqlite.db ".exit"

When executing the script, the output apears on the screen, instead of being saved to "out.csv". It's working doing the same method with the command line, but I don't know why the shell script fails to export data to the file.

What am I doing wrong?

Instead of the dot commands, you could use sqlite3 command options:

sqlite3 -header -csv my_db.db "select * from my_table;" > out.csv

This makes it a one-liner.

Also, you can run a sql script file:

sqlite3 -header -csv my_db.db < my_script.sql > out.csv

Use sqlite3 -help to see the list of available options.

sqlite3

You have a separate call to sqlite3 for each line; by the time your select runs, your .out out.csv has been forgotten.

Try:

#!/bin/bash
./bin/sqlite3 ./sys/xserve_sqlite.db <<!
.headers on
.mode csv
.output out.csv
select * from eS1100_sensor_results;
!

instead.

sh/bash methods

You can either call your script with a redirection:

$ your_script >out.csv

or you can insert the following as a first line in your script:

exec >out.csv

The former method allows you to specify different filenames, while the latter outputs to a specific filename. In both cases the line .output out.csv can be ignored.

I recently created a shell script that will be able to take the tables from a db file and convert them into csv files.

https://github.com/darrentu/convert-db-to-csv

Feel free to ask me any questions on my script :)

Although the question is about shell script, I think it will help few of those who are just bothered about transferring the data from the sqlite3 database to a csv file.

I found a very convinient way to do it with the firefox browser using SQLite Manager extension.

Simply connect to your sqlite database file in firefox ( SQlite manager -> connect database ) and then Table -> Export table. You will be served with some more options that you can just click and try....

In the end you get a csv file with the table u have chosen to be exported.

Using command line for Linux:

user@dell-Admin: sqlite3 #activate your sqlite database first
sqlite> .tables #search for tables if any available if already created one.
sqlite> .schema #if you want to check the schema of the table.

# once you find your table(s), then just do the following:

sqlite> .headers on   #export along with headers (column names)
sqlite> .mode csv     #file type is csv
sqlite> .output example.csv   #you want to provide file name to export
sqlite> SELECT * from events;    #If entire table is needed or select only required
sqlite> .quit    #finally quit the sqlite3

> Now search in your system for example.csv file and you will get it.

In one line is

sqlite3 -header -csv ./sys/xserve_sqlite.db "select * from eS1100_sensor_results;" >./out.csv

A synthesis of the answers till now:

function sqlite2csv-table() {
    local db="${1}" table="${2}" output="${3}"
    if test -z "$output" ; then
        output="${db:r}_hi${table}.csv"
    fi
    [[ "$output" =~ '.csv$' ]] || output+='.csv'

    echo "$0: outputting table '$table' to '$output'"
    sqlite3 -header -csv "$db" "select * from ${table};" > "$output" || return $?
}
function sqlite2csv() {
    local db="${1}" o="${2}"

    tables=($(sqlite3 $db ".tables")) 
    local t
    for table in $tables[@] ; do
        sqlite2csv-table "$db" "$table" "${o}_${table}.csv"
    done
}

Usage:

sqlite2csv some.db [/path/to/output]