What's the fastest way to copy a collection within the same database?
MongodbMongodb Problem Overview
I want to copy a collection within the same database and give it a different name - basically take a snapshot.
What's the best way to do this? Is there a command, or do I have to copy each record in turn?
I'm aware of the cloneCollection
command, but it seems to be for
copying to another server only.
I'm also aware of mongoimport
and mongoexport
, but as I'm doing this via PHP I'd prefer not to make calls out to the shell.
Mongodb Solutions
Solution 1 - Mongodb
> db.myoriginal.aggregate([{$out: "mycopy"}])
It is a lot faster than doing many inserts in a forEach
loop.
Solution 2 - Mongodb
You have a few options, but the fastest is:
mongodump -d db -c sourcecollection
mongorestore -d db -c targetcollection --dir=dump/<db>/<sourcecollection.bson>
or
mongoexport -d db -c sourcecollection | mongoimport -d db -c targetcollection --drop
or in php:
`mongoexport -d db -c sourcecollection | mongoimport -d db -c targetcollection --drop`;
after that you have
mongo db < script.js
where, as shown in the mongo docs, script.js contains something like:
db.myoriginal.find().forEach( function(x){db.mycopy.insert(x)} );
The slowest (by an order of magnitude or more) way to copy a collection will be to use the native php driver - simply because of moving information around. But you could issue the above mongo query if you absolutely want to avoid cli calls using the db execute function.
Solution 3 - Mongodb
For versions less than 3.0, you can use copyTo() for experimental uses (never use on Production, as it blocks write operations):
db.source.copyTo("target");
& if "target"
doesn't exist, it will be created
Solution 4 - Mongodb
First option: using mongodump
-
Get a dump from
source_collection
$ mongodump -d db -c source_collection
-
Restore to
target_collection
collection from the dump$ mongorestore -d db -c target_collection dir=dump/db_name/source_collection.bson
Second option: $out
aggregation step
-
Run an aggregate command on
source_collection
> db.source_collection.aggregate([ {$match: {emailAddress: "[email protected]"}}, {$out: "target_collection"} ])
Third option (slowest): iterate and copy all documents
-
Run a loop through all documents of
source_collection
, inserting them intotarget_collection
> db.source_collection.find().forEach((doc) => { db.target_collection.insert(doc); }) && print("Copy completed!");
Solution 5 - Mongodb
In addition to AD7six 1st solution, if you use mongoexport / import be sure about your collection data types and mongo configuration, as explained here: http://docs.mongodb.org/manual/reference/mongodb-extended-json/
Solution 6 - Mongodb
This is my implementation in python (pymongo):
def copy_collection(client, from_db, from_coll, to_db=None, to_coll=None):
to_db = from_db if to_db is None else to_db
to_coll = from_coll if to_coll is None else to_coll
assert (to_db != from_db or to_coll != from_coll), "Copy Error: Source and destination can't be same!"
documents = client[from_db][from_coll].find()
client[to_db][to_coll].insert_many([d for d in documents])
Solution 7 - Mongodb
I managed to do that with this query :-
Suppose you have collection name data1 and you want to make new collection data2 with same indexes or want to delete or add some index you can do this by using this query.
data1 have indexes | FirstName | MiddleName | LastName | Age | Occupation
and you want to make new collection data2 | FirstName | MiddleName | LastName | Age | Occupation | Date
Query It took very less time in my case (speed depend on amount of data)
var cursor = db.data1.find();
var data = [];
while(cursor.hasNest()){
var a = cursor.next();
data.push({
"FirstName" : a.FirstName,
"MiddleName" : a.MiddleName,
"LastName" : a.LastName,
"Age" : a.Age,
"Occupation" : a.Occupation,
"Date" : new Date()
});
}
db.data2.insertMany(data)
----- UPDATE -----
As this will build in memory array so for large set of data memory consumption will be high, so to solve this problem what one can do is define a flag and after some record let say 1000 you can insert the current data in collection and empty that array.
Solution 8 - Mongodb
The fastest way is db.collection.copyTo().
Note that it is deprecated since version 3.0.
Solution 9 - Mongodb
You can use the copyDatabase function in the mongo shell:
http://docs.mongodb.org/manual/tutorial/copy-databases-between-instances/