What's the fastest way to copy a collection within the same database?

Mongodb

Mongodb Problem Overview


I want to copy a collection within the same database and give it a different name - basically take a snapshot.

What's the best way to do this? Is there a command, or do I have to copy each record in turn?

I'm aware of the cloneCollection command, but it seems to be for copying to another server only.

I'm also aware of mongoimport and mongoexport, but as I'm doing this via PHP I'd prefer not to make calls out to the shell.

Mongodb Solutions


Solution 1 - Mongodb

> db.myoriginal.aggregate([{$out: "mycopy"}])

It is a lot faster than doing many inserts in a forEach loop.

Solution 2 - Mongodb

You have a few options, but the fastest is:

mongodump -d db -c sourcecollection 
mongorestore -d db -c targetcollection --dir=dump/<db>/<sourcecollection.bson>

or

mongoexport -d db -c sourcecollection | mongoimport -d db -c targetcollection --drop

or in php:

`mongoexport -d db -c sourcecollection | mongoimport -d db -c targetcollection --drop`;

after that you have

mongo db < script.js

where, as shown in the mongo docs, script.js contains something like:

db.myoriginal.find().forEach( function(x){db.mycopy.insert(x)} );

The slowest (by an order of magnitude or more) way to copy a collection will be to use the native php driver - simply because of moving information around. But you could issue the above mongo query if you absolutely want to avoid cli calls using the db execute function.

Solution 3 - Mongodb

For versions less than 3.0, you can use copyTo() for experimental uses (never use on Production, as it blocks write operations):

db.source.copyTo("target"); 

& if "target" doesn't exist, it will be created

Solution 4 - Mongodb

First option: using mongodump

  1. Get a dump from source_collection

    $ mongodump -d db -c source_collection 
    
  2. Restore to target_collection collection from the dump

    $ mongorestore -d db -c target_collection dir=dump/db_name/source_collection.bson
    

Second option: $out aggregation step

  1. Run an aggregate command on source_collection

    > db.source_collection.aggregate([
        {$match: {emailAddress: "[email protected]"}},
        {$out: "target_collection"}
      ])
    

Third option (slowest): iterate and copy all documents

  1. Run a loop through all documents of source_collection, inserting them into target_collection

    > db.source_collection.find().forEach((doc) => {
        db.target_collection.insert(doc);
      })
      && print("Copy completed!");
    

Solution 5 - Mongodb

In addition to AD7six 1st solution, if you use mongoexport / import be sure about your collection data types and mongo configuration, as explained here: http://docs.mongodb.org/manual/reference/mongodb-extended-json/

Solution 6 - Mongodb

This is my implementation in python (pymongo):

def copy_collection(client, from_db, from_coll, to_db=None, to_coll=None):
    to_db = from_db if to_db is None else to_db
    to_coll = from_coll if to_coll is None else to_coll
    assert (to_db != from_db or to_coll != from_coll), "Copy Error: Source and destination can't be same!"
    documents = client[from_db][from_coll].find()
    client[to_db][to_coll].insert_many([d for d in documents])

Solution 7 - Mongodb

I managed to do that with this query :-

Suppose you have collection name data1 and you want to make new collection data2 with same indexes or want to delete or add some index you can do this by using this query.

data1 have indexes | FirstName | MiddleName | LastName | Age | Occupation

and you want to make new collection data2 | FirstName | MiddleName | LastName | Age | Occupation | Date

Query It took very less time in my case (speed depend on amount of data)

var cursor = db.data1.find();
var data = [];
while(cursor.hasNest()){
	var a = cursor.next();
	data.push({
		"FirstName" : a.FirstName,
		"MiddleName" : a.MiddleName,
		"LastName" : a.LastName,
		"Age" : a.Age,
		"Occupation" : a.Occupation,
		"Date" : new Date()
	});
}

db.data2.insertMany(data)

----- UPDATE -----

As this will build in memory array so for large set of data memory consumption will be high, so to solve this problem what one can do is define a flag and after some record let say 1000 you can insert the current data in collection and empty that array.

Solution 8 - Mongodb

The fastest way is db.collection.copyTo().

Note that it is deprecated since version 3.0.

Solution 9 - Mongodb

You can use the copyDatabase function in the mongo shell:

http://docs.mongodb.org/manual/tutorial/copy-databases-between-instances/

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionTimView Question on Stackoverflow
Solution 1 - MongodbyoooshiView Answer on Stackoverflow
Solution 2 - MongodbAD7sixView Answer on Stackoverflow
Solution 3 - MongodbAbdelHadyView Answer on Stackoverflow
Solution 4 - MongodbIsura AmarasingheView Answer on Stackoverflow
Solution 5 - MongodbkijView Answer on Stackoverflow
Solution 6 - MongodbSagar GuptaView Answer on Stackoverflow
Solution 7 - MongodbSubhash RawatView Answer on Stackoverflow
Solution 8 - MongodbchrisView Answer on Stackoverflow
Solution 9 - MongodbJp LorandiView Answer on Stackoverflow