MongoDB won't start after server crash

My Ubuntu computer had crashed, and when I restarted it MongoDB wasn't working. I tried the following commands, and got the following output:

$ mongo
Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91
exception: connect failed

$ service mongodb status
mongodb stop/waiting

$ service mongodb restart
stop: Unknown instance: 
start: Rejected send message, 1 matched rules; type="method_call",
       sender=":1.57" (uid=1000 pid=2227 comm="start mongodb ")
       interface="com.ubuntu.Upstart0_6.Job" member="Start" error name="(unset)"
       requested_reply="0"
       destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init")

$ tail /var/log/mongodb/mongodb.log
[initandlisten] exception in initAndListen: 12596 old lock file, terminating
dbexit: 
[initandlisten] shutdown: going to close listening sockets...
[initandlisten] shutdown: going to flush diaglog...
[initandlisten] shutdown: going to close sockets...
[initandlisten] shutdown: waiting for fs preallocator...
[initandlisten] shutdown: closing all files...
[initandlisten] closeAllFiles() finished
dbexit: really exiting now

(Output reformatted to match website layout.)

What happened? How can I fix it?

The log file is telling you that you have an "old lock file". MongoDB keeps a lock file while it's running. It creates this file when it is started, and deletes it when it's stopped. When the computer crashes (or MongoDB crashes, e.g. via kill), this file is not deleted, and thus the database does not start. The existence of this file indicates unclean shutdown of MongoDB.

Two things can be done:

1. If this is a development machine and you haven't been using your database (and neither have your programs), you can remove the file manually. For MongoDB 2.2.2 running on Ubuntu 12.10, it's in /var/lib/mongodb/mongod.lock. For other versions, the file could be in a different path or it could be named mongo.lock.

2. The safer route is to follow MongoDB's Durability and Repair guide. In summary, for a machine with the above configuration, you should execute the following commands:

    sudo -u mongodb mongod --repair --dbpath /var/lib/mongodb/
    sudo service mongod start
   

all I had to do was run: sudo mongod --repair

then:

sudo mongod

Based on my experience, I usually delete the "mongod.lock" file that is inside the database folder - In my case:

*I browse to where the database is installed on my ubuntu i.e. "data" folder.(cd data); list the files (ls) *Then, I will remove the "mongod.lock" file that was automatically created when the database crashed, by issuing "rm mongod.lock" file.

After which I will either issue "./mongod" to start the mongo deamon or mongo to start the mongo shell. And everything will be fine.

If you did not use monitoring tools like Bluepill or Monit etc you will have to face this issue because after server crash due to some reason mongo didnot start its daemon automatically then you have to make it work manually, like sudo service mongod restart I figured this issue but it needs some more tasks to be done, please make sure your dbpath at /etc/mongod.conf before starting your mongo daemon.

For me it was

storage:
  dbPath: /var/lib/mongodb

When i enter mongod command it shows me MongoDB starting : pid=10795 port=27017 dbpath=/data/db 64-bit host=xyz.com make sure your dbpath is as same as mentioned in /etc/mongod.conf

To do this you can type sudo mongod --dbpath /var/lib/mongodb and then use mongod command to start your mongo process at your desired dbpath.

FYI: start your mongo process with mongod command

Check to see if you have enough free space on your server. If there is no room left mongodb won't start.

This is probably not the best solution, but if you're desperate you can try this. It seemed that only the journal was a problem for me, so I took these steps:

1. Create a new data directory. Possibly /var/lib/mongodb2
2. Update your mongod.conf to point to the new data dir.
3. Start mongoDB.
4. If it starts successfully, then you can shut down mongo again and proceed, otherwise you can stop reading here.
5. Locate your previous data dir and copy the files for your database(s) to your new data directory (example, admin.0 admin.1 admin.ns, etc)
6. Start mongoDB again (still using the new data directory)

After completing these steps (took less than 5 min), I was up and running and all data appeared to be ok.

Thank's guys. We also faced an issue where MongoDB was restarting over and over again an it was complaining about old lock file. I stopped MongoDB from Windows service list and then I deleted the mongod.lock file. After that I was able to start MongoDB service correctly and it worked fine.

Removing .lock file from mongo data directory dbpath works for me.

e.g sudo sudo rm {data-directory}/mongod.lock