SpaCy OSError: Can't find model 'en'
NlpSpacyNlp Problem Overview
even though I downloaded the model it cannot load it
[jalal@goku entity-sentiment-analysis]$ which python
/scratch/sjn/anaconda/bin/python
[jalal@goku entity-sentiment-analysis]$ sudo python -m spacy download en
[sudo] password for jalal:
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB)
100% |████████████████████████████████| 37.4MB 9.4MB/s
Installing collected packages: en-core-web-sm
Running setup.py install for en-core-web-sm ... done
Successfully installed en-core-web-sm-2.0.0
Linking successful
/usr/lib/python2.7/site-packages/en_core_web_sm -->
/usr/lib64/python2.7/site-packages/spacy/data/en
You can now load the model via spacy.load('en')
import spacy
nlp = spacy.load('en')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-2-0fcabaab8c3d> in <module>()
1 import spacy
2
----> 3 nlp = spacy.load('en')
/scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/__init__.py in load(name, **overrides)
17 "to load. For example:\nnlp = spacy.load('{}')".format(depr_path),
18 'error')
---> 19 return util.load_model(name, **overrides)
20
21
/scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/util.py in load_model(name, **overrides)
118 elif hasattr(name, 'exists'): # Path or Path-like to model data
119 return load_model_from_path(name, **overrides)
--> 120 raise IOError("Can't find model '%s'" % name)
121
122
OSError: Can't find model 'en'
How should I fix this?
If I don't use sudo for downloading the en model, I get:
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB)
100% |████████████████████████████████| 37.4MB 9.6MB/s ta 0:00:011 62% |████████████████████ | 23.3MB 8.6MB/s eta 0:00:02
Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz in /scratch/sjn/anaconda/lib/python3.6/site-packages
You are using pip version 10.0.0, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Error: Couldn't link model to 'en'
Creating a symlink in spacy/data failed. Make sure you have the required
permissions and try re-running the command as admin, or use a
virtualenv. You can still import the model as a module and call its
load() method, or create the symlink manually.
/scratch/sjn/anaconda/lib/python3.6/site-packages/en_core_web_sm -->
/scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/data/en
Download successful but linking failed
Creating a shortcut link for 'en' didn't work (maybe you don't have
admin permissions?), but you can still load the model via its full
package name:
nlp = spacy.load('en_core_web_sm')
Nlp Solutions
Solution 1 - Nlp
> FINALLY CLEARED THE ERROR !!!
Best Way to Install now
pip install -U pip setuptools wheel
pip install -U spacy
python -m spacy download en_core_web_sm
> Always Open Anaconda Prompt / Command Prompt with Admin Rights to avoid Linking errors!!!
-
Tried multiple options including :
python -m spacy download enconda install -c conda-forge spacypython -m spacy download en_core_web_smpython -m spacy link en_core_web_sm en -
None worked since im using my Company's Network. Finally this Command Worked like a Charm :-)
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz --no-deps- Updated with Latest Link :
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz --no-deps
Thanks to the Updated Github Links :-)
Solution 2 - Nlp
By using sudo python ... you install the model for a different python interpreter than your local one. In fact, it says in your log that the spaCy model is installed to /usr/lib64/python2.7/site-packages/ instead of /scratch/sjn/anaconda/lib/python3.6/site-packages/.
Try running python -m spacy download en and it should install the model to the correct directory.
Solution 3 - Nlp
-
Install Spacy
$ python -m spacy download en
-
Install the model en_core_web_sm
$ python -m spacy download en_core_web_sm >>> import spacy >>> nlp = spacy.load("en_core_web_sm")
Solution 4 - Nlp
oh well. turns out even though my which python was showing anaconda python, when I was using python download it was linking it to python2.7 local on my machine. I fixed it using below command:
$ sudo /scratch/sjn/anaconda/bin/python -m spacy download en
Solution 5 - Nlp
I am using anaconda jupyter notebook and was getting same error. Ran below commands in anaconda prompt (run as administrator) and it resolved my issue:
(base) C:\WINDOWS\system32>conda install -c conda-forge spacy
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\yadav\Anaconda3
added / updated specs:
- spacy
The following packages will be downloaded:
package | build
---------------------------|-----------------
cymem-2.0.3 | py37h6538335_0 35 KB conda-forge
cython-blis-0.4.1 | py37hfa6e2cd_0 4.3 MB conda-forge
murmurhash-1.0.0 | py37h6538335_0 17 KB conda-forge
plac-0.9.6 | py_1 18 KB conda-forge
preshed-3.0.2 | py37h6538335_1 89 KB conda-forge
spacy-2.2.1 | py37he980bc4_0 7.4 MB conda-forge
srsly-0.2.0 | py37h6538335_0 189 KB conda-forge
thinc-7.1.1 | py37he980bc4_0 1.4 MB conda-forge
wasabi-0.4.0 | py_0 19 KB conda-forge
------------------------------------------------------------
Total: 13.4 MB
The following NEW packages will be INSTALLED:
cymem conda-forge/win-64::cymem-2.0.3-py37h6538335_0
cython-blis conda-forge/win-64::cython-blis-0.4.1-py37hfa6e2cd_0
murmurhash conda-forge/win-64::murmurhash-1.0.0-py37h6538335_0
plac conda-forge/noarch::plac-0.9.6-py_1
preshed conda-forge/win-64::preshed-3.0.2-py37h6538335_1
spacy conda-forge/win-64::spacy-2.2.1-py37he980bc4_0
srsly conda-forge/win-64::srsly-0.2.0-py37h6538335_0
thinc conda-forge/win-64::thinc-7.1.1-py37he980bc4_0
wasabi conda-forge/noarch::wasabi-0.4.0-py_0
Proceed ([y]/n)? Y
Downloading and Extracting Packages
cython-blis-0.4.1 | 4.3 MB | ############################################################################ | 100%
cymem-2.0.3 | 35 KB | ############################################################################ | 100%
srsly-0.2.0 | 189 KB | ############################################################################ | 100%
thinc-7.1.1 | 1.4 MB | ############################################################################ | 100%
plac-0.9.6 | 18 KB | ############################################################################ | 100%
spacy-2.2.1 | 7.4 MB | ############################################################################ | 100%
preshed-3.0.2 | 89 KB | ############################################################################ | 100%
wasabi-0.4.0 | 19 KB | ############################################################################ | 100%
murmurhash-1.0.0 | 17 KB | ############################################################################ | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(base) C:\WINDOWS\system32>python -m spacy download en
Collecting en_core_web_sm==2.2.0
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz (12.0MB)
|████████████████████████████████| 12.0MB 409kB/s
Requirement already satisfied: spacy>=2.2.0 in c:\users\yadav\anaconda3\lib\site-packages (from en_core_web_sm==2.2.0) (2.2.2)
Requirement already satisfied: numpy>=1.15.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (1.16.2)
Requirement already satisfied: thinc<7.4.0,>=7.3.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (7.3.1)
Requirement already satisfied: wasabi<1.1.0,>=0.3.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.4.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (2.21.0)
Requirement already satisfied: setuptools in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (40.8.0)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (1.1.3)
Requirement already satisfied: srsly<1.1.0,>=0.1.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.2.0)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (2.0.3)
Requirement already satisfied: importlib-metadata>=0.20; python_version < "3.8" in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.23)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (1.0.2)
Requirement already satisfied: blis<0.5.0,>=0.4.0 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (0.4.1)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\users\yadav\anaconda3\lib\site-packages (from spacy>=2.2.0->en_core_web_sm==2.2.0) (3.0.2)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in c:\users\yadav\anaconda3\lib\site-packages (from thinc<7.4.0,>=7.3.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (4.36.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (2019.3.9)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\yadav\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.0->en_core_web_sm==2.2.0) (3.0.4)
Requirement already satisfied: zipp>=0.5 in c:\users\yadav\anaconda3\lib\site-packages (from importlib-metadata>=0.20; python_version < "3.8"->spacy>=2.2.0->en_core_web_sm==2.2.0) (0.6.0)
Requirement already satisfied: more-itertools in c:\users\yadav\anaconda3\lib\site-packages (from zipp>=0.5->importlib-metadata>=0.20; python_version < "3.8"->spacy>=2.2.0->en_core_web_sm==2.2.0) (6.0.0)
Building wheels for collected packages: en-core-web-sm
Building wheel for en-core-web-sm (setup.py) ... done
Created wheel for en-core-web-sm: filename=en_core_web_sm-2.2.0-cp37-none-any.whl size=12019131 sha256=f716e80f029462a80e9fb79ef353c1ac8c0f81d3754778bb6fec520d640fcc87
Stored in directory: C:\Users\yadav\AppData\Local\Temp\pip-ephem-wheel-cache-bvy0x0eg\wheels\48\5c\1c\15f9d02afc8221a668d2172446dd8467b20cdb9aef80a172a4
Successfully built en-core-web-sm
Installing collected packages: en-core-web-sm
Found existing installation: en-core-web-sm 2.0.0
Uninstalling en-core-web-sm-2.0.0:
Successfully uninstalled en-core-web-sm-2.0.0
Successfully installed en-core-web-sm-2.2.0
✔ Download and installation successful
You can now load the model via spacy.load('en_core_web_sm')
symbolic link created for C:\Users\yadav\Anaconda3\lib\site-packages\spacy\data\en <<===>> C:\Users\yadav\Anaconda3\lib\site-packages\en_core_web_sm
✔ Linking successful
C:\Users\yadav\Anaconda3\lib\site-packages\en_core_web_sm -->
C:\Users\yadav\Anaconda3\lib\site-packages\spacy\data\en
You can now load the model via spacy.load('en')
(base) C:\WINDOWS\system32>
Then in jupyter notebook load it like below:
nlp = spacy.load('en',parse=True,tag=True, entity=True)
Solution 6 - Nlp
If you use other python version, you can run :
> sudo python3.6 -m spacy download en
With me, my version 3.6 I hope it can help your problem!
Solution 7 - Nlp
As you are using Anaconda, open Anaconda Prompt as an Admin and execute the following command
python -m spacy download en
To load Spacy 'en' in Jupyter Notebook use the following command
spacy.load('en')
Solution 8 - Nlp
- you need to download
en_core_web_sm - if you are using
anaconda, then run this command conda install -c conda-forge spacy-model-en_core_web_sm- and load it as
nlp= spacy.load('en_core_web_sm')
Solution 9 - Nlp
Bc i didnt find my error here (For everyone who uses jupyter Notebook, Alteryx, Company Network and had this error):
i tried to create a macro with python for topic detection but got the Error that there is not a module named "en_core_web_sm"
Install following Packages at the beginning with following code:
from ayx import Package Package.installPackages(['pandas','numpy', 'matplotlib', 'gensim', 'spacy', 'pyLDAvis', 'https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.1.0/en_core_web_sm-2.1.0.tar.gz'])
And if you are referencing to the module use:
import en_core_web_sm
nlp = en_core_web_sm.load()
worked for me perf fine :))
Solution 10 - Nlp
pip install https://github.com/explosion/spacy-
models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz
#fixes an permission error when attempting to create the symlinks on windows 10
python -m spacy link en_core_web_sm en_core_web_smc
from Jupyter notes
import spacy
nlp = spacy.load('en_core_web_sm')
Solution 11 - Nlp
Go to https://github.com/explosion/spacy-models
Download the model you want to load in SpaCy
Paste the downloaded file in SpaCy folder present inside the Anaconda folder
open cmd there. Type the following command and hit enter:
pip install en_core_web_md-1.2.0.tar.gz
The above command may vary depending upon the version of the file downloaded.
Voila! Error has gone :)
Solution 12 - Nlp
Since you are using python version 3.6, try using -
python3 -m spacy download en
instead of just python -m .....
Solution 13 - Nlp
If you have already downloaded spacy and the language model (E.g., en_core_web_sm or en_core_web_md), then you can follow these steps:
-
Open Anaconda prompt as admin
-
Then type : python -m spacy link [package name or path] [shortcut]
For E.g., python -m spacy link /Users/you/model en
This will create a symlink to the your language model. Now you can load the model using spacy.load("en") in your notebooks or scripts
Solution 14 - Nlp
A quick hack to fix: Install an available model (e.g. en_core_web_sm) and then make the symlink yourself. Copy the two paths spacy says that it can't link (probably due to the virtual environment running without admin elevation) and use e.g. mklink on Windows.
E.g. mklink /D C:\Users\USER\PROJECT\venv2\lib\site-packages\spacy\data\en C:\Users\USER\PROJECT\venv2\lib\site-packages\en_core_web_sm