pylint 1.4 reports E1101(no-member) on all C extensions
PythonPylintPython ExtensionsPython Problem Overview
We've been long-time fans of pylint
. Its static analysis has become a critical part of all our python projects and has saved tons of time chasing obscure bugs. But after upgrading from 1.3 -> 1.4, almost all compiled c extensions result in E1101(no-member) errors.
Projects that previously run perfectly clean through pylint
1.3 now complain about almost every C extension member with E1101. We've been forced to disable E1101 errors, but this materially detracts from the usefulness of pylint
.
For example, this perfectly valid use of the lxml
package
r"""valid.py: demonstrate pylint 1.4 error"""
from lxml import etree
print etree.Element('mydoc')
Run this through pylint
, and it reports:
$ pylint -rn valid.py
No config file found, using default configuration
************* Module valid
E: 3, 6: Module 'lxml.etree' has no 'Element' member (no-member)
But it is perfectly valid:
$ python valid.py
<Element mydoc at 7fddf67b1ba8>
Here's where it gets really weird. A very small handful of C extensions seem to work just fine through pylint
, e.g.:
r"""valid2.py: this one works fine"""
import sqlite3
print sqlite3.version
$ pylint -rn valid2.py
No config file found, using default configuration
My question is, has anyone else witnessed this? And if so, would you be willing to share your workaround/solution?
We've experimented with trying to create plugins to suppress these warnings
(http://docs.pylint.org/plugins.html#enter-plugin), but we're having difficulty making heads or tails of the docs -- and the astroid
base class is uber-complex and has defied our attempts to grok it.
For real bonus points (and our eternal gratitude) we'd love to understand what changed in pylint
. We'd be happy to fix the code (or at least publish a best practice document for C extension authors) that would satisfy pylint
.
Platform details
$ pylint --version
No config file found, using default configuration
pylint 1.4.0,
astroid 1.3.2, common 0.63.2
Python 2.7.5 (default, Jul 1 2013, 18:09:11)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]
Python Solutions
Solution 1 - Python
Shortly after posting my question, I found the answer. The change was in fact done on purpose as a security measure. Pylint imports modules to effectively identify valid methods and attributes. It was decided that importing c extensions that are not part of the python stdlib is a security risk and could introduce malicious code.
This was done in the release of Astroid 1.3.1 https://mail.python.org/pipermail/code-quality/2014-November/000394.html
> Only C extensions from trusted sources (the standard library) are > loaded into the examining Python process to build an AST from the live > module.
There are four solutions if you want to use pylint on projects that import non-stdlib c extensions.
-
Disable safety using the
--unsafe-load-any-extension=y
command line option. This feature is undocumented and classified as a hidden option (https://mail.python.org/pipermail/code-quality/2014-November/000439.html). -
Disable safety using the
pylint.rc
settingunsafe-load-any-extensions=yes
. This is recommended over option 1 and includes full documentation in the default pylint.rc file (created with--generate-rcfile
). -
Specifically list packages or modules names that you trust to be loaded by pylint in the
pylint.rc
file using theextension-pkg-whitelist=
option. -
Create a plugin to manipulate the AST (I have no idea how to effect this -- but it's regularly discussed on on the pylint mailing list).
We opted for Option 3. We added the following line to our project pylint.rc
file:
extension-pkg-whitelist=lxml
Solution 2 - Python
@user590028, thanks a lot for your answer! I just ran into this same problem with the libraries win32api, win32evtlog, win32file, win32gui, and win32process, and your solution worked.
I used another method I think is worth posting here, which is to call pylint and pass the whitelisted packages as a parameter:
pylint --extension-pkg-whitelist=win32api,win32evtlog,win32file,win32gui,win32process myfile.py
Solution 3 - Python
For those of you using VS Code, it's a bit tricky to find where to put the command as I couldn't find my executable.
In VS Code;
- click on File > Preferences > Settings.
- Scroll down to "Python Configurations" in the left window
- scroll down to "Python Linting: Mypy Args" in the right window
- click on "Edit in settings.json" link
- edit the json to include:
"--extension-pkg-whitelist=
"
I had to do all this because PyLint isn't executable from my Windows command line...
Solution 4 - Python
If you're using VS Code for Mac, this is what you need to do in order to edit the settings.json file:
- Click on Code (i.e. the Visual Studio Code tab which is on the left of the 'File' tab) -> Preferences - > Settings
- Scroll down to Extensions and click on Python in the list.
- Click on any of the
Edit in settings.json
links. This opens upsettings.json
for editing. - Add the line
"python.linting.pylintArgs": ["----extension-pkg-whitelist=1xml"]
.