Writing a Custom Linter Plugin - Part 3

Code Quality

Now that we have seen how easy it is to write plugins , I would like to share a few practices that have helped me and my team use linting effectively.

In the last post I discussed how to write a simple linter plugin. Many linter plugins that you end up writing will essentially be that simple. Once you understand how the AST of the code looks, writing more complicated checkers will become easier. A good place to explore examples is the Official Pylint Repository on Github itself. There are a lot of interesting implementations right there.

Using Pylint in a team

The way I showed previously will only work for simple scripts or a small project. But soon, the codebase will become large and many people will get involved. More rules will be added and the number of checkers will grow. How do we use Pylint in such cases? The major problem is, keeping track of all the custom configurations and also sharing them. The solution is to use a Project level Pylint Configuration File

pylintrc

The default Pylint configuration file can be generated by running the below command at the project root folder:

python3 -m pylint --generate-rcfile > .pylintrc

The name of the file is important. Pylint will automatically detect and use this file for linting within the project. This will happen within your IDE too, if the IDE supports Pylint as an extension. Pylint prioritises this file over the existing default configuration. If you open this file, you will find a variety of configuration options like number of lines of code in a file to naming styles, logging formats and whitelists. Spend some time on this and tweak the parameters and see how the linter behaviour changes. A significant amount of time can be spend on this to get the right linter behaviour you want. I will focus on two important actions that this file enables - Loading custom plugins and whitelists.

Loading custom Plugins

This is very easy that if you had opened the file, you would have figured it out already! In the [MASTER] section, within the first few lines, you would have seen this parameter load-plugins. This is the same parameter to which we had previously passed our test checker class! All you have to do is add your various plugin file names as comma-separated values here. When pylint runs, it will automatically detect these files and load the checker definitions. You have to ensure that the directory is in the PYTHONPATH and is importable if it is a sub directory(add an __init__.py file). This way you can maintain modularity, if you end up writing too many checkers.

Whitelists

This is an interesting and very useful feature and helps tackle one very common problem - the huge size of Pylint reports in large projects. While Pylint is great it still has limitations. One of them being its inability to recognize methods and variables in compiled libraries such as OpenCV or dynamically generated functionalities. A project that heavily uses OpenCV, for example, will now have an “error” in every line that uses an OpenCV method. A similar behaviour can be observed with certain SQLAlchemy Session functions and even namedtuples. And this errors create unneccessary noise in the final report. There are two ways this can be tackled.

1. Whitelisting the entire package

This is the simple solution and it can be achieved by adding the package name to the extension-pkg-whitelist parameter. This is usually the very first configurable parameter in the rcfile. So to whitelist OpenCV, for example, you will add extension-pkg-whitelist=cv2.

2. Ignoring specific patterns and functions

While whitelisting works, you sometimes would need finer control over what is checked and what is not. Pylint allows you do this aswell. You can ignore:

  • Variable names
  • Function names
  • Module names
  • Class names
  • Specific functions in certain classes
  • Variable names
  • Even Entire files!
  • certain errors from being displayed (This can also be done at file level)

The pylintrc file offers a lot of configuration options. The more you play around with it, the more you learn about how to make the linter work for you. This project level configuration can then be shared across the team along with the custom checkers and can also be version controlled.

Final thoughts

I used Pylint as an example tool to show how to incorporate linting into your work environment. But depending on your preferences - both language and others - there are many linters out there. For Python alone, the list of available linters include flake8, Pylama, Pycodestyle and Bandit. The concepts remain the same but implementations might differ. While the example I discussed was easy, linters can be used to detect logical errors, bad implementations etc. Bandit, for example, analyses your code and points out a variety of security issues. You can find a lot of tools once you start exploring. Happy Linting!

Subscribe to the Blog!

Want to get notified about the latest updates?

Subscribe Via RSS  rss Subscribe Via Email