Python programmers generally adhere to coding conventions (particularly PEP8). But you might rightfully ask yourself, “why should I care about coding conventions”?
Coding conventions matter because they improve readability. They improve readability between developers and they improve readability across time (i.e. the same developer editing code that he wrote earlier). When using reasonable coding conventions, it is generally easier to read and understand code.
While any particular coding convention might not matter too much, code consistency does matter (i.e. consistently following a predefined set of conventions does matter).
Generally, this implies you are better off adhering to the established coding conventions of a given language. For example, if your company is hiring Python developers, then the new developer is likely already familiar with PEP8. If you use a different standard than this, then he/she must abandon the normal PEP8 conventions and adopt yours (or convince you and your company to switch to PEP8).
Similarly, numerous Python tools are built around PEP8; consequently, if you adhere to PEP8, then you will be better able to leverage these tools.
Finally, if you are working on an open-source project with random contributors around the globe, it is much easier to use the coding conventions established by the language. In other words, PEP8 will be the default assumed context.
So to make a long story, short…in Python, you should generally follow PEP8 (with potentially minor exceptions). Or worded slightly differently, your code should be PEP8’ish i.e. generally follow PEP8, but don’t be too overly pedantic about it while still maintaining consistent coding standards.
As with all things in life, there is a context to the above statements and there could be possible exceptions to it. For example, you have an existing code-base that adheres to certain coding conventions, but that is not using PEP8 (and that would be difficult/costly to convert).
So, if you agree that you should generally follow PEP8…how do you achieve that desired end state? And how do you achieve it with a minimum of repetitive, boring, mundane work?
You pull out your handy tool box and you grab a couple of useful tools…enter Pylama and Black.
Pylama
Pylama is a Python library that wraps a set of other Python tools. You can see the list of tools that Pylama wraps here.
Two tools in Pylama that I regularly use are pycodestyle (previously known as the ‘pep8’ library) and pyflakes. pycodestyle reports on style issues in your program; pyflakes actually checks your program for certain errors.
So how do you use Pylama?
The first step is to define a setup.cfg file; this could also be done using a pylama.ini file. This setup.cfg file specifies a configuration for Pylama to use. Note, setup.cfg can also be used with other tools so here I am focussing only on its use with Pylama.
For example, here is the Pylama sections of the Netmiko setup.cfg file.
[pylama] linters = mccabe,pep8,pyflakes ignore = D203,C901 skip = tests/*,build/*,.tox/*,netmiko/_textfsm/*,examples/use_cases/*
[pylama:pep8]
max_line_length = 100
The ‘linters’ variable indicates which linters are enabled. Here I specify ‘mccabe’, ‘pep8’, and ‘pyflakes’. You can use ‘pylama –help’ to see which linters are available.
Note, the work required for linting adherence is greatly reduced by using ‘black’ which we will discuss later in this article.
The above setup.cfg also specifies certain rules that will be ignored. In particular, rules D203 and C901. D203 is a rule that concerns docstrings and is actually contradictory with a separate rule (D211); consequently, it is disabled. C901 is a flake8 rule for function complexity which I don’t find worthwhile.
Note, ‘pylama’ (by default) will recurse all of the Python files from the current directory downward (i.e. down the directory tree). The ‘skip’ section tells Pylama which files and directories to ignore.
One last item, you can see I have explicitly indicated a Python maximum line length of 100 characters. PEP8 states a maximum line length of 79 characters. In practice, the Python community seems to have largely abandoned this 79 character limit (somewhere between 88 and 100 characters is common).
I find 100 characters to be a good compromise between maintaining code readability and having sufficient space to get things done.
Pylama examples
# Will look for a setup.cfg in the current directory. # Will traverse the current directory downward looking for Python files $ pylama . # Explicitly reference the setup.cfg file and still traverse downward looking for Python files $ pylama -o ./setup.cfg . # Explicit setup.cfg reference and check only one file $ pylama -o ./setup.cfg j2_loader.py
Here is some example output from executing Pylama:
$ pylama -o ./setup.cfg j2_loader.py j2_loader.py:4:1: W0611 'pprint.pprint' imported but unused [pyflakes] j2_loader.py:10:30: W291 trailing whitespace [pep8] j2_loader.py:11:27: W291 trailing whitespace [pep8] j2_loader.py:12:36: W291 trailing whitespace [pep8] j2_loader.py:16:1: E0602 undefined name 'foo' [pyflakes]
We can see from this output that we have some useful information. The most important item being an error in our program. This error is:
j2_loader.py:16:1: E0602 undefined name 'foo' [pyflakes]
The error is on line 16 (as indicated by the first number after the colon). The message also indicates the issue. Here pyflakes is telling us that we are trying to use a variable that hasn’t previously been assigned.
The importance of this error checking is that it probably provides you with a quicker feedback loop on your debugging. And generally the quicker your feedback cycle; the better off you are.
Note, you could potentially execute this lint checking directly in your editor or IDE (for example, PyCharm, Vim, et cetera). I chose not to do that here as I didn’t want to get into environment specific practices. In other words, if you aren’t using my environment, then my advice is less relevant to you.
Our Pylama check also tells us some less useful things like:
j2_loader.py:4:1: W0611 'pprint.pprint' imported but unused [pyflakes] j2_loader.py:10:30: W291 trailing whitespace [pep8] j2_loader.py:11:27: W291 trailing whitespace [pep8] j2_loader.py:12:36: W291 trailing whitespace [pep8]
We are importing pprint and not using it; consequently, we should probably get rid of that. But do you really need to get rid of the trailing whitespace on these three lines. Note, here it is just three lines, but in a larger program it could be hundreds of lines with minor style issues…which leads us to our second useful tool Black.
Black
“Any customer can have a car painted any color that he wants so long as it is black.” — Henry Ford
Black is a Python library that automatically makes code formatting choices for you. It automatically modifies your code to adhere to the Black stylistic conventions.
Note, you should be using Black in conjunction with version control software (for example, Git). Consequently, you first have all your code checked into your version control system, then you execute ‘black’ and make the stylistic changes on your code base. You can then verify these changes are all appropriate for your environment.
Some advantages of using Black are:
1. The elimination/reduction of pointless squabbling between contributors over style conventions.
2. the significant reduction in the manual work required to get the code into compliance with your style conventions.
You can run black as follows:
$ black --check . would reformat j2_loader.py All done! 1 file would be reformatted.
This will operate Black in check mode i.e. Black will tell us which files will change (but doesn’t actually change them). Here Black is going to reformat the j2_loader.py file. Once again Black will operate on all of the Python files in the current directory downward.
Actually executing Black, without the –check option, will cause Black to reformatted the files per Black’s rules:
$ black . reformatted j2_loader.py All done! 1 file reformatted.
If we now re-run Pylama, we can see that some of our linting issues have been automatically fixed:
$ pylama . j2_loader.py:4:1: W0611 'pprint.pprint' imported but unused [pyflakes] j2_loader.py:16:1: E0602 undefined name 'foo' [pyflakes]
Black won’t remove our unused import nor will Black fix our syntax error, but it has corrected the whitespace style issues.
Note, Black requires Python 3.6 or greater in order to execute. The Python code that Black is checking can be Python2 code, but Black itself requires a Python3 environment to run.