Lab 3 Some Python Topics
The aim of this lab is to introduce some core Python (3) concepts and ideas. Whilst some of these may seem unconnected I use many of these techniques and ideas day to day and will generally form part of a larger python ecosystems and deployment.
Python Modules and Packages
Python allows us to put functions and modules into a script file to be used later. These are called modules and can be imported into our programs.
A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__.
A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement.
When you run a Python module with python module.py the code in the module will be executed, just as if you imported it, but with the name set to “main”.
A package is a collection of Python modules, a module is a single Python file, a package is a directory of Python modules containing an additional __init__.py file, to distinguish a package from a directory that just happens to contain a bunch of Python scripts. Packages can be nested to any depth, provided that the corresponding directories contain their own init.py file.
I have started to collate my code into a simple package called nccapy we can copy this using the following
git clone https://github.com:/NCCA/nccapy
If we try to import this package in the repl we get the following error.
import nccapy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'nccapy'
This is because it is not in the python search path by default We can add the path using the “PYTHONPATH” environment variable, or in code using the following
import sys
sys.path.append("/extrapath")
Unit testing in Python
As python is an interpreted language unit testing is slightly different than in C++, however the main processes are the same.
We have a number of testing frameworks we can use
- pytest (needs to be installed using pip)
- unittest (part of the standard python library)
- doctest (part of the standard python library)
There are also other 3rd party frameworks we can use.
A practical example Vec3
classDiagram
class Vec3 {
+ float x
+ float y
+ float z
+ Vec3( float x, float y, float z)
+ normalize()
+ length()
}
We are going to develop a simple Vec3 class using TDD
Project Setup
mkdir Vec3
cd Vec3
mkdir tests
touch __init__.py
touch Vec3.py
touch tests/__init__.py
touch tests/test_Vec3.py
we are going to develop the class as a python module https://docs.python.org/3/tutorial/modules.html
The tests will be part of the module, test_ in file names helps with auto-discovery of the tests.
tests/test_Vec3.py
import unittest
from Vec3 import Vec3
import math
class TestVec3(unittest.TestCase) :
def test_1(self) :
self.assert(True == False) # note failing test
python -m unittest
F
======================================================================
FAIL: test_1 (tests.test_Vec3.TestVec3)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jmacey/teaching/Code/DemoPythonCode/Testing/Vec3/tests/test_Vec3.py", line 8, in test_1
self.assertTrue(True == False)
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
Now Repeat
We have written enough code to ensure we have a failing test, lets modify to make it closer to a TDD approach
import unittest
from Vec3 import Vec3
import math
class TestVec3(unittest.TestCase) :
def test_default(self) :
a=Vec3()
self.assertAlmostEqual(a.x,0.0)
self.assertAlmostEqual(a.y,0.0)
self.assertAlmostEqual(a.z,0.0)
Doctests
Doctest is a different approach, we actually write the tests in the documentation. Personally I don’t like this and prefer the unittest approach.
Example
class Vec3 :
x : float
y : float
z : float
def __init__(self, x=0.0, y=0.0,z=0.0) :
"""Initialise the Vec3 class
Default Initialiser
>>> a=Vec3()
>>> a.x
0.0
>>> a.y
0.0
>>> a.z
0.0
Let's try a user defined values
>>> a=Vec3(0.5,0.2,8.0)
>>> a.x
0.5
>>> a.y
0.2
>>> a.z
8.0
This should throw an exception as we are contructing for a string
>>> a=Vec3('a','b','c') # doctest: +IGNORE_EXCEPTION_DETAIL
ValueError: this class only accepts numeric values
"""
try :
self.x = float(x)
self.y = float(y)
self.z = float(z)
except ValueError as e:
raise ValueError("this class only accepts numeric values") from e
if __name__ == "__main__":
import doctest
doctest.testmod()
Output
python -m doctest Vec3.py -v
Trying:
a=Vec3()
Expecting nothing
ok
Trying:
a.x
Expecting:
0.0
ok
Trying:
a.y
Expecting:
0.0
ok
Trying:
a.z
Expecting:
0.0
ok
Trying:
a=Vec3(0.5,0.2,8.0)
Expecting nothing
ok
Trying:
a.x
Expecting:
0.5
ok
Trying:
a.y
Expecting:
0.2
ok
Trying:
a.z
Expecting:
8.0
ok
2 items had no tests:
Vec3
Vec3.Vec3
1 items passed all tests:
8 tests in Vec3.Vec3.__init__
8 tests in 3 items.
8 passed and 0 failed.
Test passed.
function attributes
In python everything is an object, and as such these can have attributes and methods. Functions are also included in this “everything is an object” model.
#!/usr/bin/env python
# This function uses attributes
def static_func():
print(f"value a={static_func.a} b={static_func.b}")
static_func.a += 1
static_func.b += 5
#We need to initialise the first
static_func.a = 0
static_func.b = 0
for i in range(0, 10):
static_func()
In the previous example we need to set the function attributes first, we can bypass this by adding the code to the function as follows
#!/usr/bin/env python
# This function uses attributes
def static_func():
if not hasattr(static_func, "a"):
static_func.a = 0 # it doesn't exist yet, so initialize it
if not hasattr(static_func, "b"):
static_func.b = 0 # it doesn't exist yet, so initialize it
print(f"value a={static_func.a} b={static_func.b}")
static_func.a += 1
static_func.b += 5
for i in range(0, 10):
static_func()
For each extra value we need to add the hasattr code. This is quite a lot of work, to make this easier we can use a decorator
#!/usr/bin/env python
def static_vars(**kwargs):
def decorate(func):
for k in kwargs:
setattr(func, k, kwargs[k])
return func
return decorate
# This function uses attributes
@static_vars(a=0,b=0)
def static_func():
print(f"value a={static_func.a} b={static_func.b}")
static_func.a += 1
static_func.b += 5
for i in range(0, 10):
static_func()
Exercises
Using the examples above complete the Vec3 class to match that of the ngl::Vec3 class, full solution will be in the nccapy module soon.
References
https://book.pythontips.com/en/latest/decorators.html https://docs.python.org/3/tutorial/modules.html https://www.python.org/dev/peps/pep-3129/