Testing in Python: Types of Tests and How to Write Them
You’ve finally gotten your API set up. Now other applications can talk to your fully functioning REST API. Now what? What’s the next step? Making sure it works! How do you know if your REST API is functioning correctly? Well, naturally, you test it.
Why is testing important?
When you hear a developer refer to testing, we’re typically talking about automated testing. You may have done testing by manually making calls to your API using a tool such as Postman, Insomnia, or cURL. This works, but every time you go to change something then you have to go back and test that nothing else broke from your change.
Now, imagine you have a massive REST API that’s hundreds of thousands of lines long. Think about doing that same manual testing on every endpoint for every change. My gut reaction is to cringe because I don’t want to do that much busy work.
Manually testing every change just doesn’t scale to a massive API. You don’t want to spend hours testing just because you changed a utility file. Or, even worse, you don’t want to break something and not even know it.
The solution to these problems is to automate all your manual testing, so that you can verify your API’s functionality significantly faster. This also removes the risk of forgetting edge cases if you have all use cases safely documented in existing tests.
Types of testing
Automated tests come in many types and flavors. The different types of tests can be used as general guidelines and help you make sure you’re writing comprehensive tests. The two most common types of tests are unit tests and integration tests.
Unit tests
Unit tests ideally test a small unit (or rather, amount) of code. This can often mean one function or one if statement. What level of granularity you want in your tests depends largely on what could reasonably go wrong in code you’ve written.
Your unit tests should be aimed at testing one unit of your code. Your code doing one, tiny, actionis what you want to test. For example, you want to test that your pet_the_cat function returns the correct reaction from an imaginary cat. Given a class like this:
import requests
from typing import Optional
class Cat:
def __init__(self, cat: str):
self.cat = cat
self.fed = 0
def ask_what_to_do(self) -> dict:
return requests.get("http://www.boredapi.com/api/activity").json()
def pet_the_cat(self) -> Optional[str]:
actions = {
"catname": "action",
"bean": "purr",
"hades": "scratch",
}
return actions.get(self.cat)Code language: Python (python)You could either manually call your pet_the_cat function a bunch of times, giving it different input, OR you could write a unit test that look like:
from cat import Cat
def test_pet_the_cat():
tests = {
# "input_parameter": "expected_result",
"catname": "action",
"not_there": None,
1: None,
}
failed = 0
for input_value, expected_result in tests.items():
cat = Cat(input_value)
output = cat.pet_the_cat()
if output != expected_result:
print("FAILED: '{}' returned '{}'".format(
input_value, output,
))
failed += 1
print("{}/{} Tests succeeded".format(
len(tests) - failed, len(tests),
))
if __name__ == "__main__":
test_pet_the_cat()Code language: Python (python)Now, when you want to add more test cases, you add them to the tests dict, and run all of them by running the test functions once. Now you don’t have to remember all your test cases, and you don’t have to spend time running them manually.
The most fundamental rule of unit tests is that they should test only one unit of code at a time. Just like how in the example the feed_the_cat function is tested individually.
It’s also important to note that, as part of testing smallunits of code, you also don’t want to rely on any other service (like a database). Your unit tests should all be able to run independent of each other (and in parallel) and not have dependencies on other services. This is why ask_what_to_do isn’t tested in this example: because it has an external dependency (the API call).
While we can test small bits of code, you usually don’t need to write unit tests for built-in functions. After all, why test that 1+1 is 2? You can trust that the addition functionality built into your language is trustworthy. Also, you probably don’t need to write tests for third-party code that is well tested and used. Standard code used by everyoneshould have its own tests.
You also want comprehensive code coverage, meaning you want every line of your code to be used in at least one test. This means in an if … else code block there is a unit test for the if part and a unit test for the else part.
There are a lot of tools out there that can help you measure code coverage to ensure that every possible code path is covered. For Python, I’m partial to using the tool Coverage.py, but there are different options for different languages. For example, Istanbul is a popular tool for JavaScript, but I’ve seen plenty of people use Cypress’s code coverage plugin.
Ideally, because unit tests are cheap (in cost and time, partially due to the ability to parallelize them) to run, you want total code coverage using unit tests.
Integration tests
Of course, you still need to test how all the units of code you just unit tested work together. Enter integration tests, which specifically test how units of code interact together.
It’s worth noting that integration tests can have dependencies on things such as databases, and as a result are often slower to run. Ideally they should be written so that they can run in parallel, so even if they have dependencies on databases they’ll use their own (probably temporary) database table.
from cat import Cat
def test_ask_what_to_do():
cat = Cat("cat")
runs = 2
tests_passed = 0
for _ in range(2):
try:
result = cat.ask_what_to_do()
assert isinstance(result, dict)
assert "activity" in result
# Since the result is not always the same we
# can't test everything about the result, but it's
# an external dependency that has its own tests
tests_passed += 1
except:
continue
print("{}/{} Tests succeeded".format(
tests_passed, runs,
))
if __name__ == "__main__":
test_ask_what_to_do()Code language: Python (python)Other test types
There are many more types of tests, such as contract tests and functional tests (often confused with integration tests!), but unit and integration tests are far and away the most important tests to start with.
How to write tests
The first and most important step to writing tests is to write testable code. This might seem obvious, but if you don’t do it the first time you write your code, then you’ll probably have to refactor it. The alternative is to have tests that are a nightmare to read and implement.
What does writing testable code mean, though? Testable code is often best thought about from the unit test level first – can you call a small unit of code? If you have a function that’s hundreds of lines long, that’s probably not very testable. After all, how do you, using automation, isolate part of the function and run only that?
For example, looking at the following code snippet, how would you write unit tests for this?
import requests
from typing import Optional
class Cat:
def __init__(self, cat: str):
self.cat = cat
self.fed = 0
def pet_the_cat(self) -> Optional[str]:
found_pet = requests.get("https://petstore.swagger.io/v2/pet/1").json()
if found_pet.get("message") == "Pet Not Found":
self.cat = "UNKNOWN"
actions = {
"catname": "action",
"bean": "purr",
"hades": "scratch",
"diana": "sleep",
}
if self.fed % 2 == 0 and self.cat == "diana":
self.fed += 1
return "fed"
return actions.get(self.cat)Code language: Python (python)The answer is: You don’t. It would be difficult. Instead, you want to take that same code, and make it testable:
import requests
from typing import Optional
class Cat:
def __init__(self, cat: str):
self.cat = cat
self.fed = 0
def feed_the_cat(self) -> Optional[str]:
if self.fed % 2 == 0 and self.cat == "diana":
self.fed += 1
return "fed"
def get_cat_action(self) -> str:
actions = {
"catname": "action",
"bean": "purr",
"hades": "scratch",
"diana": "sleep",
}
return actions.get(self.cat)
def find_the_pet(self) -> None:
found_pet = requests.get("https://petstore.swagger.io/v2/pet/1").json()
if found_pet.get("message") == "Pet Not Found":
self.cat = "UNKNOWN"
def pet_the_cat(self) -> Optional[str]:
self.find_the_cat()
fed = self.feed_the_cat()
if fed is not None:
return fed
return self.get_cat_action()Code language: Python (python)Now, you can write separate unit tests for each function without dependencies and an integration test for the one that has dependencies.
Ok, I have unit testable code. Now make it integration testable.
After you have code that’s easy to unit test, think about your code at the integration test level. Are there clear lines, and ideally ‘contracts’ between modules of code?
For instance, earlier in our Cat example we had an ask_what_to_do function that returns different results on every call. However, are there things we know we can expect to be true about the data returned by that function?
Yes! We know the key names to expect in an error or a success. Because of this, we can test what we do know about the results:
assert isinstance(result, dict)
assert "activity" in resultCode language: Python (python)On the other hand, if we expected an error, we could have our test assert the keys expected:
assert isinstance(result, dict)
assert “error” in result
assert “activity” not in resultCode language: Python (python)ℹ️ Notice that the unit tests shown here can run without anything other than the file and python, while the integration tests cannot. This is because the unit tests don’t depend on the REST API to be something we can actually ‘hit’ or ‘call’ – it tests the underlying code. Meanwhile, integration tests try to actually call the API, so if the API isn’t being hosted in some way then the integration tests cannot run.
Testing code that has dependencies
So, we’ve specified that unit tests shouldn’t call to external dependencies, but it’s just a fact that code is going to have external dependencies, like our database, and network calls, like making a network call to the API.
So, how do we write unit tests for Python code that relies on dependencies? There are lots of options, but we’re going to talk about two common ones – unittest and pytest.
ℹ️ Learn how you can test for code that has dependencies in this guide to database unit testing with pytest and SQLAlchemy.
Unittest is built into Python and has all the basic things you’ll need to write tests. Pytest is often used in conjunction with unittest, as they share a fair bit of functionality. It’s fair to think of pytest as a tool designed to make writing and running tests easier. In particular, pytest enables functionality like running tests in parallel. Meanwhile unittest is a tool designed to enable writing tests, but it, by default, does not have the ability to run tests in parallel.
Using these tools, we can switch out dependencies with code that make it easier to test without modifying our source code. There are two ways to do this: dependency injection and mocking. In my experience people tend to be passionate about one or the other.
Dependency injection
Dependency injection is a complex topic, but here’s the gist: imagine writing your code in such a way that it’s possible to pass in different code to any calls to external dependencies during test time, allowing for a cleaner unit test. In other words, you ‘inject’ a fake dependency, making the code not call out to an external dependency.
To illustrate, let’s imagine when I’m ordering something online, and I choose to order a test item instead of a real item. The website says “oh, ok”, and doesn’t bother to charge me or send any kind of order to its database.
For example, we make a small change to our ask_what_to_do function in our cat class:
from typing import Callable, Optional
class Cat:
def __init__(self, cat: str):
self.cat = cat
self.fed = 0
def ask_what_to_do(self, request_function: Callable) -> dict:
return request_function("http://www.boredapi.com/api/activity").json()
def pet_the_cat(self) -> Optional[str]:
actions = {
"catname": "action",
"bean": "purr",
"hades": "scratch",
}
return actions.get(self.cat)Code language: Python (python)In ask_what_to_do, rather than directly calling requests.get we call request_function which is an argument we pass in. Now, we have control over what request_function is, rather than it always being a call out to requests.get (a dependency). Thus we can pass in a different function fake_request_function which does not call out to an external dependency, thus injecting the dependency.
And now, because we’re injecting the dependency, we can write a unit test for the function, ask_what_to_do, that previously had a dependency:
from cat_1 import Cat
class FakeFailResult:
def json(self):
return {}
class FakeSuccessResult:
def json(self):
return {
"activity": "Organize your pantry",
"type":"busywork",
"participants":1,
"price":0,
"link":"",
"key":"3954882",
"accessibility":0,
}
def fake_request_function(url) -> dict:
if url != "http://www.boredapi.com/api/activity":
return FakeFailResult()
return FakeSuccessResult()
def test_ask_what_to_do():
# Since all `ask_what_to_do` does is make this request, we can easily test
# that nothing has been accidentally added
cat = Cat("catname")
output = cat.ask_what_to_do(fake_request_function)
assert output == FakeSuccessResult().json()
if __name__ == "__main__":
test_ask_what_to_do()Code language: Python (python)💡 Dependency injection has more code up-front so it doesn’t make as much sense on a small scale, but it gains a lot more value when you get to the scale of classes and multiple things being passed around or called.
Mocking
Mocking (and no, I’m not mocking you–it’s actually called mocking) is about hijacking a call, be it a function call, or a class, or whatever, and giving back fake output. So, the external dependency is never called, even though the code thinks it was.
This is a lot like if you told me to do something, and I told you I did it, even though I never did. I gave you the output of “I did it”, but I never had to actually get up and go do something, so you didn’t have to wait for me to finish doing some action.
This makes a lot of sense on a small scale because it’s not much code, but at scale it gets unwieldy quickly. As you can imagine, you don’t want 500 mocks for one test.
A quick example:
Using our original implementation of Cat:
from unittest import mock
from cat import Cat
@mock.patch("cat.requests")
def test_ask_what_to_do(mocked_requests):
mocked_result = mock.MagicMock()
mocked_requests.get.return_value = mocked_result
cat = Cat("catname")
output = cat.ask_what_to_do()
assert output == mocked_result.json()
mocked_requests.get.assert_called_once_with(
"http://www.boredapi.com/api/activity"
)
if __name__ == "__main__":
test_ask_what_to_do()Code language: Python (python)Within Python mocks there are different types–Mocks, MagicMocks, AsyncMocks–all with their own specific uses (there are also things called fixtures and different tools you can use for testing), but I’ll save those for another blog post.
Mocking vs dependency injection
The reality is that ‘mocking’ is typically better for small scale projects without a lot of dependencies, because it doesn’t require a lot of work up front. However, mocks are not very reusable, so they don’t scale well.
Meanwhile, ‘dependency injection’ is better for large scale projects, but typically requires more up front work setting up all the fake dependencies. After you have dependency injection setup it’s easier to reuse fake dependencies across your code base, though, so it can be great for scaling.
In my experience these two concepts can be polarizing because they result in fundamentally different programming styles – where do you put the dependencies, how are they called, etcetera. That said, one style can be converted to another, so which you should use is a question based on either:
- If the decision is being made after the code is written, then it should be made based on the style of existing code.
- Scale of planned development of the code. For example: will this be a concept reused elsewhere in your project? A
Userobject is always going to be reused, so it’s something that, depending on the size of your project, is probably worth creating a fake instance of and injecting.
TL;DR
There are oodles and oodles of ways to test, and while that can be fun to dive into, the things you really need to know about testing are how to and when to use unit testing and integration testing.
Unit testing tests all the small pieces of code, cheaply and quickly because it doesn’t rely on any service (for example, an API) running. Meanwhile integration testing tests if all those units of code work together and is more expensive due to dependencies like database time.
💡 Pro tip: Write testable code and tests in interviews! It can really set you apart.
For a more front-end focused take on testing you can check out CoderPad’s blog on testing types, oriented at front-end development. Different tests are better for different areas of code. It’s always good to bear in mind what you’re testing, after all.
Jennifer is a full stack developer with a passion for all areas of software development. He loves being a polyglot of programming languages and teaching others what he’s learned.