Mock Recording
Sub-second integration testing for apps, web, services and embedded software.
Faster feedback
We all want to be able to produce working code faster, and one well-etablished technique is automated testing. Simple unit testing of algorithms, calculations and data structure manipulations usually work well enough. However, many teams struggle with tests that are either slow to run or not very robust. So how can we improve the feedback loop from tests, so developers can get high quality feedback within a few seconds or minutes instead of hours or days?
Challenges
Much software has complex interactions with other software systems. Examples include:
- Web service calls from the user interface running in a browser
- Bluetooth / BLE commands from a mobile app to a managed device
- Integrations with third party services outside the control of the development team
- Audio or video streams channeled from a server via an app to underlying media APIs
Such interactions often cross process boundaries and are usually implemented on top of a standard communications protocol. Often the "other end" of the protocol is itself a software system under development, maybe even running on a device where also the hardware is under development. Common communication protocols include:
- HTTP for web services
- Bluetooth Low Energy (BLE) for communication between a phone and a device.
- HLS or RTMP for streaming media
Traditional solutions
The two traditional approaches to test software with complex interactions are
End-to-end testing has the benefit of accurately representing production conditions of the software under test. However, end-to-end testing is infamous for being very slow and very brittle. Comprehensive test suites for even medium sized software systems can easily take many hours and frequently have double digit failure rates with failures happening in different tests for each test run. Not a good thing from a feedback perspective!
Unit testing with manual mocks has the benefit of being very robust and fast, however, manual mocks tends to grow stale or simply wrong - they reflect assumptions about the interactions at the time when the mocks were created, and if not meticously updated to match changes in the connected systems, unit tests with manual mocks can easily succeed even when the actually integrated system is broken. Not a good thing from a testing perspective!
A better solution
To get fast and high-quality feedback, the development team needs tests that are fast, robust and accurate. The accuracy problem of manual mocks can be solved by recording the mocks instead of writing them by hand.
Mock recording works with a "recorder" and "player". You will create "recorder" tests that work like end-to-end tests, but with a proxy inserted between your protocol library and the real library. The recorder tests will cover all the scenarios that you need recordings for, and store the recordings locally in files to be included in your code repository as "test data". In your unit tests you can then use a "player" along with the recordings to automatically mock the protocol library with the recorded traffic. This ensures that your unit tests stay accurate because they use a recording that is re-created frequently. The recorder tests are still somewhat slow and fragile, but they do not need to run as often as the unit tests. Usually you will run them only daily or weekly.
Mock recording is a general technique that can be applied for any kind of protocol. Using a mock recording tool dedicated to the protocol can make the task a lot easier, and thankfully several such tools already exists.
Depending on your specific needs, you can read more about the specific tools on the following pages:
Lars Thorup
This web site was created in September 2021 by Lars Thorup to collect information about mock recording techniques.
Lars Thorup is a freelance software developer with experience from working at large and small companies in Denmark and the USA. He has worked with efficient test automation since the 1990s and has promoted eXtreme Programming practices since 2001. Lars has used mock recording for HTTP since 2016, and introduced mock recording for BLE at SOUNDBOKS in 2021.
You can find talks, slides and blog posts from Lars on the topic of mock recording on his blog:
- Mock recording for BLE and React Native at the React Native EU 2021 conference
- Mock recording for BLE and React Native blog post
- Mock recording for HTTP in JavaScript and Java (in Danish)
Lars can be contacted at [email protected] or via LinkedIn. Please get in touch to discuss how mock recording can be applied successfully to improve testing speed and quality on your team.
Enjoy!