Firmware is an essential part of our everyday life and encompasses a wide range of devices — from your TV remote to embedded systems that run complex industrial production systems. Testing is vital to create software that runs complex embedded systems.
Firmware testing is the same as any other kind, the difference being the means used. Using the testing suites allows a less-technical team to do the testing. Regression testing is great for the validation process and it also helps to quickly identify issues with the hardware in the field. To understand what goes into it, let’s have a better look at the firmware testing methods using an example of one of our Clients.
The Client that worked with us is a leader in network security appliances development and provides security solutions and secure remote maintenance. The company manufactures it specifically for industrial environments. Its development team is focusing on industrial Ethernet and security, embedded hardware/software development, and the integration of security components.
The Client incorporated a new security appliance into existing product lines, allowing secure and easy remote services management. The main functions of this solution are configuration creation, ability to upload configurations to field devices, access and permission management, channel operations.
The quality of the products is of the key importance for the company. Due to the tight schedule and large-scale of the project, there was a need in a partner with deep knowledge in networking technologies. Our company was able to integrate its dedicated QA team promptly and take responsibility for the quality assurance expertise of the new security appliance in order to integrate it into the existing product lines. Our team is working in close cooperation with the development team on the Client side.
Our company possessed substantial expertise in developing network appliances for Windows, Linux and OSX client systems. This allowed quick and accurate estimation of the required effort and forming a team of experienced QA engineers. The Client has provided a basic test plan, and SaM Solutions’ activities included, but not limited to:
- Test plan adaptation and extension;
- Quality procedures identification and description;
Test environment setup;
Test cases development and grouping into test suites;
- Continues process improvement was implemented with PDCA cycle, including regular Client feedback
- proactive approach based on previous experience.
Test environment had to be complicated and multi component to ensure correct system operation in various network configurations and usage scenarios. SaM Solutions removed deployment and configuration overhead at the very beginning by developing an automated test environment configuration framework based on Linux KVM. This resulted in “one-click” reconfiguration by any team member for the chosen test suite.
Performance and scalability being of the crucial value of the product, SaM Solutions specialists developed a special cluster-based automated test suite allowing almost unlimited N:M network connections setup, traffic generation and monitoring.
Languages: C, BASH
Platforms: Linux, Windows
Test case management tool: TestLink
Bug tracking system: Bugzilla
Control version system: Git
Virtualization platform: Windows XP/ 7
Firmware testing tools: PostgreSQL, IPMI, TFTP, IPTables, SSH, Telnet, XML, etc
By choosing SaM Solutions as an offshore QA service partner for firmware testing, the Client has acquired an easy way to ensure the high quality of its products. SaM Solution’s engineers have gained a high appreciation for their knowledge, flexibility, performance, and the firmware test automation frameworks developed as side projects and adopted by the Client for regression testing.