Software testing is the structured evaluation of software to find defects, prove that requirements are met and reduce business risk before and after release. For CTOs and engineering leaders, software testing is not a QA department checklist; they are a delivery capability that protects revenue, reputation, compliance and product velocity.
TL;DR / Key Takeaways
- Test by risk, not habit: prioritise the journeys, integrations, data flows and regulatory obligations that would hurt the business most if they failed.
- Use multiple testing types: unit, integration, system, acceptance, regression, performance, security and exploratory testing each expose different failure modes.
- Automate where repeatability matters: automated tests are strongest for stable, high-frequency checks in CI/CD; manual testing remains essential for judgement-heavy scenarios.
- Treat compliance as evidence: GDPR, NIS2, the Cyber Resilience Act and the EU AI Act increasingly require demonstrable controls, not informal assurances.
- Measure outcomes: track escaped defects, change failure rate, flaky tests, mean time to restore and business-critical coverage rather than raw test counts.
What is software testing and why does it matter?
Software testing is the process of evaluating software products, components and systems against expected behaviour, quality attributes and business risk. It matters because modern organisations run on software: defects can trigger downtime, security exposure, regulatory scrutiny and customer churn. Effective testing gives leaders evidence-based confidence to release faster without gambling on quality.
According to ISO (ISO/IEC/IEEE 29119-1:2022, published January 2022), the international standard defines general concepts for software testing and positions them within the wider ISO/IEC/IEEE 29119 series. That matters because it frames testing as an engineering discipline with processes, artefacts and governance, not as a late-stage bug hunt. (iso.org)
The business case is equally clear. According to CISQ, The Cost of Poor Software Quality in the US: A 2022 Report (copyright 2022; 2022 estimate), poor software quality cost the US at least $2.41 trillion, with accumulated technical debt estimated at approximately $1.52 trillion. This is a US economic estimate, not a European enterprise benchmark, but it illustrates the scale of waste created when defects, vulnerabilities and technical debt escape into production. (it-cisq.org)
For mid-sized European companies, the practical issue is rarely “do we test?” The issue is whether testing is aligned with delivery risk. A team may have hundreds of low-value checks while still leaving payment flows, cross-border VAT logic, identity management or customer data exports weakly tested.
A mature software testing strategy should answer five executive questions:
- What could fail in a way that harms customers or revenue?
- Which defects are most likely to escape given our architecture and delivery model?
- Which checks should run automatically on every change?
- What evidence do we need for auditors, customers and regulators?
- How quickly can we detect, isolate and fix production defects?
The strongest testing organisations connect these questions to product ownership, architecture and DevOps metrics. According to DORA’s 2024 Accelerate State of DevOps Report (latest corrected version v.2024.3; errata updated 8 October 2024), teams should take an experimental approach to continuous improvement and assess their current baseline before changing delivery practices. Testing should therefore improve flow and reliability together, not create a separate gate that engineering teams try to bypass. (dora.dev)
A useful operating model is the “risk-based testing pyramid”: many fast unit tests at the base, fewer integration and API tests in the middle, and carefully selected end-to-end checks at the top. This keeps feedback fast while still validating the customer journeys that matter.
What types of software testing should a modern team use?
Modern teams should use a layered mix of functional, non-functional, manual and automated testing. Unit tests check small code units, integration tests validate interactions, system tests assess end-to-end behaviour, and acceptance tests confirm business intent. Security, performance, usability and regression testing then address the risks that functional tests alone cannot cover.
According to ISTQB’s Certified Tester Foundation Level Syllabus v4.0.1 (dated 15 September 2024), the Foundation Level is aimed not only at testers but also at software developers, project managers, quality managers, product owners, IT directors and management consultants. That breadth reflects a key reality: testing is a cross-functional responsibility, not a single role. (istqb.org)
The following list is the concise map most leaders need before selecting tools or restructuring QA.
| Testing type | What it proves | Typical owner |
|---|---|---|
| Unit testing | A small function or class behaves correctly | Developer |
| Integration testing | Services, modules or APIs work together | Developer / QA |
| System testing | The full application meets requirements | QA / product team |
| Acceptance testing | The software supports business outcomes | Product owner / user representative |
| Regression testing | Existing behaviour still works after change | QA / automation team |
| Performance testing | The system meets load, latency and scalability needs | QA / platform team |
| Security testing | Vulnerabilities and misuse paths are identified | Security / DevSecOps |
| Usability testing | Users can complete tasks effectively | UX / product team |
| Compatibility testing | Software works across target environments | QA / release team |
| Exploratory testing | Unknown risks are investigated through human judgement | Experienced tester |
Unit, integration and system testing
Unit testing gives fast feedback close to the code. It is best used for business rules, calculations, data transformations and edge cases that can be isolated without a full environment. In Java, JUnit remains a common foundation; according to the JUnit 5 User Guide (version 5.13.1), JUnit 5 combines the JUnit Platform, JUnit Jupiter and JUnit Vintage, with Jupiter providing the programming and extension model for modern tests. (docs.junit.org)
Integration testing catches a different class of defect: contracts, schemas, message formats, authentication flows and service dependencies. This is especially important for microservices, ERP integrations, payments, warehouse systems and customer portals where “the code works” is not the same as “the workflow works”.
System testing then validates the application as a complete product. It should cover realistic data, configuration, roles, permissions and failure conditions. For B2B platforms, this often includes account hierarchy, invoicing, multilingual content, audit logging and third-party service degradation.
Acceptance, regression and exploratory testing
Acceptance testing translates business intent into verifiable examples. Behaviour-driven development can help here, but only if scenarios are written in language that product, engineering and QA all understand. A test that nobody trusts becomes documentation theatre.
Regression testing protects existing value. The best regression packs are selective, risk-based and maintained like production code. Weak regression suites grow until they are slow, flaky and politically impossible to delete.
Exploratory testing remains essential because not all risk is known in advance. Skilled testers use product knowledge, heuristics and curiosity to find gaps that scripted checks miss. In complex workflows, a single exploratory session can reveal unclear error handling, inconsistent permissions or hidden data quality assumptions.
Performance, security and usability testing
Performance testing should begin before a major launch, not during the incident bridge. According to Apache JMeter documentation (current official site accessed 2026), JMeter is open-source software designed to load test functional behaviour and measure performance across web applications, APIs, databases and other protocols. (jmeter.apache.org)
Security testing must include dependency risk, authentication, authorisation, input validation, secrets handling and abuse cases. According to OWASP’s Web Security Testing Guide (latest online edition accessed 2026), the guide covers a testing framework, web application security testing and reporting resources. (owasp.github.io)
Usability testing closes the gap between “functionally correct” and “usable under pressure”. In a B2B system, poor usability becomes operational cost: support tickets, training overhead, workarounds and data errors.
Which software testing tools are essential?
Essential software testing tools include unit test frameworks, API testing tools, browser automation frameworks, performance testing tools, security testing resources, test management systems and CI/CD platforms. The right toolkit depends on your architecture and team maturity. Avoid buying a platform before clarifying test ownership, environments, data strategy and maintenance capacity.
Tool selection should start with the testing layer, not the vendor shortlist. A React front end, Java back end and event-driven integration landscape need a different toolchain from a Python data platform or a regulated SaaS product with strict audit requirements.
| Tool category | Common options | Best fit |
|---|---|---|
| Unit testing | JUnit, pytest, NUnit, Jest | Fast code-level feedback |
| API testing | Postman, REST Assured, Pact | Service contracts and workflows |
| Browser automation | Selenium, Playwright, Cypress | User journeys and regression |
| Performance testing | Apache JMeter, Gatling, k6 | Load, stress and capacity checks |
| Security testing | OWASP ZAP, Snyk, Semgrep, dependency scanners | Vulnerability discovery |
| CI/CD execution | GitHub Actions, GitLab CI, Jenkins, Azure DevOps | Automated quality gates |
| Test management | Xray, Zephyr, TestRail | Traceability and reporting |
According to pytest documentation (stable documentation accessed 2026), pytest is designed to make small tests easy to write while scaling to complex functional testing for applications and libraries. That makes it a practical choice for Python teams that want readable tests, fixtures and plugin extensibility. (docs.pytest.org)
For browser automation, the Selenium and Playwright choice often reflects legacy estate, team skills and reliability needs. According to Selenium documentation (last modified 16 September 2026), Selenium is an umbrella project for tools and libraries that support browser automation and WebDriver-based cross-browser execution. According to Microsoft’s Playwright documentation (accessed 2026), Playwright provides automation across Chromium, Firefox and WebKit, with auto-waiting, assertions, tracing and parallelism. (selenium.dev)
API testing deserves special attention in enterprise systems. According to Postman documentation (accessed 2026), Postman collections can act as saved requests, workflows or test suites, and can be run to test API functionality. This is valuable when product teams need a shared, executable view of API behaviour across environments. (learning.postman.com)
CI/CD turns these tools into a delivery system. According to GitHub Docs (accessed 2026), GitHub Actions can create CI workflows that automatically build and test code to check whether a commit introduces errors. The same principle applies across GitLab CI, Jenkins and Azure DevOps: tests should run where developers already change software. (docs.github.com)
Selection criteria for CTOs
Use these criteria before committing to licences, frameworks or a “quality platform”:
- Architecture fit: does the tool support your languages, browsers, APIs, queues and deployment model?
- Developer experience: can engineers run tests locally without specialist support?
- CI reliability: does it handle parallel execution, isolation and deterministic output?
- Maintainability: are tests readable, version-controlled and owned by product teams?
- Observability: can failures be linked to logs, traces, screenshots, network calls and commits?
- Compliance evidence: can results be retained, exported and mapped to controls?
The best tooling decision is often to remove complexity. Many teams need fewer tools, better ownership and cleaner test data before they need another dashboard.
How should you apply effective software testing techniques?
Apply testing techniques by matching them to risk, feedback speed and human judgement. Automate deterministic checks, use manual exploration for ambiguity, apply boundary and equivalence techniques to reduce test volume, and integrate results into CI/CD. The objective is not more testing; it is earlier, clearer and more trustworthy feedback.
Manual and automated testing are not rivals. They answer different questions. Automation asks, “Does this known behaviour still work?” Manual testing asks, “What else might be wrong, unclear, risky or inconvenient?”
| Approach | Best use | Risk if misused |
|---|---|---|
| Manual testing | Exploratory, usability, new features, ambiguous workflows | Slow repetition and inconsistent coverage |
| Automated testing | Regression, API checks, unit tests, smoke tests, data validation | Flaky suites and false confidence |
| Static analysis | Code quality, security patterns, maintainability | Noise if rules are unmanaged |
| Dynamic testing | Runtime behaviour, performance, security, end-to-end flows | Late feedback if not automated |
| Contract testing | API compatibility between services | Poor value if contracts are not owned |
Use design techniques to reduce waste
Equivalence partitioning, boundary value analysis, decision tables and state transition testing help teams cover more risk with fewer cases. For example, an invoicing rule may not need 200 tests if values can be grouped into valid, invalid and boundary categories.
Decision tables are particularly useful for B2B products with complex combinations: customer type, region, tax status, discount level, approval threshold and payment method. State transition testing works well for workflows such as onboarding, order fulfilment, claims handling and subscription lifecycle management.
Build quality into CI/CD
A practical CI/CD testing sequence might look like this:
- Pre-commit: linting, secrets scanning and fast local unit tests.
- Pull request: unit tests, API contract tests, static analysis and targeted integration tests.
- Merge: broader regression, container build, dependency checks and smoke tests.
- Pre-release: end-to-end tests, performance baselines and security checks.
- Production: monitoring, synthetic tests, feature flags and incident feedback loops.
According to DORA’s 2024 report (published 2024; latest corrected version v.2024.3), teams that prioritise end-user needs and operate under stable organisational priorities are better positioned to drive product quality and organisational success. This is directly relevant to testing: tests should protect user value, not merely satisfy an internal QA process. (dora.dev)
Manage flaky tests aggressively
Flaky tests are more than an annoyance. They train engineers to distrust the pipeline. Once that happens, teams rerun instead of investigate, override failures and quietly move quality judgement back into informal channels.
Treat flakiness as technical debt with an owner, severity and SLA. Quarantine unreliable tests, measure recurrence and fix root causes such as timing assumptions, shared state, unstable environments, poor selectors and external service dependencies.
Connect test strategy to architecture
Architecture determines what is testable. If services lack clear contracts, data ownership is blurred and environments cannot be provisioned reliably, testing becomes expensive. Conversely, modular architecture, observability and clean deployment boundaries reduce test cost.
This is why automated testing and integration testing should not be side topics. They are core engineering capabilities that influence release frequency, incident rates and customer trust. For where a dedicated QA function fits in the development process, see our guide to QA testing in software development.
How does testing support compliance and governance?
Testing supports compliance by producing evidence that software controls work as intended. For European organisations, this means mapping tests to data protection, cybersecurity, operational resilience and AI obligations. Compliance testing should verify access controls, audit logs, encryption, retention, resilience, vulnerability handling and change traceability across the delivery lifecycle.
Compliance is no longer only a legal or security function. It is increasingly embedded in software delivery. European CTOs need test evidence that can stand up to customer due diligence, ISO audits, procurement questionnaires and regulator scrutiny.
According to EUR-Lex text for GDPR Article 32 (Regulation (EU) 2016/679, adopted 27 April 2016 and applicable from 25 May 2018), controllers and processors must implement appropriate technical and organisational measures, including where appropriate a process for regularly testing, assessing and evaluating the effectiveness of security measures. This is a direct mandate for security control validation, not simply policy documentation. (eur-lex.europa.eu)
NIS2 raises the board-level importance of operational resilience. According to EUR-Lex for Directive (EU) 2022/2555 (NIS2, adopted 14 December 2022), Member States had to transpose the Directive by 17 October 2024 and apply measures from 18 October 2024; Article 21 requires appropriate and proportionate technical, operational and organisational cybersecurity risk-management measures. Corporate obligations depend on each Member State’s national implementation, so testing plans must be localised for entities in scope. (eur-lex.europa.eu)
The Cyber Resilience Act changes expectations for software products placed on the EU market. According to the European Commission’s CRA summary (Regulation (EU) 2024/2847, adopted 23 October 2024; Official Journal 20 November 2024), the Act applies to hardware and software “products with digital elements” made available on the Union market, and manufacturers must consider cybersecurity across planning, design, development, production, delivery and maintenance. (digital-strategy.ec.europa.eu)
The CRA also creates reporting obligations. According to the European Commission’s CRA reporting guidance (accessed 2026), manufacturers must report actively exploited vulnerabilities and severe incidents affecting products with digital elements from 11 September 2026, with open-source software stewards subject to Article 24(3) reporting obligations from 11 December 2027. (digital-strategy.ec.europa.eu)
AI systems add another layer. According to the European Commission AI Act Service Desk summary of Article 15 (based on the EUR-Lex consolidated AI Act text as at 27 July 2026), high-risk AI systems must achieve appropriate accuracy, robustness and cybersecurity throughout their lifecycle, with relevant accuracy metrics declared in instructions for use. Testing AI therefore involves data quality, robustness, adversarial behaviour, monitoring and human oversight, not just functional checks. (ai-act-service-desk.ec.europa.eu)
| Regulation / framework | Testing implication | Evidence to retain |
|---|---|---|
| GDPR Article 32 | Regularly test security controls for personal data | Security test results, access reviews, incident recovery tests |
| NIS2 Article 21 | Validate cybersecurity risk-management measures | Risk-based test plans, vulnerability remediation records |
| Cyber Resilience Act | Test product cybersecurity across lifecycle | Secure development evidence, vulnerability handling records |
| EU AI Act Article 15 | Test accuracy, robustness and cybersecurity of high-risk AI | Evaluation reports, metrics, monitoring logs |
| NIST SSDF | Integrate secure development practices into SDLC | Secure coding, review and vulnerability testing evidence |
| OpenSSF SLSA | Strengthen supply-chain integrity | Provenance, build integrity and dependency evidence |
According to NIST SP 800-218 (SSDF Version 1.1, published February 2022), the Secure Software Development Framework provides high-level secure software development practices that can be integrated into each SDLC implementation. According to OpenSSF’s SLSA project description (accessed 2026), SLSA provides incrementally adoptable guidelines for software supply chain security and a common vocabulary for producers and consumers. (csrc.nist.gov)
For decision-makers, the established approach is to create a compliance-to-test matrix. Map each obligation to test scenarios, owners, environments, evidence retention and remediation workflow. This turns compliance from a yearly scramble into an engineering control system.
If your test suite is slow, flaky or missing the flows that matter, WWG's automated testing service builds risk-based unit, API and end-to-end suites into your CI/CD pipeline. If you first need to know where the gaps are, a software audit maps test coverage against architecture and delivery risk.
Talk to our engineering team →
Sources
- ISO — ISO/IEC/IEEE 29119-1:2022, Software and systems engineering — Software testing — Part 1: General concepts (published January 2022). (iso.org)
- ISTQB — Certified Tester Foundation Level Syllabus v4.0.1 (dated 15 September 2024). (istqb.org)
- CISQ — The Cost of Poor Software Quality in the US: A 2022 Report (copyright 2022; 2022 estimates). (it-cisq.org)
- DORA / Google Cloud — 2024 Accelerate State of DevOps Report and 2024 errata (latest corrected version v.2024.3; errata updated 8 October 2024). (dora.dev)
- JUnit — JUnit 5 User Guide, version 5.13.1. (docs.junit.org)
- pytest — stable documentation, pytest: helps you write better programs. (docs.pytest.org)
- Selenium — official documentation, The Selenium Browser Automation Project (last modified 16 September 2026). (selenium.dev)
- Microsoft Playwright — official documentation, Playwright enables reliable web automation for testing, scripting, and AI agents. (playwright.dev)
- Apache JMeter — official documentation, Apache JMeter™. (jmeter.apache.org)
- OWASP — Web Security Testing Guide, latest online edition. (owasp.github.io)
- Postman — official documentation, Test your API functionality. (learning.postman.com)
- GitHub Docs — Continuous integration. (docs.github.com)
- EUR-Lex — GDPR, Regulation (EU) 2016/679, Article 32. (eur-lex.europa.eu)
- EUR-Lex — NIS2 Directive, Directive (EU) 2022/2555, Articles 21 and 41. (eur-lex.europa.eu)
- European Commission — Cyber Resilience Act summary and reporting obligations for Regulation (EU) 2024/2847. (digital-strategy.ec.europa.eu)
- European Commission AI Act Service Desk — EU AI Act Article 15, accuracy, robustness and cybersecurity. (ai-act-service-desk.ec.europa.eu)
- NIST — SP 800-218, Secure Software Development Framework Version 1.1 (published February 2022). (csrc.nist.gov)
- OpenSSF — SLSA, Supply-chain Levels for Software Artifacts. (openssf.org)
FAQ
Frequently Asked Questions
Practical answers for technology leaders planning a stronger software testing strategy.
