Two proposals both say “web application penetration test”. One examines the portal from outside without an account. The other includes customer roles, APIs and an export journey. Their reports could both describe the work accurately while answering different questions.
For a CTO buying a test before a portal launch or enterprise review, the useful comparison starts with the decision the test must support. Can an outsider enter the application? Can a legitimate customer reach another customer’s records? Will a corrected export be independently retested before you make the commitment?
The test is one part of that work. Preparation, correction, deployment and acceptance need owners too.
Answer in 60 seconds
Give each supplier the same application journey, roles, environments and question. Ask them to mark what they will examine, what they will exclude and what preparation they need. Then agree who will triage findings, implement and release corrections, perform the retest, and accept any remaining risk.
Compare proposals on that basis before comparing their totals. A broader description is useful only when the included work, evidence, exclusions and retest conditions are explicit.
1. Start with the question the test must answer
Write one decision sentence: “Before opening the customer portal, we need an independent assessment of whether the agreed customer roles can exceed their access through the portal and its APIs.” Add the release or customer decision it will inform.
An external-entry question could legitimately lead to a different scope. Do not assume the terms black box, grey box or white box settle coverage. Record the information and access supplied to the testers, then identify the journeys that become testable with them.
The UK NCSC’s commissioning guidance brings risk owners, technical staff and the test team into scoping. It is foundational UK guidance, published in 2017 and reviewed in 2022, rather than evidence of current procurement demand or a French requirement.
Where a contract or applicable requirement calls for a particular qualification, establish that separately. In France, ANSSI’s PASSI resources distinguish qualification by audit activity, including penetration testing. A general security-services description does not establish the required qualification for a particular engagement.
2. Describe one journey that crosses the important boundaries
Consider this illustrative portal. It is not an IZZY client case or a real supplier proposal.
Organisation A has an administrator and a reader. Organisation B has separate accounts and synthetic records. A’s administrator invites a colleague, opens a customer document, requests an export and receives a completion notification. The colleague’s membership is then revoked.
The journey raises several related questions. Can each role do only what the product permits? Does the API apply those same permissions? Does the export stay within A’s records? What should happen to access after revocation? If the portal uses a third-party identity service or sends a webhook, where does the authorised test stop?
Provide the intended answers as product rules. For a temporary download link, for example, agree whether possession grants access until expiry or whether retrieval requires current membership. Otherwise testers and developers may evaluate different policies.
This journey is a scope discussion aid. It does not replace the provider’s expertise or prescribe a complete test methodology.
3. Walk that journey through two proposals
The two scopes below are deliberately illustrative. They contain no prices, resource estimates or claims about named providers.
| Scope question | Proposal A: external exposure | Proposal B: authenticated portal |
|---|---|---|
| Access supplied | Public URLs; no customer accounts | Accounts in two synthetic organisations, with agreed administrator and reader roles |
| Intended question | What can an unauthenticated outsider reach? | Can the supplied customer roles exceed their intended permissions? |
| Journey coverage | Public entry and exposed interfaces; authenticated export journey excluded | Agreed invitation, document, API, export and notification paths included |
| Third-party systems | Provider infrastructure excluded | Provider infrastructure excluded; portal-side integration handling included only as specified |
| Correction and retest | Recommendations delivered; implementation and retest not included | Recommendations and a defined retest window; implementation remains with the engineering owner |
Ask each supplier to state the testing effort in tester-days, the time allocated to important journeys and any limits on depth imposed by that allocation. Ask who will perform the work and what relevant application and API testing experience they bring. Keep this effort separate from calendar dates, reporting time and the retest window.
Run the export journey through A. The report may explain what was reachable without a customer account. It cannot establish whether an A administrator can obtain B’s export through an authenticated route that was explicitly excluded. That is an unanswered question, not evidence that the supplier performed badly.
Run it through B. The authenticated export is included, but “export” still needs a start and finish. Does it cover creation, the background job’s effects, retrieval of the resulting file and access after revocation? If only the export request is examined, the download question remains open.
B also leaves engineering work: fix the finding, deploy the correction to the agreed environment, supply a retestable build, and decide whether anything unresolved prevents release. A retest window does not perform those steps for you.
4. Turn the scope into executable preparation
Attach an inventory that joins the journey to the actual application: browser entry points, relevant API versions, export and notification routes, roles, organisation membership, environment and build. Name the exclusions beside the affected journey rather than hiding them in a general appendix.
Use the OWASP Web Security Testing Guide to discuss test-method references. The Application Security Verification Standard provides requirements that can also support procurement. Ask which version and applicable areas the proposal uses, and how the report will distinguish examined, unexamined and inapplicable coverage. A reference to “OWASP” alone leaves that unresolved.
Before the booked date, confirm that accounts work, their roles differ as intended and synthetic records exercise the relevant relationships. Give the provider current API documentation and a technical contact able to unblock access. Agree permitted methods, operational limits, urgent reporting, stop conditions and evidence handling.
Prefer an authorised test environment with representative synthetic data where it answers the question. If production testing is needed, settle the specific authority and operating conditions first. Including your integration does not automatically include the supplier’s infrastructure: agree a safe test equivalent or record the resulting limitation.
What we would have ready before booking a date: test accounts for each role in two organisations, a one-page description of the roles, a map of integrations, a separate environment with synthetic data, and engineering time reserved for fixes. A report that arrives with nobody free to fix it ages fast.
These preparations make the requested journey available for examination. They do not promise that every weakness will be discovered.
5. Agree correction and retest ownership before testing
A finding can be reported clearly and still stall if nobody owns the next state. Write the handoff while arranging the engagement.
| Work | Owner to name | Evidence or decision required |
|---|---|---|
| Prepare the application | Internal engineering lead | Working accounts, data, documentation and agreed build |
| Explain the finding | Testing provider | Affected scope, controlled evidence, impact and correction guidance |
| Triage the response | CTO or delegated risk owner with engineering | Priority, interim action and accountable correction owner |
| Implement and deploy | Engineering team or remediation partner | Reviewed change, regression checks and deployed build identity |
| Retest | Agreed testing provider | Result for the corrected finding and explicit limits |
| Accept what remains | Authorised business/risk owner | Release decision and recorded unresolved issues |
The NCSC guidance explicitly leaves risk assessment and decisions about fixes with the organisation. The report informs that responsibility; it does not absorb it.
On a call with a CTO, we put it this way: “The tester finds and proves. Your engineers, or ours, fix and release. The tester then checks the fix. Nobody signs off their own work.”
Define retest conditions in practical terms: when the window starts and ends, how many rounds are included, what “ready for retest” means, which environment is used, and what happens if a correction misses the window. Ask whether related routes are included or need additional scope.
If a fix disables an export, distinguish “the original access is prevented” from “the required feature works with the intended permissions”. Give engineering responsibility for functional acceptance and regression evidence alongside the independent security recheck. Record unresolved and untested cases separately from corrected findings.
6. Compare the work left to reach the decision
For each proposal, assemble the whole sequence: internal preparation, supplier testing, triage, correction, release, retest and final acceptance. Mark what the quote covers, what your team will do and what needs a separate agreement. Include calendar dependencies as well as active work; the retest cannot start before the appropriate correction is available.
You can now compare what it would take to answer the original question. Proposal A may suit external exposure while leaving authenticated customer access for another engagement. Proposal B may cover that journey while leaving implementation and some integrations outside scope. Compare the quoted fee alongside the included testing effort, assigned responsibilities and work still needed to reach your decision.
If the product is changing hands, use the codebase handoff evidence to establish who can build, release and operate the correction. That ownership is needed before a retest date becomes a dependable commitment.
Take the next practical step with the two proposals already on your desk: ask each supplier to walk the same journey and mark every unanswered question. Assign the internal work before confirming the engagement.
Prepare the test with us
Where we draw the line: we prepare the application and fix what the report finds. We do not run the penetration test ourselves or sign off its results; that assessment stays with the independent specialist you commission.
If an inherited application needs engineering preparation or corrections around an independent test, discuss that scope through our Rescue Missions & AI-Code Hardening service. Bring the journey, proposal and available codebase evidence. The engineering work and the independent penetration test should have clear, separate responsibilities.
Sources and limits
Sources checked on 10 November 2026. This is a research synthesis of NCSC commissioning guidance, ANSSI qualification resources and OWASP references. The portal and proposals are illustrative; no supplier price, qualification, result or IZZY client experience is asserted. A pentest addresses an agreed scope and period. Its report and retest do not establish that every part of an application is secure or that every contractual or regulatory requirement is met.