Scanning Behind a Sign-In

Using Octopus Scanner

How to audit account areas, checkouts and other pages that need a sign-in, without saving your password or putting the account at risk.

1. Choose a method

Octopus Scanner offers several ways to reach pages that need a sign-in. Pick the one that matches how the site signs people in and what you want to audit.

  • Login URL: You sign in yourself in a visible browser window, then the scan continues with that session. This works with single sign-on and two-factor authentication.
  • User Journey: The scanner replays a sequence of steps, such as typing a username and password, then audits the pages you choose. Use it for repeatable checks of a fixed path, such as a checkout.
  • Address with a username and password: For sites that sign in through the browser's own pop-up.
  • Interactive Mode: You move through the site yourself in a browser window and select Scan for each page or state you want audited.

The Login URL and User Journey options sit in Customise Scan, under Target & Scope. Tick Login Page or User Journey to show their settings. The User Guide describes every option in that section.

2. Sign in with a Login URL

  1. Enter the site's address in the Website URL or local file path field as usual.
  2. Open Customise Scan, tick Login Page and enter the address of the sign-in page in Login URL.
  3. Select Start Scan. A browser window opens at the sign-in page.
  4. Sign in in that window, completing any two-factor step, until the signed-in content is showing.
  5. Return to the scanning interface and select Confirm Login is Complete. The scan then continues in the signed-in session.
Target & Scope with Login Page ticked. The sign-in address is an example.
Text description

Example of the Target and Scope section. Under Choose What To Set Up, Login Page is ticked, and Sitemap, Specific URLs and Multi-Domain, Interactive Mode (Manual Step-by-Step) and User Journey are unticked. Below, the Login URL (Optional) field contains https://www.example.com/login, with the hint: For manual login, the scanner will open a browser to this page first.

The scan waits up to 20 hours for Confirm Login is Complete, so take as long as you need. If you close the sign-in window, it continues without being signed in and warns in the log that the results may be inaccurate. If the sign-in page is slow to load, the window stays open so you can still sign in.

If the scanning browser has to restart part-way through, for example after a crash, your sign-in is carried over to the new browser, and PDFs are checked while signed in too.

Entering a Login URL turns on Persistent Session so the sign-in can be reused, and it turns off Parallel Processing and Periodic Browser Restarts for that scan.

3. Automate sign-in with a User Journey

A journey replaces the normal crawl. Only the pages where you add a Scan Page step are audited, and a journey without one is rejected before the scan starts.

  1. Enter the site's address in the Website URL or local file path field. Recording needs a valid http or https address.
  2. Open Customise Scan and tick User Journey.
  3. Select Record Journey. A browser opens and records your clicks, typing, selections and key presses as steps. If the journey already has steps, the interface first asks you to confirm that the recording replaces them. Select Cancel to keep the existing steps.
  4. Sign in and move to the pages you want to audit. Work slowly and wait for each page to load.
  5. Select Finish Recording. The recorded steps appear in the form. The recording browser closes on its own after 30 minutes, but the steps recorded so far are kept.
  6. Review the steps, remove any mistakes and add a Scan Page step at each point you want audited.
A recorded User Journey that signs in, then scans the page it reaches. The steps are examples. Simplified.
Text description

Example of a User Journey with 4 steps, each with Move up, Move down, Add Page Scan below and Remove buttons. Step 1: Action Type in Element, Selector #email, Text tester@example.com. Step 2: Action Type in Element, Selector #password, Text hidden as dots, with Sensitive Value ticked. Step 3: Action Click Element, Selector button[type="submit"]. Step 4: Action Scan Page. Below the steps are the Record Journey and Add Step buttons.

You can also build a journey by hand with Add Step. The available actions are Go to URL, Click Element, Type in Element, Select Option, Press Key (Enter/Tab), Scroll Page, Wait for Element, Wait for Time (ms) and Scan Page. Use the arrow buttons, or drag and drop a step, to reorder steps, and Remove to delete one.

Anything typed into a password field while recording is marked Sensitive Value. The value is hidden and never saved, so you enter it again before each scan. If you start a scan without it, the interface shows the step and asks you to enter the value. You can tick Sensitive Value on any other Type in Element step, such as a one-time code or account number.

When a step fails, the scanner tries it up to three times. If it still fails, the journey stops and a full-page screenshot named JOURNEY_FAIL_ followed by a number is saved in the reports folder.

Single sign-on: A Go to URL step can't open a page on a different domain or subdomain from the one you are scanning. If the journey is already on that site, the step is skipped instead of opening the address. If the site sends you to another domain to sign in, reach it by clicking through, as you would in a browser, rather than with a Go to URL step.

4. Browser sign-in pop-ups

Some sites, often staging servers, ask for a username and password in the browser's own pop-up rather than on a web page. For these, leave Login URL empty and put the credentials in the website URL instead, in the form https://username:password@website.com.

5. Keep or clear the session

By default, each scan starts in a fresh browser profile and nothing carries over. Turn on Persistent Session in Advanced Options to keep cookies, local storage and sign-ins between scans, so you sign in once and later scans reuse the session.

The Persistent Session profile is stored on your computer. Once it hasn't been used for 30 days, Octopus Scanner resets it the next time it starts. Clear it as soon as you have your report: in the footer, select Options, turn on Show All Settings, then select Clear Session. This signs the scanner out of every site. You can't clear the session while a scan is running.

See Advanced Options in the User Guide for the full setting.

6. Protect the account

A signed-in scan acts as that user, so plan for what it might do.

  • Use a test account: Sign in with an account made for testing, with no real orders, payment details or personal data. Anything on the page can appear in screenshots and in the report.
  • Keep the crawler away from sign-out links: The crawler follows links, and visiting a sign-out address ends the session. Add the site's sign-out path, such as /logout, to Exclude URL Patterns under Crawl Options.
  • Know what Deep Scan will press: Enable Content Interaction (Deep Scan) selects tabs, accordions and other interactive elements to reveal hidden content.
    • It skips controls whose name suggests they would log out, sign out, delete, remove, cancel, reset, unsubscribe, deactivate, close an account, check out, place an order, buy, pay, submit, send, donate, add to a basket, subscribe, sign up, register, join, book now, apply, download, install, upgrade, invite, transfer, archive, approve, follow or vote.
    • It also skips form submit and reset buttons, and links to the same page whose address mentions signing out or deleting, such as ?logout=1.
    • Tabs and expand or collapse controls are still selected unless their name suggests one of the actions above, apart from cancel, reset and remove, which are allowed for them.
    • Deep Scan isn't available with a User Journey.
  • Avoid destructive steps in journeys: A journey does exactly what you recorded. Don't record a purchase, a deletion or a sign-out unless that is what you mean to test.
  • Check the session lasted: If a page sends the scanner to a sign-in page, the log shows an auth-wall warning and that page isn't scanned. If these warnings start part-way through a scan, the session has probably expired. Sign in again and re-scan.

7. Credentials and privacy

Your sign-in happens in a browser on your own computer, and the session is used only to drive the scan there. Sensitive Values are never written to a saved configuration, an exported configuration or a report.

Other values are treated differently:

  • Saved configurations: Other typed journey values, such as usernames, are saved in plain text so the journey can be replayed. A username and password in the website address are also saved as typed.
  • Exported configurations: Export Current leaves out Sensitive Values, and also leaves out usernames, passwords and sign-in tokens in addresses. When address credentials are left out, the confirmation tells you, so you can enter them again after importing.

Treat any saved configuration that contains a sign-in journey as sensitive. Data, Privacy & Security explains what is stored and where.

8. If something goes wrong

Was this page helpful? Email your feedback.