Irish Passport Card

Case Study - Managed Testing for Irish Passport Card



The Irish Passport Card Application project allows Irish citizens to submit a passport card request, which facilitates the users to travel within the European Union and the European Economic Area.

The new credit card-sized document will be issued as a supplement to existing passports.

Passport card application is supported on multiple platforms as a native mobile application: Windows, iOS, Android From a testing point of view, the scope of validation is on functional, system, performance and security test types.

Test Tools:

The following tools are used for this project implementation:

Test Management and Bug tracking tools: JIRA for bug tracking and Zephyr for test management

Performance Testing: JAR:Load

Mobile Security: Burp Suite, Acunetix, Nessus, Xcode, iPhone emulators, dex2jar, JD-GUI

Functional and System Integration Testing of Mobile Application

The following test categories are part of the Functional Testing:

  • Application Characteristics (AC) – Information about the application
  • Stability (ST) – Focusing on the application being stable on the device
  • Application Launch (AL) – Once an application is loaded it must start (launch) and stop correctly in relation to the device and other applications on the device.
  • User Interface (UI) – Validating the look and feel of the application's user interface
  • Localization (LO) - Applications that are to be deployed to localities other than their point of origin must account for changes in language, alphabets, date and money formats, etc.
  • Functionality (FN) - Documented features are implemented in the application and work as expected. Sources for the information are user application specification documents.
  • Connectivity (CO) - If an application has communication capabilities then it must demonstrate its ability to communicate over a network correctly. It must be capable of dealing with both network problems and server-side problems.
  • Personal Information Management - The application accessing user information needs to be able to do it in an appropriate manner and not to destroy the information.
  • Retesting - Tests specific to retesting only (Bug fixes)

Test Execution and Approach

Device Models:

It’s a very serious decision to make because the mechanism you choose should correspond to the maximum figure of target customers for you app.

Following factors are considered while selecting the device model:

  • OS Version – Mobile apps should be tested on all major stable OS versions.
  • Screen Resolution – Use a mix of difference screen resolutions to test the mobile app, because the user practice varies on different screen sizes and screen resolutions.

The following are the different models used for this project validation:

 Device  Operating System
 iPhone 6  iOS
 iPhone 6 plus  iOS
 iPhone 5s  iOS
 iPhone 4  iOS
 LG Nexus 5  Android
 Samsung Galaxy S5  Android
 HTC One  Android
 Nokia Lumia 920  Windows phone 8


Physical Devices

Testing on physical devices is imperative to understand the application activities in real-life scenarios.This is a very helpful method of mobile app testing that gave admittance to real-world testers, real devices, actual networks, and wide geographic coverage.

It gave a chance to test passport card mobile app for factors like:

  • How the app behaves on specific devices?
  • How real-world users interact with the app?
  • Different battery states on the devices
  • Multiple networks (Wi-Fi, 4G, 3G, etc.)

Mobile Application - Performance Test 

Performance testing of the mobile application and its supporting infrastructure will be the responsibility of Passport card mobile application.

The Performance Test will confirm that the system and supporting backend infrastructure is designed to handle expected volumes and can be operated to agreed throughput levels. Response times of returned pages from the mobile application will also be monitored.

Resulting applications from this performance test will not be sent to through to the QVW UAT environment.

Performance testing comprised:

Load testing: during the Load test load created by 1,255 concurrent users (mobile) will be emulated. The duration of this test was 8 hours. Its goal was to ensure that the application, servers and infrastructure will be able to handle forecasted volume of applications.

Performance matrix will cover the following areas:

  1. Response Time
  2. Hits Per Second
  3. Throughput
  4. Transaction Per Second (TPS)
  5. Total TPS (TTPS)
  6. Connections Per Second (CPS)
  7. Pages downloads per second
  8. Infrastructure Performance (CPU, RAM, etc.)

Stress testing: The stress test showed how the mobile application, servers and infrastructure handled the higher-than-anticipated load. The number of concurrent users constantly increased until the system maximum capacity was reached (indicated by lack of hardware resources, errors or very high response times).

As part of the stress testing, we executed the following load models to make sure that the system performed under stress:

 S. NO  Scenario  User Load
 1  User load of 200% to original requirement - 1,255  2,510 concurrent users
 2  User load of 300% to original requirement - 1,255  3,765 concurrent users


Test Execution and Approach

The following steps outline the high level approach to performance testing of the backend infrastructure supporting the mobile application:

  1. Develop Performance Test scripts
  2. Server monitoring setup - Monitoring of the Internal MSSQL Server and sync servers and the FTP location was done through the Passport card mobile application. Where external access was not provided, the Passport card mobile application monitored performance testing on location within the DFAT network.
  3. Load generator setup - This will be deployed in the Pre-Production environment and will be available remotely through TCP port for management and monitoring.
  4. Running preliminary testsRunning load test and analysing results
  5. Running stress test and analysing results
  6. Preparing final report

Security Testing (OWASP)

Security Testing is the responsibility DFAT ICT Unit and focuses on the submission of online applications into the DFAT network. Test Triangle performed OWASP security testing on the iPhone app in conjunction with the DFAT security plan.

Security Testing will include card application applications submitted on mobile device apps on the following platforms: iOS

Mobile Application Security Assessment

The following process is based on the OWASP guide lines, and insures all industry compliance standards are met and the leading practices are followed.

Top risks we mitigated during a mobile application security review:

  1. Weak Server Side Controls
  2. Insecure Data Storage
  3. Insufficient Transport Layer Protection
  4. Unintended Data Leakage
  5. Poor Authorization and Authentication
  6. Broken Cryptography
  7. Client Side Injection
  8. Security Decisions Via Untrusted Inputs
  9. Improper Session Handling
  10. Lack of Binary Protections

This was accomplished by a structured approach using a combination of manual and dynamic analysis, auditing authorization processes and device control: Analysis of device configuration, and auditing of device management systems and developing strong mobile security stands.

All projects followed the established project stage design.

Testing Method Stages:

  • Information Gathering - describes the steps to consider during the early stage reconnaissance and mapping phases of testing, as well as determining the application’s magnitude of effort and scoping
  • Static Analysis - analysing raw mobile source code, decompiled or disassembled code
  • Dynamic Analysis - executing an application either on the device itself or within a simulator/emulator and interacting with the remote services with which the application communicates. This includes assessing the application’s local inter-process communication surface, forensic analysis of the local file system, and assessing remote service dependencies.


 Mobile security: Burp Suite, Acunetix, Nessus, Xcode, iPhone emulators, dex2jar, JD-GUI


We summarised our findings to customer upon completion of the fieldwork.

Our key deliverables from this project was a report setting out:

  • Introductory section with details of scope and work completed
  • Detailed technical summary of findings identified including recommendations for corrective action


To Download Irish Passport Card - Managed Testing Case Study, Click Here.