In an effort to provide clarification and a better understanding regarding the ISO Presentation Attack Detection testing that iBeta conducts in accordance with  the ISO/IEC 30107-3 standard and in alignment with the ISO/IEC 30107-1 framework, we provide the following explanation.

Confirmation Letters

iBeta posts, by written vendor consent, the Presentation Attack Detection (PAD) Confirmation letters that provide the results of the iBeta PAD testing.  These letters specify the type of PAD testing and the configuration of the vendor product that was tested.  In order to fully assess the vendor product, the audience of these letters should understand the test method used to assess conformance with the ISO 30107-3 Standard.

Understanding the Test Method

iBeta is accredited by NIST NVLAP as an Independent Test Lab and ISO 30107-3 is included in our scope of accreditation.   As such, the specific test procedures, processes, and report templates, as developed by iBeta, are audited and approved as part of the NIST administration of the NVLAP.

Prior to test start, the PAD mechanism is determined:

  • PAD subsystem test – evaluates presentation attack detection only

  • Data capture test – evaluates the coupled presentation attack and quality checks

  • Full system test – evaluates the biometric comparison capabilities of the full biometric subsystem

The type of PAD mechanism determines the ISO/IEC 30107-3 mandated reporting metrics for the evaluation.  For subsystem PAD evaluations, the classification error rates (APCER, BPCER and associated non-response rates) are determined whereas when evaluating full systems, the imposter attack presentation match rates (IAPMR and associated FNMR/FMR) is determined.

In order to specify more exactly  the test method, iBeta identified  levels of testing (although similar to the FIDO Alliance Levels A & B, there are significant differences).  The testing levels and performance requirements are identified as:

Level Time Expertise Artefact source Limit
1 8 hours per subject or species None Cooperative subject and equipment is readily available in a normal home or office environment 0% penetration or match rate allowed
2 2-4 days per subject or species Moderate – participated in at least 1 other PAD test with the target modality and has an understanding of the liveness detection functionality of the test target Cooperative subject and equipment is more expensive (such as a 3D printer, resin mask, latex mask) 1% penetration or match rate allowed

In order to maintain uniformity between vendor solutions and modalities and to apply the test methods to each vendor consistently, the 6 species of attacks (called “PAIS” in ISO 30107-3 ) are selected as uniformly as  possible.  If the application uses an active liveness detector to elicit a “voluntary response” (the ISO 30107-1 term), then the species of attacks are tailored just as an impostor would tailor them to provide the movement, smile, blink, etc.  Our procedure sets a material cost limit  such that Level 1 artefacts cannot exceed $30 and Level 2 are limited to $300.  This keeps the artefacts creation or procurement costs generally consistent across all vendor solutions.

In addition, each species set of artefacts is created and applied to the PAD within a time limit.   For Level 1, if liveness detection only is being assessed, then iBeta will create and apply each of the species artefacts within 8 hours, targeting to present 150 attacks alternated with 50 genuine presentations.  If a full system is under evaluation, then the artefacts associated with the species for a single genuine subject are created and applied within 8 hours.  With 6 subjects and/or 6 species, the testing therefore requires 48 hours.

ISO 30107-3 discusses cooperative versus uncooperative subjects.  iBeta uses cooperative subjects in that the artefacts are created from biometric characteristics provided by volunteer data subjects (if not purchased) who are willing and able to pose for photos, record videos, provide their fingerprints in molding material or sit for a live cast.  We only use cooperative subjects as the artefacts created from  willing volunteers are of better quality making for a more conservative test.  iBeta is evaluating the vendor solution and not the ability of our testers to obtain latent prints, as an example.

Prior to applying any presentation attack, the configuration of the vendor solution is recorded and the version is referenced  in the confirmation letter.  In addition, because the device used to capture the biometric sample impacts the results, the exact device configuration is recorded and also referenced in the confirmation letter.

During the test effort, bona fide biometric presentations to the PAD device not only provide the  reporting metrics  required by ISO 30107-3 but also provide the indication of usability by the data subjects.  For this reason, iBeta is now limiting the BPCER and FNMR to 15% to obtain a “PASS” rating.  If a bona fide data subject cannot be recognized as live and/or matching to the enrolment reference, iBeta will suspend testing as a high BPCER or FNMR biases the test result.  These error rates were originally unconstrained but have been limited to 20% and now recently reduced to 15%.  This limit is reviewed every 6 months and is subject to further reduction as PAD technology improves.

The iBeta testing and reporting provides results that indicate that the solution testing is compliant or conforms with the ISO 30107-3 testing and reporting requirements. This itself, however, does not translate to a certification of the vendor product.

Certification vs. Conformance

When iBeta was initially accredited, the term certification was used but was corrected during the NIST/NVLAP audit in March 2019.  There is no difference in our test methods, procedures, or processes between the earlier testing and the testing after that date with the exception that the allowed limit for genuine or Bona Fide presentations is now stricter.

Date of LetterVendorPAD LevelProduct testedVersion(s) testedFNMR or BPCER limitPDF Link
09/29/21NEC JapanLevel 2NECv1.815%PDF
9/20/21Transmission y Vision Digital, S.A. de C.V.Level 1Dicio Liveness2.1.1115%PDF
08/05/21ImagewareLevel 2Biointellec Intelligent Liveness AssuranceAndroid client 9.0.3.0 and SDK 9.0.3
iOS client 1.12 and SDK 1.4.1
15%PDF
07/26/21CUBOXLevel 1CUFaceSDK1.8.015%PDF
06/25/21HyperVergeLevel 1HyperVerge Onboard
HyperVerge Liveness
2.1.6
1.9.6
15%PDF
06/15/21OnfidoLevel 1Onfido Selfie

Anti-spoofing Service
9.3.1 (Android)
21.4.0 (IOS)
Version 14
15%PDF
04/26/21InnovatricsLevel 2Digital Onboarding Toolkit4.0 Android
2.2 iOS
15%PDF
04/16/21NEC Asia PacificLevel 2Luma2.1.44 iOS
Luma 7.E.4
15%PDF
04/13/21EyeVerifyLevel 2EyeVerify2.0 (20)15%PDF
04/12/21EyeVerifyLevel 2EyeVerify2.0 (29)15%PDF
03/29/21VU SecurityLevel 1Secure Onboarding Processv1.2.5.1 Android
1.29.0 server
15%PDF
03/23/21EyeVerifyLevel 1EyeVerify2.0 (28)15%PDF
03/12/21DaonLevel 1DaonFace5.1.0.7 iOS
5.1.0 (26) Android
15%PDF
03/02/21EyeVerifyLevel 1EyeVerify2.0 (16)15%PDF
02/25/21Oz ForensicsLevel 1LivenessSDK 2.0.0 iOS
SDK 4.4.10 Android
Server ML models 0039
15%PDF
01/07/21RaizomatLevel 1IdentomatRelease 1215%PDF
01/06/21ThalesLevel 2DactyID20220015%PDF
01/04/21NEC Asia PacificLevel 1Luma2.1.44 iOS
2.1.43 Android
Luma 7.E.4
15%PDF
12/18/20ScanovateLevel 1Liveness1.7.015%PDF
12/17/20VeridasLevel 1das-Facev2.6.5
SDK v3.0.0 iOS
SDK v3.2.0 Android
15%PDF
11/24/20NEC JapanLevel 1NECv1.015%PDF
10/19/20IDMissionLevel 2Idmission7.3.4.620%PDF
10/01/20iProovLevel 2Liveness Asurancev7.0.020%PDF
09/30/20ID R&DLevel 2IDLive1.16.015%PDF
09/16/20InnovatricsLevel 1Digital Onboarding Toolkit3.0.0 Android
2.1.0(1) iOS
20%PDF
09/16/20iProovLevel 1Liveness Asurancev7.0.020%PDF
08/26/20ThalesLevel 1DactyID202.1.0.020%PDF
08/18/20iProovLevel 2iProov SDKApp 7.4.1(219)
SDK 7.5.0 (1)
20%PDF
08/17/20AwareLevel 2FaceLiveness2.7.1.1 iOS
2.7.1 Android
15%PDF
07/27/20AwareLevel 1FaceLiveness2.7.1.1 iOS
2.6.2 Android
20%PDF
07/24/20iProovLevel 1iProov SDKApp 7.4.1(219)
SDK 7.5.0 (1)
20%PDF
06/30/20OCR LabsLevel 2Digital Identification Processv1.0NonePDF
06/03/20IDMissionLevel 1Idmission7.2.2.420%PDF
05/22/20Idemia - PassiveLevel 2Web CaptureSDK V 3.10.020%PDF
04/30/20Idemia - PassiveLevel 1Web CaptureSDK V 3.10.020%PDF
04/08/20ImageWareLevel 1GoVerifyID7.5.81 iOS
7.5.20.3 Android
GoMobile 2.2.9.000
NonePDF
02/27/20ZolozLevel 2ZOLOZSDK v18.0 iOSNonePDF
12/31/19ID R&DLevel 1IDLive1.9.0NonePDF
12/27/19Idemia - ActiveLevel 2SmartBioSdkSDK V 4.20.0 (2.37)20%PDF
12/13/19HID GlobalLevel 2M-SeriesSDK 0.2020%PDF
12/10/19Idemia - ActiveLevel 1SmartBioSdkSDK V 4.20.0 (2.37)NonePDF
09/03/19OCR LabsLevel 1Digital Identification Processv1.0NonePDF
08/23/19IncodeLevel 1Omni Face highfivev1.0NonePDF
07/17/19RedrockLevel 1PalmID Loginv3.0.0.8NonePDF
05/22/19HID GlobalLevel 1M-Seriesv 7.00.00 driver
SDK v 6.01.26
20%PDF
04/18/19ZolozLevel 1ZOLOZv4.6NonePDF
02/07/19FaceTecLevel 2ZoOmv6.9.11NonePDF
09/19/18HID GlobalLevel 1V-Seriesv 7.00.00 driver
SDK v 6.01.26
NonePDF
08/20/18FaceTecLevel 1ZoOmv6.6.0NonePDF