Handwritten Signature Extraction and Matching

Signature extraction and matching is a vital tool in preventing fraud, verifying identity, ensuring legal compliance, and improving efficiency in various industries. For example:

1. Financial Services: Handwritten signature extraction and matching can be used by financial institutions to verify signatures on checks, loan documents, and other financial transactions. This helps prevent fraud and ensures that the signature belongs to the authorized person.
2. Government: Government agencies use signature extraction and matching to verify the authenticity of signatures on legal documents, passports, and other government-issued identification documents.
3. Healthcare: In the healthcare industry, handwritten signature extraction and matching can be used to verify patient signatures on consent forms and other medical documents.
4. Legal: In the legal industry, signature extraction and matching can be used to verify signatures on legal documents, such as contracts and agreements.
5. Education: Signature extraction and matching can be used by educational institutions to verify the identity of students on examination answer sheets, registration forms, and other documents.

Signature extraction is a process of detecting handwritten signatures, locating its positions, and cropping the signature portions from a document. The location coordinates are used to crop the sections of the document to extract signatures. The following figures (1, 2 and 3) illustrated the signature extractions.

Figure 1: Signatures extracted from the 1040 Tax form

Figure 2: Signature extracted from the Virginia Driver’s License

Figure 3: Signature extracted from the Indian PAN card

Signature matching is a process of comparing two or more signatures and obtaining a their match scores. Figure 4 shows the signature matching process.

Figure 4: Signature matching with score. Keypoints are used to align the orientations of the two images.

Typically, the signature matching is a two step process:

Step 1: Signatures are first extracted from two or more scanned documents. The output from this extraction is typically cropped portions of signatures only.

Step 2: The cropped signatures are compared and match score is obtained.

However, in the case when signatures are already extracted by other means and cropped images are available, step 1 may be skipped and only step2 is needed to perform the comparison and matching.

In this article, we explore how to extract signatures from scanned documents and then perform matching using Restful APIs. To use the APIs, you need to have an account and security token. Here are the steps to do so:

1. Sign up for an account at https://automate.accure.ai/register/
2. Login to https://automate.accure.ai/ using your username and password.
3. From the main menu panel (left side panel), click “APIs”. You may have to scroll down to see the link to “APIs”.
4. Key in your account password (the same password you use to login to your account), and click (do not hit enter key) “Reveal Key”.
5. API key will show up. Copy the key. You will need this key to perform the signature extraction and matching. Figure 5 shows a screenshot of the process of obtaining the API key.

Figure 5: Screenshot showing how to obtain security token or API key

API Specifications

Service Endpoint: https://automate.accure.ai/

Signature Extraction

Service Name: /v1/sign/extract

Curl Example:

curl https://automate.accure.ai/v1/sign/extract -H 'Authorization: Token 123456765432123456' -F "image=@/home/user1/Desktop/test1.png"

In the above curl command, “image” is the path to the scanned document. Replace the token with the security token you obtained above (Figure 5).

Response Example:

{
  'image':'1040EZForm.png',
  'detections': [{
  'x1': 2303.0,
  'y1': 457.0,
  'x2': 2671.0,
  'y2': 536.0,
  'confidence': 0.813737333,
  'class_id': 1.0
}],
  'transaction_id': '1676846795913847019'
}

The response contains a list of detections containing the coordinates of the rectangular regions enclosing either signatures or logos.

(x1, y1) represent the coordinates of the top left corner and (x2,y2) represent the coordinates of the bottom left of the rectangular region containing the signature within the document.

confidence: is the combined probability of detection and localization of the signature.

class_id: 0 means logo and 1 means signature.

transaction_id: a unique transaction id representing the unique API call.

Signature Matching

Service Name: /v1/sign/match

Curl Example:

curl https://automate.accure.ai/v1/sign/match -H 'Authorization: Token 123456765432123456'
-F "image1=@/home/user1/Desktop/test1.jpg" -F "image2=@/home/user1/Desktop/test2.jpg"

Response Example

{
  'transaction_id':'8989891',
  'image1':'test.jpg',
  'image2':'test2.jpg',
  'match_score':0.8,
  'status_code': 200
}

The response contains a match_score that ranges between 0 and 1. The score 1 means a perfect match and a 0 means no match.

What is considered a good match?

Depending on the level of noise in the signature image, the match_score varies, and it is up to the consumer of the API to decide where to put the threshold for a good match.

In the most common scenario, a match_score > 0.7 is considered a good match. The higher the score, the better the match is.

Status Codes

200: Ok

401: Authentication failed

415: Invalid file format

405: Method Not Allowed

500: Failed. Request was not processed

Conclusion

The two APIs discussed above provide easy access to pre-trained computer vision based neural networks to perform signature extraction and matching. These APIs are available for free for any testing and evaluation. To learn more about the signature extraction and matching APIs, their usage, and more, visit this website https://accure.ai/signature-matching.

Although the neural network models are trained and regularly updated on a wide variety of documents, it is likely that the signature extraction will fail in document types the model has never seen before. Also, the scanning quality, image clarity, and several other factors influence the accuracy of such extraction.