Auto Recognize and Parse an Unstructured Document - C# .NET 6

This tutorial shows how to automatically recognize a form and retrieve data from an unstructured document in a C# .NET 6 Console application using the LEADTOOLS SDK.

The form is recognized using the AutoFormsEngine paired with the LEADTOOLS Forms Recognition. The data is extracted using a ruleset defined in a JSON file paired with the LEADTOOLS Document Analyzer.

Required Knowledge

Get familiar with the basic steps of creating a project by reviewing the Add References and Set a License tutorial, before working on this tutorial.

Create the Project and Add the LEADTOOLS References

Start with a copy of the project created in the Add References and Set a License tutorial. If you do not have that project, follow the steps in that tutorial to create it.

The references needed depend upon the purpose of the project. References can be added by one or the other of the following two methods (but not both). For this project, the following references are needed:

If using NuGet references, this tutorial requires the following NuGet packages:

• Leadtools.Document.Sdk
• Newtonsoft.Json

If local DLL references are used, the following DLLs are needed. The DLLs are located at <INSTALL_DIR>\LEADTOOLS23\Bin\net:

• Leadtools.Barcode.dll
• Leadtools.Codecs.Fax.dll
• Leadtools.Codecs.Tif.dll
• Leadtools.Codecs.dll
• Leadtools.Document.Analytics.dll
• Leadtools.Document.Pdf.dll
• Leadtools.Document.Writer.dll
• Leadtools.Document.Unstructured.dll
• Leadtools.Document.dll
• Leadtools.Forms.Auto.dll
• Leadtools.Forms.Processing.dll
• Leadtools.Forms.Recognition.dll
• Leadtools.Ocr.LEADEngine.dll
• Leadtools.Ocr.dll
• Leadtools.Pdf.dll
• Leadtools.dll
• Newtonsoft.Json.dll

For a complete list of which DLL files are required for your application, refer to Files to be Included in your Application.

Set the License File

The License unlocks the features needed for the project. It must be set before any toolkit function is called. For details including tutorials for different platforms, refer to Setting a Runtime License.

There are two types of runtime licenses:

• Evaluation license, obtained at the time the evaluation toolkit is downloaded. It allows the toolkit to be evaluated.
• Deployment license. If a Deployment license file and developer key are needed, refer to Obtaining a License.

Initialize AutoFormsEngine, RasterCodecs, IOcrEngine, and DiskMasterFormsRepository

With the project created, the references added, and the license set, coding can begin.

In order to run the application you will need a master forms set and filled forms to recognize and process. Each unstructured master form set will have the following files:

• .bin (Contains the Attributes for Recognition (Structured & Unstructured Forms))
• .tif (Contains a B&W unfilled image of the master form (Structured & Unstructured Forms))
• .dat (Contains the table data information for Processing(Structured Forms))
• .xml (Contains the form field information for Processing (Structured Forms))
• .json (Contains the key/value pairs from unstructured Document Analyzer Processing (Unstructured Forms))

The files used for this tutorial can be downloaded in this link.

In the Solution Explorer, open Program.cs. Add the following statements to the using block at the top:

using System; 
using System.IO; 
using System.Collections.Generic; 
 
using Leadtools; 
using Leadtools.Codecs; 
using Leadtools.Ocr; 
using Leadtools.Forms.Recognition; 
using Leadtools.Forms.Auto; 
using Leadtools.Document; 
using Leadtools.Document.Analytics; 
using Leadtools.Document.Data; 
using Leadtools.Document.Unstructured; 
 
using Newtonsoft.Json; 

Add the following global members:

static AutoFormsEngine autoEngine; 
static RasterCodecs codecs; 
static IOcrEngine ocrEngine; 
static DiskMasterFormsRepository formsRepository; 
static string formToRecognize; 

In Program.cs, create a new method named InitFormsEngine() and add the code below to initialize the AutoFormsEngine, RasterCodes, IOcrEngine, and DiskMasterFormsRepository objects. This method will be called inside the Main method below InitLEAD.

private static void InitFormsEngine() 
{ 
    codecs = new RasterCodecs(); 
 
    ocrEngine = OcrEngineManager.CreateEngine(OcrEngineType.LEAD); 
    ocrEngine.Startup(codecs, null, null, @"C:\LEADTOOLS23\Bin\Common\OcrLEADRuntime"); 
 
    formsRepository = new DiskMasterFormsRepository(codecs, @"FILE PATH TO DIRECTORY WITH MASTER FORMS"); 
    autoEngine = new AutoFormsEngine(formsRepository, ocrEngine, null, AutoFormsRecognitionManager.Default | AutoFormsRecognitionManager.Ocr, 30, 80, true); 
 
    Console.WriteLine("Engines initialized successfully!"); 
} 

Add the Forms Recognition Code

Add a new method named RecognizeForm() to the Program class. This method will be called below InitFormsEngines in the Main method. Add the code below to the new method to recognize a given form.

private static void RecognizeForm() 
{ 
    var json = JsonConvert.SerializeObject(codecs); 
    string resultMessage = "Form not recognized"; 
    formToRecognize = @"FILE PATH TO FILLED FORM"; 
 
    AutoFormsRunResult runResult = autoEngine.Run(formToRecognize, null); 
    if (runResult != null) 
    { 
        FormRecognitionResult recognitionResult = runResult.RecognitionResult.Result; 
        resultMessage = $@"This form has been recognized as a {runResult.RecognitionResult.MasterForm.Name} with {recognitionResult.Confidence}% confidence."; 
    } 
 
    Console.WriteLine("Recognition Results:"); 
    Console.WriteLine(resultMessage); 
    Console.WriteLine("========================================================================="); 
} 

Add the Analyze and Parse Text Code

In the Program class, create a new method named AnalyzeAndParseDocument(). Call this method below the RecognizeForm method in the Main method.

private static void AnalyzeAndParseDocument() 
{ 
    string _dir = Path.GetDirectoryName(formToRecognize); 
    string _fileName = Path.GetFileNameWithoutExtension(formToRecognize); 
    string ruleset = Path.Combine(_dir, _fileName + ".json"); 
 
    LEADDocument document = DocumentFactory.LoadFromFile(formToRecognize, new LoadDocumentOptions()); 
    document.Text.OcrEngine = ocrEngine; 
    // Create Analyzer 
    DocumentAnalyzer analyzer = new DocumentAnalyzer() 
    { 
        Reader = new UnstructuredDataReader(), 
        QueryContext = new FileRepositoryContext(ruleset) 
    }; 
 
    DocumentAnalyzerRunOptions options = new DocumentAnalyzerRunOptions { ElementQuery = new RepositoryQuery() }; 
 
    List<ElementSetResult> results = analyzer.Run(document, options); 
 
    string resultsMessage = string.Empty; 
 
    foreach (ElementSetResult result in results) 
        foreach (ElementResult item in result.Items) 
            Console.WriteLine($"{(item.GetFriendlyName())} = {(item.Value)}"); 
} 

Note

Shipped and installed with the LEADTOOLS SDK are sample master form sets and sample filled forms for recognition and processing. This tutorial uses these samples. The sample files can be found at this file path: <INSTALL_DIR>\LEADTOOLS23\Resources\Images\Forms

Shut Down the OCR Engine

Add the code below in the Main method to properly dispose the resources used in this application.

static void Main(string[] args) 
{ 
    try 
    { 
        InitLEAD(); 
        InitFormsEngine(); 
        RecognizeForm(); 
        AnalyzeAndParseDocument(); 
 
        codecs.Dispose(); 
        autoEngine.Dispose(); 
        if (ocrEngine != null && ocrEngine.IsStarted) 
            ocrEngine.Shutdown(); 
    } 
    catch (Exception ex) 
    { 
        Console.WriteLine(ex.ToString()); 
    } 
    Console.WriteLine("Press any key to exit..."); 
    Console.ReadKey(); 
} 

Run the Project

Run the project by pressing F5, or by selecting Debug -> Start Debugging.

If the steps were followed correctly, the console displays the recognized form as well as the parsed results from the filled form.

For this example, a Healthcare form was used. It was correctly recognized, with a confidence of 91% (where 0 means no confidence and 100% means complete confidence).

• Sample Document: <INSTALL_DIR>\LEADTOOLS23\Resources\Images\Forms\Unstructured\Healthcareform.pdf
• Sample Ruleset JSON: <INSTALL_DIR>\LEADTOOLS23\Resources\Images\Forms\Unstructured\Healthcareform.json

Wrap-up

This tutorial showed how to create a master forms set using the DiskMasterFormsRepository class, and how to recognize and parse a filled form using the AutoFormsEngine and DocumentAnalyzer classes.

See Also

• LEADTOOLS AutoFormsEngine Documentation
• LEADTOOLS DiskMasterFormsRepository Documentation
• LEADTOOLS IOcrEngine Documentation
• LEADTOOLS LEADDocument Documentation
• LEADTOOLS DocumentAnalyzer Documentation
• LEADTOOLS DocumentAnalyzerRunOptions Documentation
• LEADTOOLS SDK Download