This tutorial shows how to set up AWS Lambda for use with the LEADTOOLS SDK and convert a document with .NET Core.
Overview | |
---|---|
Summary | This tutorial covers how to set up AWS Lambda for use with the LEADTOOLS SDK and convert a document with .NET Core. |
Completion Time | 60 minutes |
Visual Studio Project | Download tutorial project (774 KB) |
Platform | AWS Lambda .NET Core Application |
IDE | Visual Studio 2017, 2019, AWS Lambda Visual Studio Extension |
Runtime License | Download LEADTOOLS |
Get familiar with the basic steps of creating a project by reviewing the Add References and Set a License tutorial, before working on the Convert Documents Using AWS Lambda - C# .NET Core tutorial.
To set up the development environment to use AWS in Visual studio, complete the following 2 tutorials from Amazon:
After these tutorials are completed, the AWS Toolkit for Visual Studio should be installed and a basic understanding of how to create a new AWS Lambda Project for .NET Core and publish to AWS should be acquired.
In Visual Studio, create a new AWS Lambda Project (.NET Core - C#) Project. Give the project a name and location and click Create.
Select an Empty Function and click OK.
The references needed depend upon the purpose of the project. For this project, the following NuGet Package is needed:
Leadtools.Document.Sdk
Right-click the C# project in the Solution Explorer and select Manage NuGet Packages....
Browse for LEADTOOLS, then select the Leadtools.Document.Sdk
NuGet package and install it. Accept LEAD's End User License Agreement.
For a complete list of which Codec DLLs are required for specific formats, refer to File Format Support.
Right-click the C# project and select Add, then Class. Name it LEADRequest.cs
and click Add.
Add the following to the using
block:
using Leadtools.Document.Writer;
Copy the code below to add the needed functionality to the class:
public class LEADRequest
{
public string InputUrl { get; set; }
public DocumentFormat DocumentFormat { get; set; }
public LEADRequest()
{
}
}
Right-click the C# project and select Add, then Class. Name it LEADLambdaHandler.cs
and click Add.
In the new class, add the following to the using
block:
using System;
using System.Diagnostics;
using System.IO;
using System.Net.Http;
using Amazon.Lambda.Core;
using Leadtools;
using Leadtools.Document;
using Leadtools.Document.Converter;
using Leadtools.Document.Writer;
using Leadtools.Ocr;
Copy the code below to add the needed functionality to the class:
public class LEADLambdaHandler
{
// Global Variables
IOcrEngine ocrEngine;
DocumentConverter documentConverter;
static HttpClient httpClient = new HttpClient();
// Constructor which handles all initialization to make sure the function is as fast as possible once it's warmed up
public LEADLambdaHandler()
{
InitEnvironment();
Platform.LibraryPath = "/opt/native-runtimes/";
InitLEADTOOLS();
}
// Method which initializes the Lambda environment for use by the LEADTOOLS SDK
void InitEnvironment()
{
// Set the LD_LIBRARY_PATH in function console to include:
// /var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib:/tmp
ExecuteBashCommand("ln -s /lib64/libdl.so.2 /tmp/libdl.so");
}
// Initialize the LEADTOOLS SDK Classes
void InitLEADTOOLS()
{
SetLicense();
RasterDefaults.TemporaryDirectory = "/tmp";
RasterDefaults.SetResourceDirectory(LEADResourceDirectory.Fonts, "/opt/ShadowFonts");
ocrEngine = OcrEngineManager.CreateEngine(OcrEngineType.LEAD);
ocrEngine.Startup(null, null, "/tmp", "/opt/OcrLEADRuntime");
documentConverter = new DocumentConverter();
documentConverter.SetOcrEngineInstance(ocrEngine, true);
}
// Helper method for executing shell scripts in the Lambda environment
string ExecuteBashCommand(string command)
{
command = command.Replace("\"", "\"\"");
var proc = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = "/bin/bash",
Arguments = "-c \"" + command + "\"",
UseShellExecute = false,
RedirectStandardOutput = true,
CreateNoWindow = true
}
};
proc.Start();
proc.WaitForExit();
return proc.StandardOutput.ReadToEnd();
}
// Set license code TODO: Replace the licString and developerKey with a valid license and key
void SetLicense()
{
string licString = "[License]\n" + "License = <doc><ver>2.0</ver>`ADD LICENSE HERE`</doc>";
string developerKey = "ADD DEVELOPMENT KEY HERE";
byte[] licBytes = System.Text.Encoding.UTF8.GetBytes(licString);
RasterSupport.SetLicense(licBytes, developerKey);
}
// Main conversion function
public string ConvertDocument(LEADRequest request, StringWriter sw)
{
var isUrl = Uri.IsWellFormedUriString(request.InputUrl, UriKind.RelativeOrAbsolute);
if (isUrl)
{
var response = httpClient.GetAsync(request.InputUrl).Result;
if (response.IsSuccessStatusCode)
{
var stream = response.Content.ReadAsStreamAsync().Result;
using (var document = DocumentFactory.LoadFromStream(stream, new LoadDocumentOptions()))
{
string ext = DocumentWriter.GetFormatFileExtension(request.DocumentFormat);
string fileName = Path.Combine("/tmp", Path.ChangeExtension(Path.GetFileName(request.InputUrl), ext));
DocumentConverterJobData jobData = DocumentConverterJobs.CreateJobData(document, fileName, request.DocumentFormat);
DocumentConverterJob job = documentConverter.Jobs.CreateJob(jobData);
documentConverter.Jobs.RunJob(job);
if (job.Errors.Count > 0)
foreach (var error in job.Errors)
sw.WriteLine($"Error during conversion: {error.Error.Message} {error.Error.StackTrace}");
else
return fileName;
}
}
else
sw.WriteLine("Download of URL is not successful");
}
else
sw.WriteLine("Url is invalid.");
return null;
}
}
The License unlocks the features needed for the project. It must be set before any toolkit function is called. For details, including tutorials for different platforms, refer to Setting a Runtime License.
There are two types of runtime licenses:
Note
Adding LEADTOOLS NuGet and local references and setting a license are covered in more detail in the Add References and Set a License tutorial.
When an AWS Lambda function is triggered, the FunctionHandler
method is called. Everything in the Function
class that is global will be called before that when the function is first starting up. While the container is warm, it will only call the FunctionHandler
method until the execution context changes.
In order to leverage this functionality, most of the initialization code for the LEADLambdaHandler
is done in the constructor to keep each function call as fast as possible while the container is warm.
Open the Function.cs
file that was included as part of the project and add the following to the using
block under the rest of the using statements.
using System.IO;
Add the following global variable to the Function
class:
public LEADLambdaHandler LEADHandler = new LEADLambdaHandler();
Replace the FunctionHandler
method with the one below to accept a LEADRequest
as the input instead of a string input, and then call the LEADHandler.ConvertDocument
method:
public string FunctionHandler(LEADRequest request, ILambdaContext context)
{
StringWriter sw = new StringWriter();
try
{
string outputFile = LEADHandler.ConvertDocument(request, sw);
if (outputFile != null)
sw.WriteLine($"Successfully saved to {outputFile}.");
else
sw.WriteLine("Error occurred. Output file not saved.");
}
catch (Exception ex)
{
sw.WriteLine(ex.Message);
sw.WriteLine(ex.StackTrace);
if (ex.InnerException != null)
{
sw.WriteLine(ex.InnerException.Message);
sw.WriteLine(ex.InnerException.StackTrace);
}
}
return sw.ToString();
}
Once all the code is added, build the project to ensure everything is working as intended.
Note
This step can be skipped if desired. The ZIP included in this project contains the required Lambda Dependencies already.
The LEADTOOLS SDK requires certain dependencies to be installed on a Linux machine in order to function properly. For a full list, see Getting Started with the LEADTOOLS Linux Libraries and Demo Projects.
AWS Lambda offers a Docker image that contains the same Linux distro to test functionality before deploying to Lambda. This can also be used to install the needed dependencies and create a deployment package. This requires Docker Desktop to be installed.
docker pull amazonlinux
C:\temp\docker
docker run -it -v C:\temp\docker:/var/task amazonlinux:latest
rpm -Uvh https://packages.microsoft.com/config/centos/7/packages-microsoft-prod.rpm
yum -y install dotnet-sdk-3.1
yum -y install uuid-devel uuidd gcc gcc-c++ glibc-devel kernel-devel kernel-headers libX11 libXt libX11-devel libXt-devel sqlite sqlite-devel freetype fontconfig
csproj
dotnet publish
<PROJECT-DIR>\bin\Debug\netcoreapp3.1\publish\runtimes\linux-x64\nativeassets\netcoreapp
to the C:\temp\docker
C:\temp\docker
directory, create a new text file and copy and paste the following shell script and save it as CopyDeps.sh
:indir="/var/task/"
outdir="/var/task/lib/"
ldcache=$(ldconfig -p)
mkdir -p $outdir
for i in $(ls ${indir}/*.so); do
deps=$(ldd $i | grep "=>" | grep -v "liblt")
while IFS= read -r line; do
result=""
list=( $line )
dep=$(echo $ldcache | grep "${list[0]}")
if [ ! -z "$dep" ] ; then
cp -vL "${list[2]}" $outdir 2> /dev/null
fi
done <<< "$deps"
done
sh /var/task/CopyDeps.sh
C:\temp\docker\lib
and this will contain all the dependenciesliddl.so.2
libexpat.so.1
libz.so.1
The LEADTOOLS SDK requires environment dependencies to be installed in the Lambda function. In order to keep packages small and flexible, Lambda Layers are used. For more information on what a Layer is, refer to AWS Lambda Layers.
The LEADTOOLS SDK OCR Engine requires dependencies to be included with any OCR deployment. More information on what is required can be found at LEADTOOLS OCR Module - LEAD Engine Runtime Files.
Since the AWS Lambda environment is needed, Shadow Fonts will also be required. More information on the Shadow Fonts can be found at LEADTOOLS Drawing Engine and Multi-Platform Consideration.
lead-deps-layer
lib
folder from C:\temp\docker\lib
to the lead-deps-layer
lead-deps-layer
named native-runtimes
csproj
dotnet publish
<PROJECT-DIR>\bin\Debug\netcoreapp3.1\publish\runtimes\linux-x64\nativeassets\netcoreapp\
.so
files to the lead-deps-layer\native-runtimes
folderC:\temp\docker\lib
folder to the lead-deps-layer
ShadowFonts
folder from <INSTALL_DIR>\LEADTOOLS21\Bin\Common\ShadowFonts
to the lead-deps-layer
OcrLEADRuntime
folder from <INSTALL_DIR>\LEADTOOLS21\Bin\Common\OcrLEADRuntime
to the lead-deps-layer
lead-deps-layer
: lib
, native-runtimes
, ShadowFonts
, and OcrLEADRuntime
Zip the contents of the lead-deps-layer
folder. Ensure that the zip lead-deps-layer.zip
does not contain a subfolder of the same name and that instead, the zip structure looks like the following screenshot:
Once the layer zip is created, it needs to be uploaded to AWS Lambda.
lead-deps-layer
lead-deps-layer
lead-deps-layer.zip
Once the layer is completed, the function can now be published to AWS.
This will run dotnet publish
on the project and then zip up the files and dependencies and upload it to Lambda.
The package created in the previous step will be large since it includes a lot of the dependencies that were added to the layers. In order to reduce the size, the zip package will need to be modified and manually uploaded to Lambda.
<PROJECT-DIR>\bin\Release\netcoreapp3.1\publish
directoryLEADTOOLS
folder which contains the OCRRuntimes that were added to the lead-deps-layer
<function-name>.zip
overwriting the one that was created automatically<function-name>
to open the function<PROJECT-DIR>\bin\Release\netcoreapp3.1\publish\<function-name>.zip
, hit OK, then click Save in the top rightThis will update the function package so that it is as small as possible.
Once the package is uploaded, the LD_LIBRARY_PATH
environment variable of the function needs to be updated in order to properly load the dependencies.
LD_LIBRARY_PATH
and for the value input /var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib:/tmp
Click Save to save these changes
Once the environment variables are set, the layers need to be added to the function.
lead-deps-layer
Click Add to add this layer to the function
Ensure that the memory of the function is set to 2048 MB:
2048 MB
Click Save to save the changes
Once the previous step is completed, the function is ready to be tested.
hello-world
Event Template{
"InputUrl": "https://demo.leadtools.com/images/tiff/ocr1.tif",
"DocumentFormat": 1
}
Click Create to create the test event
If everything was setup correctly, the Execution results should return succeeded
and look something like:
This tutorial showed how to create a new AWS Lambda function, gather the needed dependencies, and publish it to AWS.