Using AWS Lambda and Event-Driven Architectures for Libraries

AWS Lambda can speed up routine library processes

Posted on May 29, 2023 in Blog Posts

In our previous blog post, we introduced a basic serverless application that includes a single AWS Lambda function. This function processes events and generates respective responses, exemplifying an event-driven architecture, a popular software design pattern in serverless applications. 

In this article, we will delve deeper into AWS Lambda and event-driven architecture. We’ll discuss design considerations, benefits, trade-offs, and potential applications specifically tailored to library settings. We will also introduce additional AWS-managed services. Typically in real-world serverless applications, multiple specialized services all communicate with one another to accomplish a given task.

---

🌟 Catch up on the serverless series:

• Part 1: “Introduction to Serverless”
• Part 2: “Introduction to Function as a Service (FaaS) and AWS Lambda”

---

Key characteristics of AWS Lambda functions

AWS Lambda, a computer service managed by Amazon Web Services (AWS), is primarily designed to process events. If you examine the sample Lambda function code from the previous blog post, you’ll notice that the first argument of a Lambda function is called an “event.” This event object is an immutable, JSON-formatted document containing data. Almost all AWS services can generate events and pass them to AWS Lambda. Moreover, events can originate from third-party applications or services outside of the AWS ecosystem.

Upon receiving events, AWS Lambda prepares the necessary computing environment for underlying Lambda functions to execute your code and process these events. The service auto-scales based on demand, invoking Lambda functions only when necessary. This strategy results in cost-efficient resource utilization. However, because Lambda functions are triggered only as needed and require time to prepare their execution environment, they have an inherent latency known as a “cold start.” Further, to utilize the resource only when needed, a Lambda function will terminate after it completes its task or reaches its maximum execution time limit of 15 minutes.

The design nature of Lambda functions is such that they are best suited for processing simple tasks within a short execution time. They are widely used for rapid processing tasks, with each function designed to handle the event passed onto it, free from assumptions about the overall workflow or volume of transactions. Importantly, Lambda functions are stateless: the environment exists solely for a single invocation. 

AWS Lambda thus facilitates the development of serverless applications within an event-driven architecture. In this architecture, serverless applications are designed to react to events as they occur. When an event is triggered, it invokes the corresponding AWS Lambda function to handle it. Each function performs one task and one task only in response to these events before passing the result onto the next target client.

Key characteristics of Event-Driven Architecture

Event-driven architecture (EDA) is a design pattern used to build serverless applications that respond to specific events, such as file uploads to cloud storage or new entries in a database. In this architecture, services either produce or listen to events. Event producers create events that event listeners detect and process, performing actions such as updating a database record or triggering AWS Lambda functions to run your custom code.

EDA is characterized by its decoupled services that communicate asynchronously through event messages. This design pattern enables each service to process events independently and at varied speeds, enhancing scalability, extensibility, and throughput. The inherent decoupling in EDA simplifies system complexity, reduces the amount of code, increases system flexibility, and promotes the design of near-real-time systems. AWS Lambda functions often act as orchestrators in this architecture, interacting with a variety of other services.

However, EDA does come with certain trade-offs. Due to the interactions between multiple components, the application may experience variable latency, and managing asynchronous event messages can be challenging. Consequently, EDA might not be optimal for applications that require consistently low-latency performance, such as e-commerce transaction services. Most importantly, each service within EDA is designed to perform a single task efficiently rather than handle a complex, time-consuming operation all at once. These characteristics are key considerations when deciding whether to implement EDA in your specific use case.

Library use cases

Let’s take an example of an event-driven service tailored for a digital library collection use case. Suppose you curate a set of raw images from digital scanning. Each image is a large file size, and you’d like to create a thumbnail for each one to display more efficiently on the website. You need an automated thumbnail creation service to create a thumbnail as soon as the file is uploaded to the S3 bucket. The architecture design of this service, using an event-driven architecture with Lambda, is outlined below:

When files such as images, PDFs, or video files are uploaded into an S3 bucket, they trigger an AWS Lambda function. This function contains the code to create a thumbnail file and save it to another S3 bucket. The entire application is serverless and managed by AWS, eliminating the need for you to manage any underlying servers.

Other use cases that can be implemented with EDA include automated metadata indexing, document translation, file conversion, and more. 

Final remarks

In this blog post, we examined the key characteristics of AWS Lambda functions and event-driven architecture (EDA), demonstrating their role in building efficient, scalable serverless applications. A practical library use case was also demonstrated. In our upcoming blog post, we’ll guide you through how to select well-established, common patterns from the AWS serverless community, helping you find and implement the right pattern for your specific use case. We’ll also introduce some use cases we have implemented and present related architectural designs. 

If you’re interested in understanding whether your existing library services could transition to an event-driven serverless architecture application, don’t hesitate to send me an email. I look forward to discussing the possibilities with you.

---

🔥 Sign up for LibTech Insights (LTI) new post notifications and updates.

✍️ Interested in contributing to LTI? Send an email to Deb V. at Choice with your topic idea.

Related Posts