File Automation for Fintech
How SFTP Gateway Powers Automated Data Processing for Global Finance
// About the customer
File Automation for Fintech + SFTP Gateway
A fintech company helping global enterprises detect and prevent payment errors through AI-powered spend analytics needed a bulletproof automated file ingestion system. Their platform processes uploaded client data files through AWS Lambda functions to deliver real-time financial insights—making file transfer reliability mission-critical to their entire service.
Operating across Europe (~120 users) and North America (~40 users), they required infrastructure that could meet enterprise SLA commitments while seamlessly integrating SFTP uploads with cloud-native AWS processing workflows. Thorn Technologies implemented SFTP Gateway to create a fully automated pipeline from client uploads to Lambda-triggered analytics, enabling the company to document uptime guarantees and scale its automated processing across multiple regions without manual intervention.
// problem
The problem
This innovative fintech platform relies entirely on automated file workflows: clients upload data via SFTP, files are stored in S3, and Lambda functions immediately process them to deliver payment error detection and spend analytics. Any interruption in this pipeline breaks client workflows and delays critical financial insights.
However, before upgrading their infrastructure with Thorn Technologies, they faced several challenges that threatened their ability to meet enterprise client expectations.
01
Single Point of Failure
Each region ran on a single EC2 instance with no failover capability. If an instance went down, client uploads failed, and automated processing stopped completely. Enterprise customers increasingly demanded documented Recovery Time Objectives (RTO) and high-availability architectures that the single-instance setup couldn’t provide.
02
SLA Documentation Requirements
While their existing system worked reliably in practice, enterprise clients required formal uptime guarantees and documented HA capabilities on paper. Multiple prospects requested concrete SLA commitments and architecture diagrams showing redundancy—requirements that the company couldn’t meet with their current infrastructure.
03
Local Disk Dependency
Files uploaded via SFTP were temporarily stored on the local disk before cron jobs synced them to S3. This created a data loss risk if the instance failed before the sync was completed, and introduced processing delays between upload and Lambda execution that could impact time-sensitive financial analytics.
04
Scalability Constraints
As user counts grew from dozens to over 160 across two regions, the company required infrastructure that could scale horizontally, utilizing load balancers and auto-scaling groups. Their aging architecture wasn’t designed for the cloud-native scaling patterns modern enterprise platforms require.
Their existing system was approaching end-of-life, with no further security patches or feature updates available. The company needed to modernize to a supported platform that could meet both current SLA requirements and future compliance needs as it expanded into regulated industries.
Recognizing that infrastructure limitations were constraining growth and preventing them from closing enterprise deals, they sought a partner who could deliver documented high availability while preserving their seamless automated workflows.
// Solution
The Solution
Thorn Technologies worked closely with the fintech company to implement a comprehensive solution that transformed their file ingestion infrastructure while maintaining the automated processing workflows their business depends on:
Direct-to-S3 Streaming Architecture
SFTP Gateway streams uploaded files in-memory directly to Amazon S3, eliminating the need for local disk storage and avoiding sync delays. Files become immediately available to downstream Lambda functions the moment the upload completes, eliminating the gap between client upload and processing execution while reducing the risk of data loss from local disk failures.
HA-Ready Cloud-Native Design
The stateless architecture supports load balancing, auto-scaling groups, and multi-instance configurations that meet the demands of enterprise clients. The company can now document concrete RTO commitments and high availability capabilities, meeting client SLA requirements that were previously unattainable.
Preserved Automated Workflows
The solution maintained the company’s critical /uploads folder convention and Lambda processing triggers with zero disruption to client workflows. Files still “disappear” from SFTP view after upload (moved to S3), and Lambda functions still execute automatically—but now with enterprise-grade reliability and documented uptime guarantees.
Multi-Region Independence
Separate European (~120 users) and North American (~40 users) deployments operate independently with regional S3 buckets and Lambda configurations. This architecture provides regional isolation, enables compliance with data residency requirements, and allows staged rollouts of infrastructure changes.
Event-Driven Processing Integration
S3 bucket events trigger AWS Lambda functions automatically as files arrive, maintaining the hands-off automation that makes the platform efficient. The solution handles v3’s S3 event patterns correctly, filtering out temporary creation events to ensure Lambda functions execute once per file with complete data.
Performance-Optimized Scaling
Instance sizing was optimized for the CPU-intensive nature of SFTP encryption at scale, transitioning from undersized instances to appropriately sized, compute-optimized configurations. This ensures consistent performance as file volumes and concurrent user counts grow.
Seamless Migration Path
Automated Python scripts exported complete configurations, including users, SSH keys, folder permissions, and server host keys, then imported them into the new environment. The blue/green deployment strategy, with temporary DNS routing enabled for validation before cutover and preserved host keys, eliminated client-side known-hosts warnings, achieving zero client disruption.
// Results
The Results
The transformation to enterprise-grade file ingestion infrastructure delivered both immediate operational benefits and strategic business value. The company can now document high availability architectures and provide concrete RTO commitments to enterprise prospects—directly addressing the primary barrier preventing them from closing larger deals.
Operationally, the direct-to-S3 streaming eliminated sync delays and reduced the risk of data loss while maintaining the automated Lambda processing workflows that power their financial analytics platform. The multi-region deployment now serves over 160 users across Europe and North America with independent scaling and regional data residency.
The cloud-native, stateless architecture provides the foundation for horizontal scaling as the business grows. Load balancers, auto-scaling groups, and multi-instance configurations are now possible—transforming infrastructure from a growth constraint into a competitive advantage. Most importantly, the entire upgrade preserved client workflows completely: uploads still function exactly as before, folders behave identically, and Lambda functions continue to execute automatically without any changes to downstream processing logic.
Beyond meeting immediate SLA documentation needs, the solution positions the company for future growth into regulated industries where compliance requirements demand documented infrastructure capabilities, audit trails, and formal uptime commitments.
// team
The Team
Based in the Washington-Baltimore region, Thorn Technologies specializes in developing cloud file transfer software products for organizations around the globe. Thorn Tech clients include Fortune 500 companies spanning multiple industries, such as marketing, data analytics, fintech, retail, healthcare, data storage, and cloud security.
Thorn Tech products include SFTP Gateway, a simple, secure, scalable way to give SFTP users access to cloud storage locations and StorageLink, a web-based file transfer product, allowing users to move files to your private cloud storage locations from a web browser using a simple drag-and-drop interface.
Download Our Free Ebook
Modern File Transfers: How Smart Organizations Reduce Costs and Risks​
