Case Study

Accelerated Informatica ETL modernization to AWS Glue with 78% auto-transformation

Migrated 96 Informatica ETL scripts to AWS Glue with LeapLogic’s end-to-end automated transformation capabilities.

Business objectives

A US federal defense, healthcare, and IT research firm wanted to modernize its legacy healthcare ETL platform to facilitate data-driven growth. They were looking for a modernization partner to help them seamlessly migrate
Informatica ETL workloads to AWS with the following business objectives:

  • Ensure better business SLA than legacy Informatica environment
  • Achieve a significant performance gain for ETL processing on a modern platform
  • Address the complexity and constraints of existing legacy systems
  • Accelerate risk-free decommissioning of the legacy footprint while minimizing business disruption

Technical needs

To drive analytical-driven growth and meet the dynamic business demands, the healthcare agency wanted to:

  • Conduct a comprehensive assessment of Informatica ETL scripts, providing actionable insights for optimization
  • Migrate 96 Informatica ETL scripts to AWS Glue, ensuring a seamless transition to the modern cloud-native platform
  • Convert 20-30 UNIX scripts to Python scripts or their equivalent

Solution

The Impetus team analyzed the Federal agency’s existing infrastructure and Informatica ETL scripts to devise a robust migration solution to AWS Glue. The team leveraged LeapLogic, Impetus’ automated cloud accelerator, for automated transformation and end-to-end operationalization of 96 Informatica ETL scripts to AWS Glue in a remarkably short timeframe.

The following are the key highlights of the solution:

  • Assessed Informatica files to determine complexity, common patterns, and inventory scoping, providing optimization recommendations.
  • Auto-transformed Informatica logic to AWS Glue Console jobs, reducing operational costs and faster execution through Amazon Redshift SQL.
  • Developed Python-based utilities for client-specific requirements, including Read/Write operations to Oracle and Amazon Redshift, file handling, and performance improvements.
  • Converted Shell scripts to AWS Glue or Python-based utilities, addressing various file formats such as Zipped and XML.
  • Generated YAML files for AWS Glue job creation via Amazon CloudFormation, minimizing manual effort.
  • Cataloged metadata of Amazon Redshift tables in AWS Glue to enhance data governance.
  • Implemented functional and system integration, user acceptance testing, and performance tuning.
  • Packaged the transformed code to ensure error-free syntax and smooth deployment, optimizing code responsiveness.

 

AWS Stack used:
AWS Glue Console, Amazon Redshift, Amazon CloudFormation, Amazon Redshift SQL

 

Impact

Automated transformation and operationalization of Informatica ETL workloads yielded significant benefits for the healthcare agency, including:

  • 50% time and cost savings: Reduced migration efforts by half, resulting in cost-effective operations
  • Rapid modernization: Automatically transformed ~80% of Informatica workloads to AWS Glue using LeapLogic, expediting the modernization process
  • Enhanced SLAs compared to the legacy Informatica setup, boosting data processing efficiency
  • Heightened performance: The modern platform delivered substantial performance gains for ETL processing, enabling quicker data-driven insights

 

Customer testimonial:

“Throughout our year-long journey to migrate from Informatica PowerCenter to AWS Glue, the Impetus team has been a true partner. From the first meeting, Impetus provided the knowledge, skills, and solutions required to transition effectively and efficiently to our 850 ETL sessions. Additionally, they regularly demonstrated their willingness to support our evolving requirements to ensure we were successful. I highly recommend Impetus and look forward to working with them again in the future.”

Senior Program Manager