A New Green ETL Optimization Technique for Sustainable Data Warehousing Environment: A Scalable Energy Efficient Solution

Abstract

Energy consumption is an emerging concern in many fields, including information technology, particularly in data warehousing environments where Extract, Transform, Load (ETL) processes account for a significant portion of operational costs and resource utilization. Despite advances in hardware-level optimization, limited attention has been given to software-level energy optimization within ETL workflows. In reality, software is as important as hardware, and it is equally responsible for a decrease or increase in energy consumption. We argue that for modern applications in which energy efficiency is a priority, ETL processes should be optimally designed. This paper addresses this gap by proposing a Green ETL (GETL) approach designed to reduce energy consumption while maintaining high performance. The proposed method integrates transformation-level reuse through a shared transformation cache and adaptive parallel execution using Apache Spark, enabling efficient resource utilization and elimination of redundant computations. The proposed GETL removes unnecessary calculations and reduces both execution time and energy consumption, without requiring any modifications to the underlying data processing engines. To evaluate the effectiveness of the proposed GETL, experiments were conducted using the Transaction Processing Council Data Integration (TPC-DI) benchmark across multiple scale factors. The results demonstrate that the proposed approach achieves an average energy reduction of approximately 30%, with higher savings observed under large-scale workloads. In addition, GETL improves execution efficiency and reduces resource utilization compared to a traditional Spark-based ETL implementation.

Document