We are seeking a Senior Data Engineer to design, build, and optimize data pipelines that process millions of records daily and manage terabytes of data supporting critical risk calculations and regulatory reporting.

Mandatory Skill(s)

• Must have 8 years of data engineering experience, preferably in financial services or high-volume data environments;
• Advanced expertise in SQL Server, including query optimization, execution plan analysis, and performance tuning;
• Strong experience in index design, maintenance, and partitioning strategies for large tables;
• Hands-on experience in deadlock analysis, resolution, and database troubleshooting;
• Solid knowledge of database design, normalization, and managing large-scale databases;
• Strong understanding of data engineering principles, including data modeling, ETL best practices, and data quality frameworks;
• Experience in Java/J2EE development for building scalable data applications;
• Proficiency in Apache Spark (DataFrames, RDDs, Spark SQL) for developing and optimizing large-scale ETL pipelines and distributed data processing. 

Desirable Skill(s)

• Experience with SAP BusinessObjects and reporting tools;
• Background in banking, risk management, or regulatory reporting;
• Familiarity with data governance and quality frameworks;
• Knowledge of cloud platforms (AWS, Azure, or GCP).

Responsibilities

• Design and develop robust data solutions on SQL Server, ensuring optimal performance through advanced query optimization, indexing strategies, and partition management;
• Design and optimize large-scale ETL pipelines using Apache Spark to process high-volume data from multiple banking data sources;
• Ensure data quality and reliability through comprehensive validation frameworks and proactive monitoring;
• Collaborate with analytics and reporting teams using SAP BusinessObjects and other tools to deliver regulatory reports and business intelligence solutions;
• Drive continuous improvement by evaluating and adopting emerging technologies and best practices in big data engineering;
• Troubleshoot and resolve complex data pipeline issues, performance bottlenecks, and system failures.

If you are interested in this role, click on the “Apply to this job” button below or you could also write in with your CV to Aarthy Sezhian at aarthy.s@sciente.com quoting the job title.