Senior Data Engineer

Role & ResponsibilitiesAbout the Role:We are seeking a highly skilled Senior Data Engineer with 5-7 years of experience to join our dynamic team. The ideal candidate will have a strong background in data engineering, with expertise in data warehouse architecture, data modeling, ETL processes, and building both batch and streaming pipelines. The candidate should also possess advanced proficiency in Spark, Databricks, Kafka, Python, SQL, and Change Data Capture (CDC) methodologies.Key responsibilities:Design, develop, and maintain robust data warehouse solutions to support the organization's analytical and reporting needs.Implement efficient data modeling techniques to optimize performance and scalability of data systems.Build and manage data lakehouse infrastructure, ensuring reliability, availability, and security of data assets.Develop and maintain ETL pipelines to ingest, transform, and load data from various sources into the data warehouse and data lakehouse.Utilize Spark and Databricks to process large-scale datasets efficiently and in real-time.Implement Kafka for building real-time streaming pipelines and ensure data consistency and reliability.Design and develop batch pipelines for scheduled data processing tasks.Collaborate with cross-functional teams to gather requirements, understand data needs, and deliver effective data solutions.Perform data analysis and troubleshooting to identify and resolve data quality issues and performance bottlenecks.Stay updated with the latest technologies and industry trends in data engineering and contribute to continuous improvement initiatives.