Data Engineer Scala / Spark / Kafka Srikakulam

Role: Data Engineer Scala / Spark / Kafka

nLocation: India

nEmployment Type: Contract

nExperience: 5+ Years

nDuration: 612 Months

nStart Date: ASAP

n

n

nWe are looking for an experienced Data Engineer to design, build, and optimize scalable data platforms capable of processing large volumes of batch and real-time data.

nThe ideal candidate will have robust hands-on experience with Scala, Apache Spark, Kafka, and Big Data technologies . You will be responsible for developing robust data pipelines, designing scalable data models, troubleshooting performance issues, and collaborating with engineering and analytics teams to ensure reliable access to high-quality data.

nThe role involves extensive work with distributed processing technologies, streaming platforms, and modern cloud-based data infrastructure .

n

nKey Responsibilities

n

n

Design, develop, and maintain scalable data pipelines for batch and real-time processing.

n

Develop data processing applications using Scala and Apache Spark .

n

Build and maintain real-time streaming pipelines using Kafka and Spark Structured Streaming .

n

Design and implement scalable data models and data processing solutions.

n

Work with Hive, SQL, S3, Presto, Delta Lake , and other Big Data technologies.

n

Analyze and optimize Spark jobs and distributed data workloads.

n

Troubleshoot data pipeline, performance, and production issues.

n

Ensure data quality, reliability, scalability, and availability.

n

Collaborate with engineering, analytics, and other technical teams to deliver data solutions.

n

Contribute to the design and development of contemporary data lake and lakehouse architectures.

n

n

nRequired Skills

n

n

5+ years of professional experience in Data Engineering, Big Data Engineering, or a related field.

n

Strong programming skills in Scala .

n

Hands-on experience with Apache Spark for both batch and streaming workloads.

n

Strong proficiency in Hive and SQL .

n

Hands-on experience with Kafka and real-time event streaming.

n

Strong understanding of Big Data technologies and ecosystems.

n

Experience with S3, Hive, Presto, and Delta Lake .

n

Experience designing and implementing scalable data models.

n

Solid debugging and problem-solving skills.

n

Experience with performance analysis and optimization of distributed data workloads.

n

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

n

n .