Big Data Engineer

We are seeking a skilled Big Data Engineer to join our dynamic team in Karnataka or Tamil Nadu. The ideal candidate will have a strong background in big data technologies and a passion for data-driven solutions. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining scalable data processing systems that handle large volumes of data efficiently.

Key Responsibilities:

• Design and implement data processing pipelines using Hadoop and Spark to support data analytics and reporting.
• Develop and maintain ETL processes to ensure data quality and integrity.
• Collaborate with data scientists and analysts to understand data requirements and provide necessary data solutions.
• Optimize existing data processing workflows for performance and scalability.
• Write efficient SQL queries to extract and manipulate data from various sources.
• Provide application support and troubleshoot issues related to data processing and storage.
• Document processes and maintain best practices for data engineering.

Required Skills:

• Strong experience with big data technologies such as Hadoop and Spark.
• Proficiency in Python and Pyspark for data processing and analysis.
• Solid understanding of SQL for database management and data manipulation.
• Familiarity with Unix and Shell Scripting for automation tasks.
• Experience in application support and troubleshooting.

If you are passionate about big data and eager to work in a collaborative environment, we encourage you to apply for this exciting opportunity.