GCP, Biq Query, Data Flow, Dataproc 88003

Positions:2 Full Time

Experience

7 - 11 Years

Job Description: Senior Data Engineer

Position Overview

We are seeking a talented and experienced Senior Data Engineer (Engineer 3) to join our organization. In this role, you will be responsible for designing, building, and maintaining comprehensive data solutions including data infrastructure, pipelines, and platforms for collecting, storing, processing, and analyzing large volumes of data efficiently and accurately. You will collaborate with business and technology stakeholders to deliver scalable, reliable, and cost-effective data solutions that support analytics, machine learning, and business intelligence initiatives.

Key Responsibilities

Collaborate with business and technology stakeholders to understand current and future data requirements and translate them into actionable data solutions

Design, build, and maintain reliable, efficient, and scalable data infrastructure for data collection, storage, transformation, and analysis

Plan, design, build, and maintain scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow

Design, implement, and maintain existing and future data platforms such as data warehouses, data lakes, and data lakehouses for both structured and unstructured data

Design and develop analytical tools, algorithms, and programs to support data engineering activities including writing scripts and automating tasks

Ensure optimum performance of data systems and identify improvement opportunities through monitoring and optimization

Design and implement data architecture with specialized knowledge in various data transformation mechanisms

Leverage big data tools and programming frameworks that meet organizational requirements for data storage, processing, analytics, and data science while considering long-term strategy, tool selection, and cost-effectiveness

Deliver appropriate analytical data modeling techniques based on data usage needs for Data Science, Machine Learning, Visualization, and other business requirements

Develop scalable and maintainable data solutions that support organizational growth

Incorporate CI/CD pipelines to improve efficiency and quality of development and deployment processes, accelerate release cycles, and enhance overall product stability

Establish benchmarks and adopt proactive monitoring and reliability techniques to ensure data system performance and availability

Adopt and implement appropriate data governance processes to ensure compliance with data protection and data safeguard requirements at both regional and organizational levels

Own and execute initiatives and projects within timelines, mitigate risks, and ensure delivery of high-quality solutions on schedule

Participate in periodic retrospectives to identify opportunities for continuous improvement

Mentor junior team members and contribute to team excellence by proactively identifying and resolving roadblocks

Productionize components and models at scale, working effectively with large data volumes

Stay current with emerging technologies and introduce innovative solutions and best practices to the organization

Maintain strong communication with cross-functional teams and stakeholders through clear oral, written, and interpersonal communication

Required Qualifications

Experience: 7+ years of professional Data Engineering work experience

Google Cloud Platform (GCP) Expertise: Hands-on experience with BigQuery, Dataflow, Dataproc, Data Fusion, and Cloud SQL

Infrastructure & Orchestration Tools: Proficiency in Terraform, Tekton, and Apache Airflow

Data Processing & Streaming: Strong experience with Kafka, Pub/Sub, and PySpark

Programming Languages: Expert-level proficiency in Python and API development

Database Technologies: Solid experience with PostgreSQL and relational database design

Data Architecture: Demonstrated expertise in designing and implementing scalable data solutions and data models

Big Data Technologies: Strong knowledge of big data tools and frameworks for large-scale data processing

CI/CD & DevOps: Understanding of continuous integration and continuous deployment practices

Problem-Solving: Strong analytical and troubleshooting skills with ability to optimize data systems

Communication: Excellent oral, written, and interpersonal communication skills with ability to work across teams

Preferred Qualifications

Experience with Jira for project tracking and collaboration

Knowledge of Data Governance frameworks and best practices

Experience with machine learning platforms and model deployment

Familiarity with data visualization tools and business intelligence platforms

Background in cloud data architecture and multi-cloud environments

Experience with containerization technologies and Kubernetes

Key Competencies

Technical Excellence: Deep expertise in data engineering technologies and best practices

Scalability & Performance: Ability to design systems that handle large data .