Data Engineer (Remote)
Intineri infosol Pvt Ltd Verified employer
Actively hiring · Posted 1d ago · JEV261195AAF3CB86
Job description
About the Role: We are seeking an experienced Data Engineer to lead and execute the migration of existing Databricks-based pipelines to Snowflake. The role requires strong expertise in PySpark/Spark, Snowflake, DBT, and Airflow with additional exposure to DevOps and CI/CD practices. The candidate will be responsible for re-architecting datapipelines, ensuring data consistency, scalability, and performance in Snowflake, and enabling robust automation and monitoring across environments.Key ResponsibilitiesDatabricks to Snowflake Migration· Analyze and understand existing pipelines and frameworks in Databricks (PySpark/Spark).· Re-architect pipelines for execution in Snowflake using efficient SQL-based processing.· Translate Databricks notebooks/jobs into Snowflake/DBT equivalents.· Ensure a smooth transition with data consistency, performance, and scalability. Snowflake· Hands-on experience with storage integrations, staging (internal/external), Snowpipe, tables/views, COPY INTO, CREATE OR ALTER, and file formats.· Implement RBAC (role-based access control), data governance, and performance tuning.· Design and optimize SQL queries for large-scale data processing. DBT (with Snowflake)· Implement and manage models, macros, materializations, and SQL execution within DBT.· Use DBT for modular development, version control, and multi-environment deployments. Airflow (Orchestration)· Design and manage DAGs to automate workflows and ensure reliability.· Handle task dependencies, error recovery, monitoring, and integrations (Cosmos, Astronomer, Docker). DevOps & CI/CD· Develop and manage CI/CD pipelines for Snowflake and DBT using GitHub Actions, Azure DevOps, or equivalent.· Manage version-controlled environments and ensure smooth promotion of changes across dev, test, and prod. Monitoring & Observability· Implement monitoring, alerting, and logging for data pipelines.· Build self-healing or alert-driven mechanisms for critical/severe issue detection.· Ensure system reliability and proactive issue resolution.Required Skills & Qualifications· 5+ years of experience in data engineering with focus on cloud data platforms.· Strong expertise in:· Databricks (PySpark/Spark) – analysis, transformations, dependencies.· Snowflake – architecture, SQL, performance tuning, security (RBAC).· DBT – modular model development, macros, deployments.· Airflow – DAG design, orchestration, and error handling.· Experience in CI/CD pipeline development (GitHub Actions, Azure DevOps).· Solid understanding of data modeling, ETL/ELT processes, and best practices.· Excellent problem-solving, communication, and stakeholder collaboration skills. Good to Have· Exposure to Docker/Kubernetes for orchestration.· Knowledge of Azure Data Services (ADF, ADLS) or similar cloud tools.· Experience with data governance, lineage, and metadata management. Education· Bachelor’s / Master’s degree in Computer Science, Engineering, or related field.
Key skills
How to apply
- Sign in or create your free profile — takes about two minutes.
- Apply with one tap — your profile is shared directly with Intineri infosol Pvt Ltd.
- Track your application and get updates by email and WhatsApp.