Data Engineer (Remote)

Only a solid grounding in computer engineering, Unix, data structures and algorithms would enable you to meet this challenge. 7+ years of experience architecting, developing, releasing, and maintaining large-scale big data platforms on AWS or GCPUnderstanding of how Big Data tech and NoSQL stores like MongoDB, HBase/HDFS, ElasticSearch synergize to power applications in analytics, AI and knowledge graphsUnderstandingof how data processing models, data location patterns, disk IO, network IO, shuffling affect large scale text processing - feature extraction, searching etcExpertise with a variety of data processing systems, including streaming, event, and batch (Spark,  Hadoop/MapReduce)5+ years proficiency in configuring and deploying applications on Linux-based systems5+ years of experience Spark - especially Pyspark for transforming large non-structured text data, creating highly optimized pipelinesExperience with RDBMS, ETL techniques and frameworks (Sqoop, Flume) and big data querying tools (Pig, Hive)Stickler of world class best practices, uncompromising on the quality of engineering, understand standards and reference architectures and deep in Unix philosophy with appreciation of big data design patterns, orthogonal code design and functional computation models