Event & Unstructured Data
Multi Recruit Verified employer
Actively hiring · Posted 5d ago · JEV26117A1FD1C89B
Job description
Key responsibility is to design & develop a data pipeline for real-time data integration, processing, executing of the model (if required), and exposing output via MQ / API / No-SQL DB for consumption
Provide technical expertise to design efficient data ingestion solutions to store & process unstructured data, such as Documents, audio, images, weblogs, etc
Developing API services to provide data as a service
Prototyping Solutions for complex data processing problems using AWS cloud-native solutions
Implementing automated Audit & Quality assurance Checks in Data Pipeline
Document & maintain data lineage from various sources to enable data governance
Coordination with BIU, IT, and other stakeholders to provide best-in-class data pipeline solutions, exposing data via APIs, loading in down streams, No-SQL Databases, etc
Skills
Programming experience using Python & SQL
Extensive working experience in Data Engineering projects, using AWS Kinesys, AWS S3, DynamoDB, EMR, Lambda, Athena, etc for event processing
Experience & expertise in implementing complex data pipeline
Strong Familiarity with AWS Toolset for Storage & Processing. Able to recommend the right tools/solutions available to address specific data processing problems
Hands-on experience in Unstructured (Audio, Image, Documents, Weblogs, etc) Data processing.
Good analytical skills with the ability to synthesize data to design and deliver meaningful information
Know-how on any No-SQL DB (DynamoDB, MongoDB, CosmosDB, etc) will be an advantage.
Ability to understand business functionality, processes, and flows
Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently
Functional knowledge
Real-time Event Processing
Data Governance & Quality assurance
Containerized deployment
Linux
Unstructured Data Processing
AWS Toolsets for Storage & Processing
Data Security
Key skills
How to apply
- Sign in or create your free profile — takes about two minutes.
- Apply with one tap — your profile is shared directly with Multi Recruit.
- Track your application and get updates by email and WhatsApp.