Risk Analyst (Data Annotation) (Remote)

About the Role:We're seeking a sharp, systems-thinking Risk Analyst with a Data Annotation focus to join our Product Management team. You'll help evaluate and scale labeled data workflows for the next generation of risk and compliance assessment frameworks.You'll work closely with product managers, engineers, and risk domain experts to design scalable assessment frameworks, map regulatory obligations to annotation workflows, and embed high-quality risk logic into how our platform handles training data and supervised learning. Ideal candidates bring experience in information security, audit, legal, compliance, and cloud risk, plus an emerging understanding of AI workflows and data governance.This role supports our AI-powered risk products by providing high-quality annotations and evaluations of AI-generated outputs. You'll focus on tactical annotation work while applying risk and compliance domain expertise to ensure accuracy and consistency in our training datasets.Key ResponsibilitiesPerform detailed annotations of AI-generated outputs, evaluating accuracy, correctness, and compliance alignmentReview datasets after new deployments and report on accuracy metricsTest updated control sets and validate annotation qualityReport annotation conflicts, edge cases, or unclear guidelines to improve data quality standardsApply knowledge of IT compliance frameworks (SOC 1, SOC 2, ISO 27001) to annotation tasksCollaborate with product managers and AI teams to maintain annotation integrity and consistencyRequired Qualifications2–3 years of experience in IT audit, legal, risk management, compliance, or related fields (Big Four or consulting firm experience preferred)Knowledge of IT compliance standards including SOC 2, ISO 27001, Pen Tests etc.Understanding of IT infrastructure risks and control assessmentsStrong attention to detail with ability to follow complex instructions and deliver results effectivelyGood communication skills to collaborate with cross-functional teams and report findings clearlyNice-to-Have QualificationsExposure to IT audit, legal or internal audit processesFamiliarity with data annotation or AI/ML workflowsUnderstanding of GDPR, NIST, or other regulatory frameworksBackground in data quality or governance practicesWhy Join Certa?At Certa, you'll play a critical role in building AI-powered risk intelligence products that transform how organizations assess and manage third-party risk. Join us in creating the future of compliance automation.