Zifo Logo

Zifo

Senior Data Engineer

Reposted 28 Days Ago
Remote
Hiring Remotely in United States
Senior level
Remote
Hiring Remotely in United States
Senior level
Designs and operates data infrastructure for a clinical trial platform. Builds pipelines that extract, normalize, validate, and link information from clinical documents and publications into Aurora and GraphDB. Develops embeddings, vector search, RAG workflows, data-serving APIs, lineage tracking, and regulatory audit trails. The role also requires AWS-based orchestration, clinical data quality controls, biomedical knowledge graph modeling, and deployment automation using infrastructure-as-code and containerization.
The summary above was generated by AI

Design, build, and operate the data infrastructure for a clinical trial design platform that extracts insights from clinical protocols, regulatory documents, and published articles to accelerate trial design decisions. This role owns the pipelines that ingest, parse, transform, and serve clinical data - from raw PDF extraction through structured storage in Aurora and GraphDB, to serving curated knowledge for LangGraph-based agentic AI workflows.


Requirements
  • Build ingestion pipelines for clinical trial protocols, ICF documents, SmPCs, CSRs, and published articles (PubMed, CTIS, ClinicalTrials.gov) - handling PDF parsing, text extraction, and structured data normalization
  • Design and implement data models in Amazon Aurora (relational) and GraphDB (knowledge graph) to represent trial design entities: endpoints, eligibility criteria, study arms, interventions, therapeutic areas, and their relationships
  • Develop embedding and vectorization pipelines to prepare extracted clinical text for RAG-based retrieval in LangGraph agentic workflows - chunking strategies, metadata enrichment, and vector store population
  • Build and maintain ETL/ELT workflows that transform unstructured clinical content into queryable, linked data across both relational and graph stores
  • Implement data quality validation specific to clinical data - protocol section classification accuracy, entity extraction completeness, cross-reference integrity (NCT IDs, EudraCT numbers, MeSH terms)
  • Build data serving APIs (Python/FastAPI) that expose curated datasets to the Angular frontend and LangGraph agent layer
  • Set up data lineage tracking and audit trails to support regulatory traceability of AI-generated trial design recommendation
Required Skills
  • Strong Python development experience, including PDF/document parsing libraries such as PyMuPDF, pdfplumber, unstructured.io, or similar.
  • Advanced PostgreSQL-compatible SQL, including Amazon Aurora; experience with schema design, migrations, query optimization, and indexing strategies for large clinical datasets.
  • Hands-on experience with Neptune, Neo4j, or similar graph databases; proficiency in SPARQL or Cypher; experience with ontology and knowledge graph modeling for biomedical entities.
  • Experience with AWS services including Aurora PostgreSQL, S3, Lambda, Step Functions, SQS/SNS, and IAM, particularly for data pipeline orchestration.
  • Experience with PDF text extraction, document section classification, and named entity recognition (NER) for clinical/biomedical text; familiarity with embedding models and vector stores such as OpenSearch, pgvector, or Pinecone.
  • Experience building data-serving APIs using FastAPI, including asynchronous programming patterns and backend integration.
  • Experience preparing data for LangChain/LangGraph applications and designing RAG pipelines, including chunking, retrieval, reranking, and prompt-data integration.
  • Experience with Airflow, Prefect, AWS Step Functions, Temporal, or similar workflow orchestration tools; ability to design multi-stage DAGs with dependency management, retry logic, monitoring, and error handling.
  • Experience with Terraform or AWS CDK, Docker, and Git, including automated pipeline testing and deployment on AWS.

Domain Knowledge

  • Understanding of clinical trial structure: protocol sections (objectives, endpoints, eligibility criteria, study design, statistical considerations)
  • Familiarity with clinical data standards or terminologies (MeSH, MedDRA, SNOMED, ATC codes, CDISC) is a strong plus
  • Awareness of regulatory data integrity requirements (21 CFR Part 11, EU Annex 11, ALCOA+ principles)

Nice to Have

  • Experience with biomedical knowledge graphs (e.g., linking drugs -> targets -> diseases > trials)
  • Prior work with PubMed/MEDLINE data, ClinicalTrials.gov API, or EMA/CTIS data.
  • Apache Spark or Databricks for batch processing of large document corpora
  • dbt for transformation layer management over Aurora

Benefits

CURIOSITY DRIVEN, SCIENCE FOCUSED, EMPLOYEE BUILT. Our culture is unlike any other, one where we debate, challenge ourselves, and interact with all alike. We are a curious bunch, characterized by our passion to learn and spirit of teamwork. Zifo is a global R&D solutions provider focused on the industries of Pharma, Biotech, Manufacturing QC, Medical Devices, specialty chemicals and other research-based organizations. Our team’s knowledge of science and expertise in technology help Zifo better serve our customers around the globe, including 18 of the Top 20 Biopharma companies. 

We look for Science – Biotechnology, Pharmaceutical Technology, Biomedical Engineering, Microbiology etc. We possess scientific and technical knowledge and bear professional and personal goals.  While we have a “no doors” policy to promote free access within, we do have a tough door to walk in. We search with a two-point agenda – technical competency and cultural adaptability. 

We offer a competitive compensation package including accrued vacation, medical, dental, vision, 401k with company matching, life insurance, and flexible spending accounts.  

If you share these sentiments and are prepared for the atypical, then Zifo is your calling! 

Zifo is an equal opportunity employer, and we value diversity at our company.  We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. 


Zifo Chicago, Illinois, USA Office

2058, E Grand Avenue, Suite 11A, Chicago, IL, United States

Similar Jobs

Yesterday
Remote or Hybrid
115K-145K Annually
Senior level
115K-145K Annually
Senior level
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
Design, build, and support scalable data pipelines, cloud-native processing systems, data models, integrations, and BI solutions. Ensure data quality, observability, testing, CI/CD, documentation, and production reliability. Partner with technical and business stakeholders, participate in Agile delivery, resolve incidents, and mentor engineers while advancing data architecture and engineering standards.
Top Skills: Amazon AthenaAmazon EmrAmazon MwaaAmazon RedshiftAmazon S3Amazon SnsAmazon SqsApache AirflowApache HiveApache IcebergSparkAWSAws GlueAws LambdaAws Step FunctionsCi/CdDatabricksEmr ServerlessGitGithub ActionsJavaLookerMicrostrategyPostgresScalaSinglestoreSnowflakeSQLTableau
8 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
125K-234K Annually
Senior level
125K-234K Annually
Senior level
Fintech • Mobile • Software • Financial Services
Build and maintain production data pipelines, dbt models, Airflow workflows, and governed datasets supporting People Analytics. Integrate Workday and other HR systems into reliable Snowflake data products, implement data quality monitoring, row-level security, semantic models, and AI agents. Partner with engineers, analysts, data scientists, and business stakeholders to deliver trusted data solutions, communicate technical trade-offs, enforce governance, and mentor other engineers.
Top Skills: Ai AgentsAmazon RedshiftApache AirflowCi/CdDbtEtl/EltGitlabGoogle Cloud PlatformLlmsPythonSnowflakeSQLWorkday
An Hour Ago
Remote or Hybrid
United States
97K-146K Annually
Senior level
97K-146K Annually
Senior level
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Analyze and automate supplier-quality data using ETL tools, dashboards, KPIs, and statistical techniques. Build actionable insights and predictive reporting for quality, engineering, production, and leadership teams. Collaborate cross-functionally on continuous improvement, coach employees on data-driven decision-making, deliver presentations, and document analytical processes. The role requires expertise in Alteryx or comparable ETL tools, Tableau or Power BI, SQL or Python, UX/UI design, and supplier-quality analytics.
Top Skills: AlteryxPower BIPythonReactSQLTableau

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account