Designs and builds scalable data platforms and pipelines for scientific, clinical, manufacturing, and process-engineering data. Responsibilities include developing batch and streaming ETL/ELT systems, integrating Databricks, Snowflake, cloud platforms, MES, SCADA, and laboratory systems, and enforcing governance and GxP compliance. The role provides hands-on technical leadership, establishes data engineering standards, and collaborates with scientists, process engineers, and platform teams. It requires three days onsite weekly in Indianapolis and unrestricted U.S. work authorization.
Location: Indianapolis, IN Metro (Hybrid / 3-Day Onsite) (Open to Regional/EST Candidates with Onsite Travel)
Contract Type: Contractor Full-Time / Enterprise Project Engagement (Outsourced via Xenon7)
About Xenon7Where elite tech talent meets world-class opportunities! At Xenon7, we work with leading enterprise clients and innovative startups on high-impact projects across Data, AI, Cloud, and Software Engineering. Our expertise in AI solution architecture and specialized technical talent allows us to partner with enterprise leaders on transformative initiatives, driving innovation and business growth.
Job SummaryWe are seeking a Senior Data Engineer with extensive experience in data architecture and platform engineering to drive data initiatives for a top-tier life sciences client. This role sits at the critical intersection of scientific research informatics, clinical data systems, and manufacturing process engineering.
In this position, you will own the architectural vision and hands-on execution of scalable data platforms and pipelines. You will bridge complex data domains—from small and large molecule research, genomics, and clinical trial datasets to active pharmaceutical ingredient (API) manufacturing processes, batch data, and industrial control systems. Operating in a 3-day onsite hybrid capacity in Indianapolis, you will collaborate directly with process engineers, clinical scientists, and platform teams to build high-performance data infrastructure that accelerates drug discovery and manufacturing operations.
Key ResponsibilitiesScientific & Clinical Data Platform Architecture- Design, build, and maintain production-grade data pipelines and architecture tailored for scientific, clinical trial, and research informatics data (small/large molecule, genomics, proteomics, LIMS).
- Structure complex, multi-modal clinical and scientific datasets to enable advanced analytics, enterprise reporting, and downstream machine learning models.
- Ingest, harmonize, and model operational technology (OT) and manufacturing process datasets, including API manufacturing pipelines, batch processing data, MES, SCADA, and OSIsoft PI systems.
- Unify disparate laboratory and facility data pipelines into centralized, highly available enterprise data platforms.
- Build robust ETL/ELT pipelines using modern cloud platforms (Databricks, Snowflake, AWS/Azure), PySpark, and SQL.
- Ensure all data pipelines and platform integrations strictly adhere to enterprise governance, data residency, and GxP regulatory standards within a heavily monitored environment.
- Partner directly with process engineers, chemical engineering leads, and research informatics directors to translate operational friction into robust technical specifications.
- Establish engineering best practices, data modeling standards, and pipeline monitoring frameworks across the enterprise data stack.
RequirementsExperience & Mindset
- Experience: Senior-level proficiency (10–20+ years) in software development, data platform architecture, and complex ETL/ELT engineering.
- Domain Adaptability: Demonstrated ability to engineer data pipelines across non-standard, highly specialized domains (e.g., transition between process/chemical engineering data and clinical/scientific research informatics).
- Location & Work Auth: Must hold unrestricted US Work Authorization (no sponsorship available) and be able to work 3 days per week onsite in the Indianapolis, IN area.
- Culture & Communication: Exceptional problem-solving mindset, strong adaptability, and the ability to articulate complex technical architecture to cross-functional engineering teams.
- Languages & Frameworks: Advanced Python, PySpark, and expert-level SQL.
- Data Platforms: Hands-on expertise with Databricks, Snowflake, or AWS/Azure enterprise data ecosystems.
- Orchestration & ETL: Extensive experience with Airflow, dbt, Spark, and enterprise data orchestration engines.
- Data Pipelines: Proven track record building streaming and batch data architectures via REST APIs, message brokers, and database integrations.
- Deep exposure to either scientific/clinical informatics (CDISC/SDTM, LIMS, clinical trials, multi-omics) OR chemical/process engineering data (API manufacturing, batch data, SCADA, MES, OSIsoft PI).
- Academic background in Chemical Engineering, Bio-process Engineering, Computer Science, or a related STEM discipline.
- Direct experience working inside regulated GxP environments in the Life Sciences or Specialty Chemicals sectors.
- Certifications: Databricks Certified Data Engineer Senior/Professional, Snowflake SnowPro Core/Advanced, or AWS Data Engineer Associate/Professional.
- Not a pure Data Scientist or ML Researcher: You will not be building or training machine learning models; you are designing and scaling the underlying data architecture, pipelines, and platform infrastructure.
- Not a non-coding Architect: This is a 100% hands-on engineering lead role requiring direct pipeline construction and technical execution.
- Not a Fully Remote Position: This role requires a steady hybrid commitment of 3 days onsite per week at the client site in Indianapolis.
Similar Jobs
Big Data • Cloud • Productivity • Software • Database • Analytics • Automation
Build and maintain Databricks-based data platforms, including ingestion, transformation, storage, governance, data modeling, and serving pipelines. Establish medallion architecture standards, canonical data models, quality controls, lineage, schema evolution, and reliable batch or incremental processing. Improve pipeline observability, scalability, idempotency, and recoverability while moving curated data to systems such as ClickHouse. Collaborate across application and analytics teams to create durable, governed production datasets.
Top Skills:
Amazon AuroraAmazon RdsApache AirflowSparkBigQueryCdcClickhouseCloud Object StorageDatabricksDelta LakeIamOpenmetadataPostgresSnowflakeUnity Catalog
AdTech • Consumer Web • Digital Media • eCommerce • Marketing Tech • SEO
Build and support scalable data platforms and pipelines, leading migrations such as Snowflake to BigQuery and transitioning reporting to Looker. Responsibilities include data architecture, ETL/ELT, API and marketing integrations, data quality, production troubleshooting, warehousing, performance optimization, and platform modernization. The role partners with analytics and business teams, owns projects through production, documents solutions, and provides technical guidance.
Top Skills:
AWSAzureBigQueryConfluenceDraw.IoGCPGitJIRAKafkaLookerLucidchartMiroModeNotionPower BIPythonSnowflakeSparkSQLTableauTalend
Information Technology • Productivity • Software • Infrastructure as a Service (IaaS)
Design and scale lakehouse architecture, streaming and batch pipelines, and reliable data platforms using Kafka, Spark, Airflow, Iceberg, Databricks, and related technologies. Build Medallion-layer data systems, manage open table formats, optimize distributed queries, monitor platform reliability, resolve data quality issues, and collaborate with analysts, data scientists, and product teams.
Top Skills:
Apache AirflowApache HudiApache IcebergApache KafkaSparkDatabricksDelta LakePythonSQLStarburstTrino
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
