Architect and operate secure, compliant AWS data pipelines and analytics integrations. Ensure data quality, lineage, residency, and privacy-preserving engineering to meet HIPAA, SOC 2, GDPR, and CCPA. Lead data modeling, build CI/CD for data infrastructure, enforce least-privilege access, implement IaC governance, mentor engineers, perform code reviews, and produce runbooks and compliance evidence.
Position Summary
The Sr Analytics Engineer architects, builds, and operates secure, scalable data pipelines and analytics integrations within a governed AWS cloud environment. This role carries direct accountability for data quality, privacy-preserving engineering, and compliance with HIPAA, SOC 2, GDPR, and CCPA. The individual serves as the technical authority on analytics engineering standards and partners with Security, Legal, and Compliance teams to ensure all data products meet enterprise governance requirements.
AWS Technology Stack
Deep hands-on expertise required across the following AWS service areas:
- Data Storage & Lake: S3, Lake Formation, Glue Data Catalog, DynamoDB
- Processing & ETL: Glue (PySpark), Lambda, Step Functions, EMR
- Analytics & Query: Athena, Redshift / Redshift Serverless, QuickSight
- Integration & Streaming: Kinesis, API Gateway, Transfer Family, AppFlow
- Security & Encryption: KMS, IAM, Secrets Manager, Macie, GuardDuty, CloudTrail
- Monitoring & Compliance: CloudWatch, AWS Config, X-Ray, Systems Manager
- Networking & Isolation: VPC, PrivateLink, AWS Clean Rooms
Governance & Compliance Responsibilities Regulatory Compliance
- Design data architectures compliant with HIPAA (PHI encryption, minimum necessary access, audit retention), GDPR/CCPA (DSAR automation, right-to-erasure, consent management), SOC 2 Type II (access reviews, change management, continuous monitoring), and PCI DSS where applicable.
- Implement data residency and sovereignty controls ensuring data remains within approved AWS Regions.
Data Governance & Quality
- Enforce data classification standards (Public, Internal, Confidential, Restricted) via Macie and Lake Formation LF-Tags. Implement automated data quality frameworks with freshness, completeness, and accuracy SLAs.
- Maintain data lineage documentation, data retention/purge policies via S3 Lifecycle Rules and DynamoDB TTL, and participate in Data Governance Council reviews and compliance audits.
Security & Access Governance
- Implement least-privilege access models with quarterly reviews, IAM Access Analyzer scans, and encryption-at-rest/in-transit standards with KMS key policy management.
- Design masking, tokenization, and anonymization pipelines for non-production environments. Enforce IaC governance via CloudFormation/Terraform with security review gates and drift detection.
Responsibilities
Key Responsibilities
- Architect production-grade, fault-tolerant batch and real-time data pipelines within a compliance-governed AWS environment.
- Lead data modeling (dimensional, SCD, data vault) and build API/file-based integrations with encryption and audit logging at every point.
- Build CI/CD pipelines for data infrastructure with mandatory security scanning and approval gates.
- Conduct code reviews, mentor engineers, and produce technical documentation including ADRs, runbooks, and compliance evidence packages.
Qualifications
Qualifications Required
- Bachelor’s degree in Computer Science, Data Engineering, or related field; 7+ years in Analytics/Data Engineering.
- Expert SQL and Python (PySpark, Pandas, Boto3); deep AWS experience across the data, security, and governance stack.
- Demonstrated experience implementing governance frameworks and regulatory compliance controls (HIPAA, SOC 2, GDPR, CCPA) in production.
- Experience with IaC (CloudFormation, Terraform, CDK), CI/CD for data pipelines, and data security principles.
Preferred
- AWS certifications (Data Analytics Specialty, Security Specialty, Solutions Architect Professional).
- Experience with data quality frameworks (Great Expectations, dbt, Monte Carlo), and regulated industries (healthcare, financial services).
Similar Jobs
Fintech • Financial Services
Design, build, and maintain scalable dimensional data models, semantic/metrics layers, and production data pipelines (dbt, Snowflake). Partner with Data Science, Data Engineering, BI, and DevOps to enable AI/ML, real-time analytics, governance, and data quality while mentoring junior engineers and enforcing best practices.
Top Skills:
CortexDatafoldDbtFivetranKafkaLlmPythonRagRedshiftSnowflakeSnowflake IntelligenceSQLStreamkapTableau
Fintech • Financial Services
Design, build, and maintain dimensional data models, semantic layer, and production data transformation pipelines (dbt/Snowflake). Partner with Data Engineering, Data Science, BI, and Product to deliver governed, AI-ready data products, monitor data quality, enable ML features, and coach junior engineers to uphold code and governance standards.
Top Skills:
CortexDatafoldDbtFivetranKafkaPythonRagRedshiftSnowflakeSnowflake IntelligenceSQLStreamkapTableau
Marketing Tech • Software
As the first Data Engineer, you will architect the Snowflake governed query layer, enable AI-driven tools, and define data access standards for various departments, significantly shaping Knak's data infrastructure and analytics capabilities.
Top Skills:
DbtGitPythonSnowflakeSQLTableau
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
