The role involves design and development of data solutions using Azure tools, integrating Generative AI, and mentoring junior engineers.
Role: Senior Azure GenAI Engineer
Location: Mount Laurel, NJ
Duration: FTE/C2C
Primary Skills: Azure, Databricks, Pyspark, ADF, GenAI
We are seeking a highly skilled Senior Azure Data Engineer with expertise in Databricks and PySpark to join our dynamic team. This role involves designing, developing, and maintaining scalable data solutions on Azure, leveraging modern data engineering tools and frameworks. Additionally, 20% of the role will focus on integrating and applying Generative AI (GenAI) technologies to enhance data processing, analytics, and automation capabilities.
Key Responsibilities
- Design, develop, and optimize data pipelines and workflows using Azure Data Factory, Azure Databricks, and PySpark.
- Build and maintain scalable, robust, and secure data architectures on Azure cloud platforms.
- Collaborate with data scientists, analysts, and business stakeholders to integrate data engineering solutions with GenAI models and applications.
- Develop and implement ETL/ELT processes to ingest, transform, and load large volumes of structured and unstructured data.
- Monitor and troubleshoot data pipeline performance, ensuring data quality, integrity, and availability.
- Utilize Azure Synapse Analytics, Azure Data Lake Storage, and related services to support big data and analytics initiatives.
- Lead efforts to incorporate GenAI technologies (e.g., large language models, AI-driven data augmentation, automated data insights) into existing data workflows.
- Stay current with emerging Azure technologies, Databricks features, PySpark enhancements, and GenAI advancements.
- Mentor junior engineers and contribute to best practices, documentation, and knowledge sharing.
Required Skills and Qualifications
- Proven experience with Azure Data Engineering tools: Azure Databricks, Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics.
- Strong proficiency in PySpark and Python for building and optimizing large-scale data pipelines.
- Solid understanding of Generative AI concepts, including experience working with large language models, AI-driven data augmentation, or related GenAI applications.
- Ability to collaborate effectively with AI and data science teams to integrate GenAI models into production data workflows.
- Knowledge of data governance, security best practices, and compliance standards in cloud environments.
- Excellent problem-solving skills and ability to work in an agile, fast-paced environment.
- Strong communication skills for cross-functional collaboration.
Top Skills
Adf
Azure
Azure Data Lake Storage
Azure Synapse Analytics
Databricks
Genai
Pyspark
Similar Jobs
Aerospace • Hardware • Information Technology • Security • Software • Cybersecurity • Defense
The Digital Content Developer designs and develops digital content to enhance digital literacy and support organizational change management initiatives.
Top Skills:
Adobe Creative CloudCanvaMicrosoft 365Power AppsPower AutomatePower BIPower PlatformSharepoint OnlineTeams
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
The Oncology Field Medical Director leads medical strategy and engagements for hematology, providing expert information and support to various healthcare professionals while collaborating with internal teams and managing patient-centric medical communications.
Top Skills:
Digital TechnologyMedical Content DeliveryTherapeutic Area Expertise
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
The Hematology Field Medical Director is responsible for executing medical strategy, providing expertise in hematology, and ensuring effective communication with healthcare professionals across multiple customer segments while leading projects and enhancing patient-centric medical communications.
Top Skills:
Digital TechnologyMedical Knowledge
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
