Azure Data Engineer (Databricks)

3Cloud is a technology services firm helping clients transform their business through the power of the cloud. We leverage Microsoft Azure to help clients speed up innovation, operate more efficiently and be more responsive to their client’s needs. As a Microsoft partner, we specialize in Azure migration, cloud-scale custom application development, Internet of Things (IoT), analytics, and managed development (DevOps) achieved through Azure-enabled infrastructure automation. 3Cloud is headquartered in Chicago with an additional office in Dallas. Remote based, US only. 

As a consultant in the Data and Analytics Practice you will be responsible for delivering quality solutions using the Microsoft’s Azure suite of tools. This position has a focus on Databricks development, using Python, Scala, and Apache Spark. As a Databricks Developer, you will be able to demonstrate knowledge and expertise in data engineering best practices and concepts using Databricks. You will have intermediate to advanced knowledge with Spark architecture, including the Spark DataFrames API, and use of that API to explore, preprocess, join, and ingest data in Spark. You will also know how to read source data from a variety of formats and save data in Delta tables. You will be able to work with streaming data using various streaming APIs. You will also use your skills in SQL, including syntax expertise, performance tuning, identity management, and data access control. You will also understand Azure DevOps for structured development, including task management, and managed code deployment (including CI/CD pipelines). If you are an individual who is highly interested in Azure and Databricks technologies, you may be just who we are looking for to join our team.  This is a remote based role in US only. 

Responsibilities                                                                                                            

• Support the development of high performing, reliable and scalable solutions
• Clearly communicate technical details to business and management personnel
• Work independently or on a team to design and develop database solutions
• Assist business development team with pre-sales activities and RFPs

Qualifications

• Bachelor’s Degree desired in Computer Science, Information Technology, or related field
• Minimum of 5 years of experience with database design, at least 2 years with Azure technologies, and previous Consulting experience
• Knowledge of Databricks development, including Scala, Python, and Spark
• Advanced knowledge of SQL, including ability to write stored procedures, triggers, analytic functions, and performance tuning for Synapse
• Ability to develop utilizing the following technologies:
  • Data Movement (Apache Spark, plus Azure Data Factory and legacy SSIS)
  • Data Warehousing (Azure Synapse, CosmosDB)
  • Azure Storage Technologies (Data Lake, Blob Storage)
• Expertise in Spark DataFrames API and architecture to ingest and manipulate data, including exploring, preprocessing, joining, filtering, dropping sorting, partitioning, and renaming/manipulating columns in the dataset
• Understanding of network infrastructure and security including
  • Workspace Deployment
  • Azure Cloud Concepts
  • Network Security

Professional Skills:

• Eagerness to contribute in a team-oriented environment
• Desire to work in an information systems environment
• Excellent communication (written and oral) and interpersonal skills for both technical and non-technical teams
• Passionate about learning new technologies
• Analytical approach to problem-solving; ability to use technology to solve business problems
• Ability to work in a fast-paced environment

Additional Preferred Experience:

• Machine Learning/AI Practitioner expertise, including
  • Predictive, Prescription, and Descriptive Analytics using Data Science tools and technologies
  • Experience with Azure ML Studio/Services, and R programming language
  • Understanding of analysis models, including Supervised vs Unsupervised, Regression vs Classification, clustering, and cross-validation
  • Building, Tuning, and deploying analysis models with Spark ML and ML Flow
• Knowledge of Power BI is a plus
• Certifications are a plus