Senior Data Engineer at project44
Our mission is to usher in a new era of trust and predictability to transportation and logistics, but there’s still a lot of work to be done. The industry is massive, as is the opportunity. We’re looking for bright, ambitious individuals to join our growing global team and help us enable a more productive and successful world. We’re changing the way the world ships, and we’re looking for you to help us get there!
About the Role:
As a Sr Data Engineer at project44, you'll have opportunities to work on ETL pipeline, support & solve various data integration challenges and improve core data infrastructure.
- Design, build and improve our ETL platform and pipelines utilizing Airflow/Argo Workflow and integrates with Data lake (Snowflake) and Postgres/Kafka
- Ensure standards for engineering excellence, scalability, reliability, and reusability
- Debug production issues across services and multiple levels of the stack
- Partner with the insights and data science teams to automate processes to improve data sets for analytical and reporting needs
- Write test cases, code coverage, perform QA and participate with stakeholders on UAT
- Participate in an on-call rotation to mitigate any data pipeline failures
- Focus on development/improvement of framework to support repeatable and scalable solutions
- Take initiative to recommend/develop innovative approaches to getting things done
- Work as a team player and encourage collaboration
- Bachelor's Degree in computer science or equivalent experience
- 5+ years experience building data pipelines using Airflow, Spark or Kafka Streams
- Excellent problem solving skills
- Proficiency in Python, Java or other similar languages
- Familiarity with architecture of event collection pipelines and analytical data stores such as Snowflake
- Strong programming/scripting knowledge in building and maintaining ETL using Java, SQL, Python, Bash, Go
- In-depth hands-on knowledge of public clouds - GCP(preferred)/AWS, PostgreSQL (version 9.6+), ElasticSearch, MongoDB, MySQL/MariaDB, Snowflake
- Ability to work in a fast-paced, rapidly changing environment
- Understanding of Agile and its implementation for Data Warehouse Development
- Excellent communication and interpersonal skills; able to communicate clearly and concisely
- Strong experience with Kafka or equivalent event/streaming based systems
- Experience with Docker, Kubernetes
- Develop and deploy CICD pipelines for Data Engineering
- Experience and knowledge of optimizing database performance and capacity utilization to provide high availability and redundancy
- Proficiency with high volume OLTP Databases and large data warehouse environments
Diversity & Inclusion
We're designing the future of how the world moves and is connected through trade and global supply chains. We can only deliver a truly world-class product and experience if our teams are as diverse and unique communities we are building for. So it's up to us to create a company where anyone can bring their authentic self to work everyday. We're constantly working to improve, and we accept our responsibility to elevate the voices left in the margins. It's on every one of us.
Our focus on inclusion manifests in the way we hire, the customers we serve, and the regions we prioritize. We're building a company that every one of us at project44 is proud to work for: a company that celebrates you for being you.
We pride ourselves on celebrating everyone — project44 is an equal opportunity employer actively working on creating a diverse and inclusive work environment where underrepresented groups can thrive. If you share our values and our passion for helping the way the world moves, we’d love to review your application!
For any needed accommodations during the hiring process, please email [email protected] Even if you don’t meet 100% of the above qualifications, you should still seriously consider applying. Studies show that you can still be considered for a role if you meet just 50% of the role’s requirements.
More about project44
Since 2014, project44 has been transforming the way one of the largest, most important global industries does business. As transportation and logistics continues to evolve and customer expectations around delivery become more demanding, industry technology must rise to the occasion. In just a few short years, we’ve created a digital infrastructure that eliminates the inefficiencies caused by dated technology and manual processes. Our Advanced Visibility Platform is used by the world’s leading brands to track shipments, collaborate with supply chain partners, drive operational efficiencies, and create outstanding customer experiences.