Manager - Data Engineering at Sprout Social
At Sprout Social, we build software used by 25,000 companies to create stronger relationships with their customers and create the world’s most beloved brands. As a SaaS product used 24/7/365 by organizations of all sizes, sectors, and geographies, we consume and generate vast amounts of data which hold great potential to improve our business and products.
Relying on your experience with data warehousing, writing ETL pipelines, and deploying REST and streaming services, you will enable analytics and data science teams to provide critical insights and new product features to Sprout. Our systems touch and are used by all parts of Sprout’s business, from integrating data from across Sprout’s varying databases and partner APIs, creating and maintaining the event service leveraged by Sprout’s product teams, and creating customer-facing scoring services that improve our products. Our team uses a variety of technologies, including Python, Java, Spark, Redshift, Airflow, EMR, Elasticsearch, Docker, and Kinesis.
We’re building diverse, collaborative teams that get the best results sustainably. Embedded as a member of the Data Science and Analytics team, you will work in tandem with data analysts, data scientists, data engineers, and business stakeholders to define requirements for data warehousing and data services. Your role on the team will be that of 'player-coach,' working alongside your direct reports while also carving out time to help direct and support their professional development. While you should be excited about writing and shipping production-level code, we want you to be equally excited about the trajectory of your team, forming them as stellar data engineers and contributing to shaping our long-term roadmap.
We’re looking for an experienced and passionate manager to build great software with us, to coach and mentor our growing data engineering team, and to help shape our data engineering practice. If you can confidently create data pipelines and warehouses that process and store terabytes of data, can write and maintain services to serve up data, and are passionate about learning and mentoring others, we’d love to talk with you!
These are the minimum qualifications that our hiring team is looking for in this role:
- 5+ years working as a data engineer
- 2+ years as a technical lead or manager of a data or platform engineering team
- Experience working with data scientists, analysts and other business stakeholders to define data warehousing needs.
Additionally, these are the preferred qualifications that would indicate a particularly strong candidate:
- 5+ years conducting software development in Python and some experience with Java, owning the entire development lifecycle
- Experience using and deploying services to cloud-based platforms such as AWS, Azure or Google Cloud.
- Experience deploying and maintaining RESTful and/or streaming scoring services.
Within 1 month, you will:
- Complete Sprout’s New Hire training program alongside other new Sprout team members.
- Become familiar with data sources available in our data lake.
- Learn about Airflow and our ETL processes.
- Pair with our data engineers to learn about the libraries and processes owned by data engineering.
- Begin meeting with data science & analytics stakeholders to understand existing responsibilities and needs.
Within 3 months, you will:
- Add data sources to our production data warehouse.
- Modify and/or write new data pipelines using Airflow and push them to production.
- Become familiar with our Java-based service used to store and process all events produced by Sprout’s products.
- Work with data science, analytics, and business stakeholders to develop technical requirements for new data sources and services.
- Help re-architect our data warehouse to help meet growing demands for growth and flexibility.
- Participate in Sprout’s Engineering Manager training program.
Within 6 months, you will:
- Develop data processing pipelines using EMR and Spark.
- Serve as on-call for support rotations.
- Take ownership and improve existing ETL jobs in Airflow, processing hundreds of millions of data points a day.
- Help develop processes for monitoring and alerting around our data warehouse, ETL jobs and services.
- Participate in a round of performance reviews and develop career growth plans with your direct reports.
- Help to define structure around data warehousing and ML Operations functions within Data Engineering.
Within 12 months, you will:
- Help hire additional data engineers to fill out data warehousing and/or ML operations roles.
- Own management of the data engineering practice at Sprout.
- Help develop new frameworks and libraries to allow the Data Science & Analytics team to iterate more quickly on new ETL and data sources.
- Improve data visibility across Sprout by leading deployment of tools such as Amundsen
- Own the event-generation service used across Sprout’s products.
- Help improve our existing deployment procedures.
- Identify technical debt and performance bottlenecks within our systems, come up with a plan to improve the code, and get it pushed to production.
- Create processes to improve the integrity of data stored in our data warehouse.
- Expand your skills by learning from other engineers around Sprout.
- Optimize Redshift tables for use in serving data into Tableau dashboards and data science models used by the whole company.
- Surprise us! Use your unique ideas and abilities to change your team in beneficial ways that we haven’t even considered yet.
Of course what is outlined above is the ideal timeline, but things may shift based on business needs and other projects and tasks could be added at the discretion of your manager.
About Sprout Social
Sprout Social powers open communication between individuals, brands and communities through elegant, sophisticated software. We are relentless about solving hard problems for our customers and committed to both customer and team success. Our team’s shared belief in Sprout’s mission promotes a culture of openness, empowerment and fun.
We’re proud to regularly be recognized for our team, product and culture. Our benefits program includes:
- Insurance and benefit options that are built for both individuals and families
- Progressive policies to support work/life balance, like our flexible paid time off and parental leave program
- High-quality and well-maintained equipment—your computer will never prevent you from doing your best
- Wellness initiatives to ensure both health and mental well-being of our team
- Ongoing education and development opportunities via our [email protected] program, employee-led diversity, equity and inclusion initiatives and mentorship programs for aspiring leaders
- Growing corporate social responsibility program that is driven by the involvement and passion of our team members
- Beautiful, convenient and state-of-the-art offices in Chicago’s Loop and downtown Seattle, for those who prefer an office setting
Whenever possible, Sprout wants to provide our team with the flexibility to work in the location that makes the most sense for them. For those that prefer an office setting, this role may be based in our Chicago or Seattle locations. If you prefer to work from your home, we can accommodate that for many locations across the United States. We are not set up in all states, however, so please take a look at the drop down box in our application to see whether your state is listed.
Sprout Social is proud to be an Equal Opportunity Employer and an Affirmative Action Employer. We do not discriminate based on identity- race, color, religion, national origin or ancestry, sex (including sexual identity), age, physical or mental disability, pregnancy, veteran or military status, unfavorable discharge from military service, genetic information, sexual orientation, marital status, order of protection status, citizenship status, arrest record or expunged or sealed convictions, or any other legally recognized protected basis under federal, state, or local law. Learn more about our commitment to diversity, equity and inclusion in our 2020 DEI Report.
If you need a reasonable accommodation for any part of the employment process, please contact us by email at [email protected] and let us know the nature of your request and your contact information. We'll do all we can to ensure you're set up for success during our interview process while upholding your privacy, including requests for accommodation. Please note that only inquiries concerning a request for reasonable accommodation will be responded to from this email address.
For more information about our commitment to equal employment opportunity, please click here (1) Equal Opportunity Employment Poster (2) Sprout Social's Affirmative Action Statement (3) Pay Transparency Statement.
Additionally, Sprout Social participates in the E-Verify program in certain locations, as required by law.