Artificial intelligence is changing the game in nearly every industry — from designing spaceships to creating dating apps. We’re using AI to write emails, plan meals and design our gardens. The new technology is making its mark across our lives, especially so in the field of data analytics.
Analytics leaders spend their days with data, and they’re perpetually learning about new tools and technologies. AI has incredible potential to streamline and transform their daily work.
“AI is already helping developers get a head start on projects where they needed to write some totally new code,” said Jay Hakim, VP of data and business intelligence at Beyond Finance.
“As impressive as they are, many companies do not yet have the data maturity to utilize them to improve internal business operations,” says Arkenaz Khaligan, associate principal of data analytics engineering at The Options Clearing Corporation.
Built In Chicago sat down with these leaders to hear more about the ways emerging technologies are impacting their work.
Beyond Finance helps clients overcome debt through customized financial products.
What are the emerging technologies that you see impacting analytics right now?
Right now, all the buzz is about using generative artificial intelligence to assist us in our work. This can manifest itself in ways we’re aware of now, but also in ways we haven’t yet explored. We have shown that AI can accelerate writing some code but not really complete it.
We have also discussed how models can provide easy correlation metrics in our business. In the areas of free-form text like online chat and conversion of voice to text, it can also provide sentiment analysis which would make it much easier to hone in on exception analysis.
When it comes to day-to-day work, we’ll be exploring the use of data build tools in our development to make the time spent writing transformations more productive. We’re also going to explore reverse extract, transform, and load tools, examining their capabilities and costs to potentially replace some homegrown frameworks.
How do you and your team members stay atop these technologies?
Using the latest technologies is always a challenge. In the past five years, many new technologies that sound like game-changers have popped up, only to fade away in about two years.
People who join our company have different technology ideas from their past work experiences, and they can share the pros and cons. I always weigh people’s experiences more heavily than reading a 5-minute summary of a product. In addition, there are a number of conferences held in the Chicago area and online that discuss the capabilities of each of the tools.
I always weigh people’s experiences more heavily than reading a 5-minute summary of a product.”
The other facet is finding the time to properly assess the capabilities of these tools and thinking about how they will be incorporated into our work. Fortunately, we have a lot of autonomy in our workplace to use the right tool for the job, so it’s totally up to us.
If we think something will provide more value and we want to incorporate it, we do it. That’s an excellent thing about working at a company that moves fast and has high expectations.
How have you incorporated any of these technologies into recent projects?
We anticipate the DBT will quickly increase productivity and improve quality once it is fully adapted. However, we want to think through how it will fit into our larger ecosystem of transformations and deployment.
We’ll look to share the learnings with other developers to see if that can be leveraged more widely on our team.
I don’t think these technologies will replace people — just make them more efficient by reducing the time it takes to complete projects.
The Options Clearing Corporation delivers risk management, clearing and settlement services for a sophisticated mix of financial products.
What are the emerging technologies that you see impacting analytics right now?
Artificial intelligence and machine learning are some emerging technologies that are currently making headlines and have a great impact on analytics.
For us, the emerging technologies that can have an impact on our day-to-day work are geared toward automation and efficiency. These technologies allow us to allocate our time and resources in the areas where we add value for the business stakeholders. No stakeholder is excited to hear it will take who-knows-how-long to be able to access analytics insights, so we leverage technology to deliver value efficiently in a way that scales and is manageable to maintain.
How do you and your team members stay atop these technologies?
It is not easy to stay on top of these technologies when it feels like there is something new released every day. To stay focused, our team avoids chasing tools that promise to immediately solve all our challenges or use cases. Instead, we focus on sustainable solutions by enhancing our data management practices and infrastructure.
It is not easy to stay on top of these technologies when it feels like there is something new released every day.”
We also try to avoid growing too dependent on one company or technology and try to find open-source technologies to leverage whenever possible. We often learn about these technologies from articles, blog posts, or someone in our professional network.
If we determine that the technology we need is not available open source, we do our own thorough research and reach out to trusted experts in our network who can offer advice before we invest in a tool.
How have you incorporated any of these technologies into recent projects?
We have incorporated an open-source development framework that combines modular SQL with data engineering best practices to make data transformation reliable and efficient. In the future, we would like to leverage the tool’s data documentation feature. That would enable each data model we build to be automatically incorporated into a data documentation website that we and our stakeholders could leverage to understand data dependencies and use cases.
Another technology that has proven to be useful is a lightweight database management tool. For some use cases, we need a tool to provide the framework of a data warehouse without the overhead. This allows us to quickly develop and iterate on a prototype, and we can later port the lightweight data warehouse into our production system.
We are currently onboarding a SaaS tool to support the automation of extraction, transformation and loading, or ETL, of data from across our company. Once in place, this tool will allow us to scale the modeling of our company’s data by cutting out the labor-intensive ETL steps which would otherwise have to be custom coded for each source. Instead, we can focus on incorporating business logic into our data models to serve quality data sources to our stakeholders.
