Data Observability Does More Harm Than Good
Data observability tools are helpful... until they spam your Slack.
Data observability tools are helpful... until they spam your Slack.
As a data leader, you must know what is happening in your data stack. As you scale your dbt models and warehouse spending, you look for observability tools to see what is happening. At first, you pour time into setting up all the alerts, funnels and tables to monitor, and it's fantastic —catching errors and proving ROI. You feel like you're the Marie Kondo of data stack organization.
But six months down the road, what typically happens?
Notification Overload: Your Slack channel gets spammed, and you stop responding or caring as false positives appear.
Lack of Context: You notice the alerts are coming in without context and provide no information on why a problem has occurred, how to fix it, or how it impacts other parts of your stack.
Capacity Shortage: As you get more alerts, your backlog expands, your stress heightens, and you feel that you can't solve all the problems you have. Each alert means you will get distracted and spend 3 to 4 hours figuring out what went wrong and solving the issue, taking you away from delivering new objectives.
This is why teams try not to adopt these tools until they need to! Who would want to deal with that?
The current rigid structure of observability tools doesn't allow for custom context, flexibility, and real value to be generated. Because teams have either struggled to implement them or implemented and experienced the pain points above, we hear this a lot about dbt:
dbt is out of control.
Scaling dbt has been a nightmare! We know it is messed up. But we don't know how to manage it or fix it?
We are swamped with new objectives, so we can't dedicate time to refactoring our current setup.
If you relate to this, you are not alone. This is why we are building Artemis. We are building the next generation of observability, context-relevant insights paired with issue resolution, helping you turn your dbt mess into a manageable model layer. We accomplish this in a few ways.
Surface Tasks, not alerts: The insights we surface leverage context from your stack to give you a task to solve, leading you to a solution faster, not creating more work.
Personalized Context: Our personalized knowledge graph provides deep context and root cause analysis to explain the issues, what caused them, how to fix them, and their impacts on downstream or upstream models and tables.
Auto-Resolve: Our action engine takes your tasks, makes the changes, updates pipelines, and refactors your models for you. All you have to do is review the PR and push it to production! You can be as little or as much involved as you want. Interested in setting and forgetting? You can do that! Want to be in the nitty gritty? That's also not a problem. You are always in control.
We give you total control, context, and solutions, so your Slack channel isn't annoying, and Jira doesn’t get bloated; it shows your team and managers how much you are getting done.