Aurimas - great blog! I love data and the ecosystem has gotten so complex with so many choices - this is the right way to think about simplifying yet taking advantage of the options.
I noticed Data Observability is missing. It has been omitted for far too long and we're in a bit of a mess due to that. Let's correct it. IMO, the DO layer should go all across the layers you have included. Happy to collaborate. Thx
Thank you for your kind words Kamal. And thank you for sharing your thoughts with which I completely agree. I would have DO as part of each layer as it can still be decomposed since different practices will apply in different stages. A central metadata store for DO is needed for sure!
Hi Aurimas,
Thanks for the reading, it's really complex each one of the steps and it's useful understand the start point where the data is born.
Thanks!
Great Article
Thank you Ajay.
What an excellent read, Aurimas!
Thank you!
Aurimas - great blog! I love data and the ecosystem has gotten so complex with so many choices - this is the right way to think about simplifying yet taking advantage of the options.
I noticed Data Observability is missing. It has been omitted for far too long and we're in a bit of a mess due to that. Let's correct it. IMO, the DO layer should go all across the layers you have included. Happy to collaborate. Thx
Thank you for your kind words Kamal. And thank you for sharing your thoughts with which I completely agree. I would have DO as part of each layer as it can still be decomposed since different practices will apply in different stages. A central metadata store for DO is needed for sure!
If you decide to update the diagram, pls share. I'd reshare this to my network on LI. Thx
Thank you for the great article.
A question please: which tool are you using to create these amazing diagrams?