For those who are not familiar with the variations of roles in the field of data, this is what I’ve learned so far about those roles. You have data analysts, data architects, data engineers and data scientists.
A data analyst works on, no surprise, analysing data. Often these are people preparing reports (for instance on sales performance) for management. When working in Microsoft Azure, you would spend a lot of time using Power BI to create reports, using polished data sets to perform transformations and calculations. Your goal is to create visualizations that anyone else in your organisation can easily understand.
To create those polished data sets for analysts to work with, you need a data infrastructure. That’s where the data architect comes in. An architect thinks through what the data needs are on one end, knows the perks of the raw data coming in at the other end and then designs the infrastructure in between.
The data engineer then uses the input from the data architect to build the infrastructure.
Sometimes you want to dig deeper into your data and discover more complex patterns. That’s where the data scientist comes in. These math wizards apply statistical, machine learning and AI models to data and are able to tweak these models using there mathematical knowledge.
I was rather surprised to learn how working in data, a relatively new field to work in, already split up in so many roles. And then I’m not even talking about all the specialisations you could choose within these roles for hard core programmers. For instance my trainer worked for many years solely on optimising SQL statements for a living.
My education prepares me for two roles mainly: the analyst and the engineer. I’m most definitely happy with all the skills I learned about using Power BI. That will help me a lot when I start digging for stories using data sets. The data engineering part is absolutely not my cup of tea. It’s really theoretical and an in-depth crash course on database management and Azure cloud infrastructure. You can compare it to fitting electrical pipes in your home. It needs to be done, otherwise you can’t live in your home, but it’s not as exciting as decorating your home. At least I get more excited about decoration and design than fitting pipes. That said, I’m still very happy to get a solid understanding of the inner workings of databases and the cloud tools one needs to create usable data from raw data. It helps me to be able to instruct others to click the right buttons, write the proper SQL statements and build the pipelines for me, so that I can dig for the data story gold 😉
We should have known better. Five days ago I wrote that the dishwasher we originally ordered wouldn’t fit in our kitchen and thus went back to the shop we bought it from. Then we quickly decided to order another one, to be delivered a week from now. The gamble we took was to order a ‘second chance’ model instead of a new one. That gamble turned sour, but not for the reason I thought it was a gamble (that it was previously repaired). The Man noticed that the credit card was still not credited for the new order and called customer service. It turned out that the entire order, including the appointment for installation, was cancelled without noticing us. We could have been waiting all week, expecting a new dishwasher next Saturday, for nothing. When on the phone the employee offered to place a new order “and that can be delivered tomorrow”. With that promise he forgot that the delay is in the installation. Two weeks, maybe even longer, to make an appointment for that.
Of course we were livid. The Man send a very lengthy e-mail to customer service this afternoon and as I’m writing this we’re waiting for their response.
Considering the shortage in dishwashers, I didn’t wait for the company to come up with a solution. Their own stock is even lower than when we first ordered, three weeks ago. I checked their site but they have absolutely no model left that fits our kitchen. Acting quickly is key, so again I scoured the web for dishwashers. It really feels like a scavenger hunt by now. I ended up with a company in Dokkum, way up north, that has one Siemens model on stock that suits us. We even called the shop, just before closing hours, to make sure our kitchen panel will indeed fit. They don’t do installation, but I’d rather have a machine and find a local professional to pay for the installation (or even try installing it myself!) than end up without a new dishwasher for many more weeks. We placed our order and now we have to wait and see if indeed this dishwasher can and will be delivered soon. Fingers crossed!
Though talking from the viewpoint of the US, I really enjoyed listening to Allie Ward’s episode Quarantinology. Covid-19 regulations are being lifted, now what? Ward talked to several researchers covering the history of quarantine, what to expect in the near future regarding infection rates and how to deal with settling in to our new lives. Indeed, we’re not getting back to normal. Normal is dead, according to Cole Imperi. Imperi also talks about ‘shadow losses’, things we lost during the pandemic, and our need to grief for them. That part really made sense to me. I highly recommend listening to this episode of Alli Ward’s excellent show.
I am slowly morphing into a true data nerd. This week I started learning the data engineering part of data. Learning about lambda and delta lake architecture, ETL, OLTP, OLAP, Apache Spark, notebooks, parquet files, SCD Types 1 2 3 and 6, pipelines, serverless and dedicated SQL pools, PolyBase and there’s more to come.
Since Fall 2020 I haven’t had that many occasions where I had to pay attention to what I’m wearing. As I spend most of my days sitting at my desk while watching my computer screen, especially since starting my course at Techionista Academy, I prefer to wear loose-fit trousers. Not the sporty kind or the pyjama kind, but nicely designed trousers with an elastic band and soft fabric. For months, the only reason to go outside was to go for a walk and for the larger part I did that at the end of the afternoon when it was already dark. So wearing my loose-fit pants outside was perfectly fine.
The past few weeks I slowly came out of hibernation mode (as Spring just keeps disappointing me). I went shopping on two or three occasions, a good excuse to wear more regular ‘can be seen in this by strangers’-trousers. And yesterday I had to ‘dress up’ for a visit to the exam centre in my home town.
In contrast to many people I didn’t gain corona kilo’s, I lost a few. The consequence is that my preferred pair of trousers for such occasions now require a belt. And after not wearing trousers requiring belts for many months, for the first time in my life I noticed how annoying belts actually are when in desperate need to pull down your trousers for a toilet visit.
The past two weeks I spent most of my time studying for the MS DA-100 exam, also known as ‘Analyzing Data with Microsoft Power BI’. This morning I took the exam and passed with a very decent score of 893/1000 (although I have to admit I was a bit annoyed not breaking the 900 barrier). After the training and passing the exam I am now skilled enough to start my own data analyzing projects. I’m looking for ideas where to apply my new skills.