Data: Difference between revisions
No edit summary |
No edit summary |
||
| (11 intermediate revisions by the same user not shown) | |||
| Line 8: | Line 8: | ||
=====September===== | =====September===== | ||
* (27) I imported 600MB of json inventory data into [[Technology:Databricks]] as my first poc. I learned that my data was incomplete 😱 | |||
=====October===== | =====October===== | ||
* (3) Trying to find a good starting point, I landed on https://www.reddit.com/r/dataengineering/ which lead me to https://dataengineering.wiki/Learning+Resources which lead me to https://dezoomcamp.streamlit.app/. I'm currently on Module 2 https://dezoomcamp.streamlit.app/Module%202%20Workflow%20Orchestration | |||
** I need to investigate what exactly the following things are, and how to use them (or if I want to use them) | |||
I need to investigate what exactly the following things are, and how to use them (or if I want to use them) | *** airtable | ||
*** maige (orchestration) | |||
*** streamlit.app | |||
** I'm wondering if this might be a better way to keep notes https://quartz.jzhao.xyz/ | |||
** I don't know where the Maige stuff is going to take me, I understand orchestration concepts I think I'll be better off | |||
** sticking to spark https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf | |||
Latest revision as of 21:25, 3 October 2024
My Data Journey
Started September 27th 2024 whilst attempting to learn Technology:Databricks
I want to document my journey here.
My Data Journal
2024
September
- (27) I imported 600MB of json inventory data into Technology:Databricks as my first poc. I learned that my data was incomplete 😱
October
- (3) Trying to find a good starting point, I landed on https://www.reddit.com/r/dataengineering/ which lead me to https://dataengineering.wiki/Learning+Resources which lead me to https://dezoomcamp.streamlit.app/. I'm currently on Module 2 https://dezoomcamp.streamlit.app/Module%202%20Workflow%20Orchestration
- I need to investigate what exactly the following things are, and how to use them (or if I want to use them)
- airtable
- maige (orchestration)
- streamlit.app
- I'm wondering if this might be a better way to keep notes https://quartz.jzhao.xyz/
- I don't know where the Maige stuff is going to take me, I understand orchestration concepts I think I'll be better off
- sticking to spark https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
- I need to investigate what exactly the following things are, and how to use them (or if I want to use them)