Jump to content

Tech learning roadmaps…


dasari4kntr

Recommended Posts

What is the famous 𝗖𝗔𝗣 𝗧𝗵𝗲𝗼𝗿𝗲𝗺 and what should you know about it as a Data Engineer?
 
As a modern Data Engineer you are most likely to work in the Cloud utilizing all kinds of Distributed technologies and Databases. Understanding CAP theorem will help you choose a Database that is fit for your application use case.
 
Let’s take a closer look.
 
First, let’s define what we mean by distributed:
 
➡️ A Cluster is involved.
➡️ Cluster is composed of multiple Nodes that communicate with each other through the Network.
➡️ Multiple Nodes could be deployed on a single Server but in most production setups you will see a Node per Server.
➡️ Data is distributed between multiple Nodes.
 
Now, let’s define what CAP stands for:
 
➡️ 𝗖𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝗰𝘆 - All Nodes in the cluster should see the same data. If we perform a read operation it should return data of the last write regardless of which node we are on.
➡️ 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - The Distributed System remains operational all of the time regardless of the state of individual nodes. The system continues to operate even with multiple nodes down.
➡️ 𝗣𝗮𝗿𝘁𝗶𝘁𝗶𝗼𝗻-𝗧𝗼𝗹𝗲𝗿𝗮𝗻𝗰𝗲 - If a Network Partition occurs (communication between some nodes in the cluster breaks) the Distributed System does not fail even if some messages are dropped or delayed.
 
The 𝗖𝗔𝗣 𝗧𝗵𝗲𝗼𝗿𝗲𝗺 states that you can only have a combination of two in the distributed system so your system is one of three:
 
👉 CA - Consistent and Available.
👉 AP - Available and Partition-Tolerant.
👉 CP - Consistent and Partition-Tolerant.
 
️ In real world you will not find many CA Systems as Network Partitions are common in Distributed Databases so they must be Partition-Tolerant. This means that we are usually balancing between AP and CP Systems.
 
Different databases are designed with different CAP guarantees. For example:
 
👉 MongoDB is a CP Database.
👉 Cassandra is a AP Database.
 
Having said this, you can also configure consistency on application level in Cassandra but more on that in the future posts. 𝗦𝗼 𝘀𝘁𝗮𝘆 𝘁𝘂𝗻𝗲𝗱 𝗶𝗻.

--------

Follow me to upskill in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space.

Also hit 🔔to stay notified about new content.
𝗗𝗼𝗻’𝘁 𝗳𝗼𝗿𝗴𝗲𝘁 𝘁𝗼 𝗹𝗶𝗸𝗲 💙, 𝘀𝗵𝗮𝗿𝗲 𝗮𝗻𝗱 𝗰𝗼𝗺𝗺𝗲𝗻𝘁!

Join a growing community of Data Professionals by subscribing to my 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿.

Link to comment
Share on other sites

Working in peoplesoft from 14 years and this technology became saturated with not many new implementations. Could anyone suggest technologies i can learn and switch my career?

Link to comment
Share on other sites

8 minutes ago, enigmatic said:

@dasari4kntr

any links/videos/courses to learn chatgpt or others to generate code, summarize docs, pdf's , convert from python to node etc. 

TIA 

discussed in this same thread bro..

check the below one…

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...