# Cache in System Design – Complete Guide | Redis, LRU, Consistency, Stampede, Hot Key **Channel:** Mohit Chhabra **Source:** https://www.youtube.com/watch?v=PiUu7DyBtWc **Transcript page:** https://www.withtranscript.ai/video/PiUu7DyBtWc ## Chapters - 0:00 — Cache Questions and Video Introduction - 1:53 — What Cache Is and Why It Is Fast - 8:16 — Why Not Store Everything In Cache - 10:38 — Multiple Cache Layers In Systems - 18:11 — Server Cache Versus Distributed Cache - 24:35 — Cache Hits, Misses, and Eviction - 35:38 — TTL and Data Freshness Control - 46:52 — Cache Write Strategies Explained - 56:26 — Choosing the Right Write Strategy - 1:06:08 — Redis Operations and Cache Data Model - 1:15:32 — Handling Cache Inconsistency - 1:30:08 — Cache Stampede and Its Solutions - 1:45:12 — Hot Key Problems and Fixes - 1:55:51 — When to Add Cache in Interviews - 2:03:55 — Closing Remarks and Outro ## Transcript **[0:00]:** What is a cache? How is cache fast? Cache is so fast. Why not keep everything inside the cache? Is cache only between the server and database? Why do we have so many types of cache here? Why do we need radius if database already has its own cache? Which one is more faster like the radius one or the in-memory cache? What is TTL? But why do you need to set the TTL? What are the different cache write strategies? Like when we talk about the cache between server and the database, is this one cache or there are multiple cache? How does that data look like? How do you keep data in cache? How do you fetch that data? How can we handle the inconsistency of data in the cache? Like we have a lot of servers, right? So the cache is sitting between the servers and the DB. **[0:48]:** But we have multiple servers. So how do we get the consistent data? The first one is what is cash stampede or the thundering herd problem in cash? How do we solve cash stamp? What is a hotkey problem? But what's the problem with hotkey? How come all the requests are going to particular shard? How do we solve the hotkey problem? When to bring the cash component in the interviews. Now after this let me tell you one of the basic mistake that people make related to cash in the interviews. Hello everyone, my name is Mohit Chabdra X Salesforce X Walmart and in today's video we are going to discuss about cash. So cash is an important component which is part of the system design and maybe if you're listening to this word for the first time it might sound weird to you but don't worry after watching this video you'll get **[1:38]:** complete understanding of cash. Now before we start this video, what I want from you is to make sure that you like this video, subscribe to this channel and share my videos with your friends and family so that I can create more such wonderful content for you. Okay, let's go back to the topic. **[1:53]:** Now, first question comes into mind is what is a cash? Now, first of all, the spelling is C A C H E and pronunciation you can say it like cash just like the cash or money, right? So, we say it cash. Now, what is a cash? In very simple terms, cash is a fast temporary storage. Okay, two keywords here. It is very fast and it is a temporary storage. Now let me explain cash to you in a very simple example. Okay, so here you can see the chef is trying to make pizza. Okay. And in order to make pizza, chef requires a lot of things like the pizza base is required, vegetables are required, some spices are required, some sauces or ketchups are required and much more. Correct? So we need the raw ingredients. So all of the ingredients are present in the kitchen. Right? **[2:40]:** But there are some ingredients which are required for mostly every dish and those are very frequently used like for example here the salt and pepper. Okay? So these are generally used in all of the dishes. If you have to make pizza, you will add salt to it. If you want to make pasta, even then you will add salt to it. If you are making noodles, even then you will add salt to it. Correct? So salt and pepper are the things which will be used in all of the dishes. Correct? So that's why chef has kept salt and pepper near to him. Otherwise if these were not present here, chef would go to this distance, take salt and pepper and come back and use in the dish. Next time making something else again go there pick salt and pepper come back and use in the dish. **[3:23]:** So every time if chef has to go there search for salt and pepper and then come back and use it in the dish it will take a lot of time correct. So in order to save some time chef has kept these things which are frequently used near to him so that you can directly pick and use it and then again keep it near to you. So clearly this approach is quite fast even you might have also seen it like if you do cooking right so you keep some of the things near to you which you know will be used frequently correct but you don't keep everything around you otherwise it will become messy right you have limited space around you and you want to just keep those things which are very frequently used correct so in similar way let's relate this example to our technical details okay so we know first of all we have the client right so let Let me put client here and then we know we have the servers in which we have our business logic. **[4:17]:** Correct? Now here in this example, chef is acting as a server because chef is preparing pizza the end response and server also provides you the end response. But in order to create pizza, chef needs raw ingredients. And similarly, in order to create the response, generally server would require the data. And data is present in the database. So let me put database here. So here in this example, chef is acting as a server and these ingredients are acting as the data and data is stored in the database. So these ingredients are present in the kitchen. So kitchen is acting as the database inside which we have the raw ingredients or the data. Okay. Now talking about salt and pepper as we said these are for the pasture access and we have limited storage around us. So this is acting as a cache the topic for today. **[5:13]:** So let me put cache here. So here comes the cache. The salt and pepper are acting as a cache. Basically we have salt and pepper present in the kitchen or the database but we are also keeping them near to us that is we are also keeping them in cache so that we can have quicker access to it instead of server going to the database always server can pick data from the cache which will be faster here you can easily understand because it is very near to you so it's quicker for you it's faster for you to access it instead of going there and searching for things you can simply pick it from here which is within your reach. Right? So in this example, you can clearly imagine this cache is quite fast. But when we talk about technical terms, how is cash fast? Because our definition mentioned that cash is a fast temporary storage. **[6:04]:** Okay. So how is it fast? Actually when we talk in technical terms okay the question is why cache is fast and the answer is like when we talk about database you can imagine it as like the hard disk or the SSDs like even in your laptop or personal computer you generally have storage like 512GB SSD or 1TB SSD right so similarly you can imagine the speed for database correct so when the server tries to get the response from the database it is just like you are getting response from the SSD or the hard disk and the response would generally come in milliseconds. So the server will get response from the database in milliseconds. But when we talk about cache, it is stored in RAM or in your memory. Okay, and RAM is very fast. Okay, when the server will try to get the response from the cache as cache is storing the things in the RAM and which is very quick. **[7:01]:** So the response will come in just nanconds. Why? Because RAM is much much faster as compared to SSD. So SSD is faster than the hard drives. But RAM storage is much much faster than the SSDs. Okay. So the response from cache is generally 100x or,000x faster than the database. So that's why the response from cache is very quick as it is storing the data in the RAM. What is the full form of RAM? Random access memory. So R A M and RAM is a temporary storage and it is very fast. Now because cache data is stored in RAM that's why cache is a fast temporary storage. Okay. So now you can clearly identify these two keywords fast and temporary storage. Okay. So as it also says the temporary storage. So here you need to keep thing in mind that database is persistent storage that once you have stored something in database it gives you durability persistence that data will stay there but cache is temporary storage although cache is **[8:10]:** quite fast as compared to database but this is temporary storage and this is permanent storage. **[8:16]:** Now one question may come into your mind like cache is so fast why not keep everything inside the cache. So there are multiple reasons why we can't do that. First reason is as cache is using RAM. So it is quite expensive. Okay. So you will only keep that data which is quite frequently used. Otherwise it won't be cost effective. Okay. So because RAM is expensive. So you will keep limited things in it which are very frequently used. Okay. Like the things which you keep on reading. So those thing you want to keep in cash. Another point is just see this example. Okay. So here cash is this amount of storage which is very near to you which you can access quickly. Okay. So this is limited storage. You can't keep all of the things here and even if you keep all the things here it will become messy right? **[9:07]:** So it will become again difficult for you to search for things. Say you have like 30 items present in this space then again you have to search for your thing. You can't simply pick the thing because there are so much stuff out there. Similar to that the main point of cache is that it is like quick access very fast but if you add so much data inside cache it will be difficult to find your thing quickly right so that's why we do not keep everything inside the cache and one more thing as I told you earlier cache is a temporary storage so it is not an alternate to database database provides you durability persistent storage correct whereas cache provides you temporary storage if cache goes down you will lose all the data which was present in cache. So you would always require database. So it's not an alternate for database. **[9:54]:** That's why you don't keep all the things inside cache. Now one question may come like how to imagine cache like what is it actually? You can simply imagine the structure for cache like a hashmap or the dictionary. Correct? You might have worked on hashmap, maps or dictionary in different programming languages, right? Maybe in Java, Python or any other programming language. So just like hashmap or dictionary cache has key value pair. So corresponding to a key you will maintain a value. Okay. So it has key value pairs. We will discuss more about it. I'll show you some examples through which you'll get more clarity. But for now you can just imagine it as like the structure that you maintain in the hashmaps or maps or the dictionaries. Correct? So key value pairs. **[10:38]:** Now because I have put cache here between the server and database. One question may come into your mind which is is cache only between the server and database. And the answer to this question is no. Cache can be present at multiple levels. So let me explain you that by going back to the diagram that we have here. Okay. So clearly you can see we have the client here, we have the server here, we have the database. Okay. So the first component that you see in this diagram is the client. So what does client mean? So any platform that the end user is using to use your product is acting as a client. So it can be the mobile app or the browser. So using the mobile app or the websites on the web browsers, the end user can use your product. Right? For example, if we talk about YouTube, the end user is using the YouTube product either via the YouTube mobile application or by the YouTube website on the web browser. **[11:34]:** Correct? So mobile app or browser act as a client. So the first kind of cache is the client side cache. Okay, the client can also use the cache to store the things which are frequently used because it gives quicker and faster access to the things to the client. Now where is this client side cache present? Inside the user's device. It can be in the mobile phone like it's an iPhone or the Android device or your web browser. Okay. So on the user's device you are caching the things. So basically the mobile app or the browser which are acting as a client they cache things like the images the API responses CSS or JS files or any data which is not dynamic. Correct. So example can be say you open any application. So first time you make a server API call like client gets the data from the server using the API call and if that data is somewhat static so what client can do is like keep that data inside the **[12:29]:** client side cache. So next time instead of making API call the client can simply show that instantly from the app memory. What's the use of that? You don't need to make a API call now and you can simply show that from the cache. Now because that is present inside the device only it will be very quick. Whereas if you make the API call that is a network call and there can be some delay due to it. Okay. So of course if you pick things from cache it will be very fast but you can't always do that because a lot of data that you get from the server or back end is generally dynamic which keeps on changing right so you can't always use the client side cache but there is still some data which is static. So for that static data you can use client side cache. **[13:14]:** Okay. So let me change color of this because we are representing cache by it. So this is kind of a first type of cache which is client side cache. Now apart from clientside cache when we talk about the client component there is one more component which acts similar to cache and that component is CDN. Okay. Now CDN stands for content delivery network CDN. Okay. So if you want to know in detail about CDN you can refer to this video of mine where I have talked in detail only specific to CDN. What is CDN? Where is it used? Why is it used? How is it different from other cache? So everything I have discussed in this video. What you can do is like after watching this video, you can refer to this video and you will get more clarity about CDN. But if I have to tell you in summary or in brief. **[14:04]:** So basically CDN acts as the edge servers which are present across the word and in those edge servers you keep static assets like images, videos and the use cases. As these CDN edge servers are present across the world, so they will be located quite near to the end user. So instead of the end user getting the images or videos or any static content from the back end layer, client will get it directly from the CDN and as CDN is near to the end user. So there will be very low latency in the response. So when we try to design any system which contains the use of images or videos like if we talk about YouTube because YouTube contains long videos. Similarly when we talk about Instagram in your user feed you see a lot of images and videos. So for all of those systems CDN come into picture. **[14:57]:** Okay, CDN will be used and the role of CDN is to cache the static assets like the images and videos and when the end user tries to fetch those images, videos, those are generally fetched from the CDN. So for CDN I have mentioned here edge cache because CDN has a lot of edge servers located in all the places in the world and these edge servers cache images, videos and other static assets. So here I have mentioned CDN which is the edge cache. It will be used for caching the images, videos or any static files. Okay. Again all of these are cache. So cache is very fast but at the same time it is like a temporary storage. So these are not alternate for your database. Okay. So when we talk about images and videos for durable persistent storage instead of database we store images and videos in the file storage. **[15:55]:** For example, Amazon S3 is used in which we generally store the images and videos. So instead of database, we store images and videos in the file storage like Amazon S3. Why do we do that? For that you can refer to this video of mine where I have talked in detail about like difference between database versus file storage. Why to store images, videos in the file storage and a detailed conversation about all the different types of databases and when to use which one. So you can refer to this storage. Now we won't talk about the file storage here because we are talking specific to cache in this video. Okay. So basically this client side storage was your first level of caching. Then we have the CDN which is kind of your second level of caching. So just like the CDN I have also put some information for the client side cache. **[16:42]:** So basically client side cache is like where your mobile app or browser cache the things inside the user device only. Whereas CDN as I told you are the multiple edge servers which are present all over the word and they are used specifically to cache static assets like images, videos or any other static files. Okay. So the overall flow would be if the end user is trying to fetch the images or videos say for user feed of Instagram or to watch a video on YouTube the client will get it from the CDN just in case those videos are not present in CDN. CDN will get it from the S3 storage. Why S3 storage? Because as I told you images and videos are stored in the file storage and we can use Amazon S3 as a file storage. Okay. So client will get it from CDN. In case it's not present in CDN, CDN will fetch it from S3, give it to the client and then store it in CDN as well. **[17:40]:** So that just in case next time if the same client or any other client ask for the same file, CDN will provide it directly without going to the S3. Okay. Why do we do that? Because if CDN goes to S3, that will be another network API call and there will be high latency. If you're getting data directly from CDN, it will be very fast because CDN are very near to the client and CDN has cached the data. Correct? So till now we have discussed two levels of caching. Client side cache and the CDN edge cache. **[18:11]:** Now the third level of caching would be at the server level. Okay. So basically inside the server we can have in-memory cache. Now what is the use of this? So the use is say for example you get a request from the client instead of going to database or to this cache what server can do is like get some of the data directly from in-memory cache which is present inside the server only and server can directly process this data or return it to the client. So of course this will be very very fast because this is present inside the server only. So you don't need to make any call from the server to database or server to outside cache. it can directly pick it from the in-memory cache that will be quite fast but generally this is not much used why the reason is when we talk about system design correct so we have distributed systems right so you **[19:04]:** don't have one server you have like multiple servers I have drawn two here but in reality you will have like many many servers correct because we are talking about distributed systems and if you want to know the basics of system design like why there are multiple servers. What you can do is like you can refer to these videos of mine and you will get complete clarity on the basics of system design. Okay. So go check out these videos after completing this video. So coming back to our topic. So in reality of distributed system we have multiple servers. Okay. So if we have in-memory cache then of course different different servers have their different different in-memory cache and there can be inconsistency among the data right some of the data might be missing here some of the data might be old here so all such kind of issues can come so this will be very quick but it is not mentioned when we talk about designing big systems or we don't use it that much okay so I'm just **[20:03]:** here telling you different different levels where cash can be present but after this I'll also tell you which of these are very important to remember and which you can simply ignore. Okay. So this was the third level which is inside the server only. The drawback of this is because we have multiple servers. So this is not distributed. This is like you have something for a particular server. But what about this server? This server is not having access to this. This server has its own in-memory cache and these in-memory cache can have inconsistency of data. So different data can be present in these and some of the data might be missing in one or some of the data might be old in some of the cache right. So we have consistency issues in this. Now the fourth one is database internal cache. Okay just like server can have the in-memory cache similar to that the database can also have like the internal cache. **[21:01]:** So this would be the internal cache which is present in the database. So example of database internal cache would be like for MySQL database we have MySQL buffer pool and for Postgress we have Postgress shared buffers. So these act as like the database internal cache. We will discuss about it but for now let's move to the fifth level of the cache. So the fifth one is this cache only which we designed in the first go. Okay. So this is the distributed cache. Okay. Like we had the in-memory cache here for server or inmemory internal cache for database. This is specific to a component. Okay. So this cache is specific to this server. Another server is not able to use it. Correct? So this is not distributed. But when we talk about this component, this is a distributed cache which means all of the servers can use it. **[21:52]:** If you have like hundreds of server, each of the server can fetch the data from this cache because this is a distributed system. Now how is it a distributed cache? because we use system like radius or memcache for this. Okay, radius is a distributed system. Don't worry about radius for now. I'll create a separate video on radius through which you'll get more clarity how it works and everything in detail about it. But for now, just remember radius is a very popular or famous system which is used for caching and much more things. Okay, not just for caching, it is used for many things. But one of a very big role of radius is for distributed caching. So the good part of radius is that one it is a distributed system. So you don't have to worry about things like single point of failure and also it can handle huge amount of load. **[22:43]:** Okay. So that's why for distributed cache this component we generally use radius. Now one question may come into your mind like why do we have so many types of cache here? The answer is the role of cache is that it provides faster access. So it will save time. So here you can clearly see cache is saving time at every layer. So because of that you get lesser load on the next component. So the deeper you go it will be more costly and it will take more time right. So if you can handle it in the initial stages using cache so it will be faster for you and lesser cost as well. Right? So at every layer you can see like clients try to make like quick loading of their UI. So they use like client side caching. Similarly in order to show images or videos to the end user quickly we use CDN. **[23:34]:** Just imagine if user has opened Instagram and it is just loading. It is taking so much time to show you the feed. The end user will become frustrated and stop using the application. Right? So for that we have the CDN. So all the static assets and the big assets like images, videos as those are mostly static those are cached by the CDN layer and then we have inmemory cache for server database and then this major component comes which is the distributed cache which sits between the server and the database. Okay. So as I told earlier we have talked about different different places where cache can be used. Now let's talk which of these are important and which of these you need to remember. Okay. So first coming to client side cache when we talk about system design generally we are talking about like the backend systems right. **[24:22]:** So like we are focusing on the highle design interview rounds right and in that generally you are asked questions like design a system like YouTube or Instagram. So that is heavily focused on the back end. **[24:35]:** So we can ignore the client side caching. Okay. Because we are focused on back end. Next coming to CDN. This is important. Okay. And if you see any of the videos that I've created like design YouTube, design Instagram and much more. So in those you will see I have used CDN. So CDN is important. Okay. So you can refer to this CDN video through which you'll get more clarity. Okay. Going to the third one server cache. As I told you this is not a great option when we talk about distributed systems because this is specific to the server. Correct? So you can ignore this one as well. Okay. Now coming to database internal cache. First let me give you a brief about this database internal cache because there might be doubt in your mind like if we have this why do we need this right so the basic question that comes into mind is **[25:23]:** like why do we need radius if database already has its own cache so first let's understand what is a database cache like I gave you example for my SQL and posgress where it has like buffer pool and shared buffers so what it does it like it caches the recently used rows like the basic point of cache is that it will cache frequently used things. Correct? So similarly for database it will cache the recently used rows and it will index the pages. So indexing is used to increase the speed for the response of the database query. So again for database you can refer to this video but coming back. So basically DB cache has this work. So the flow would become something like when a server makes a call to database, database will first check in the database memory or the cache and if it is not present, it will fetch from the database disk. **[26:15]:** Correct? And as we know the cache response is much much quicker as compared to the disk. So if you get data from this DB cache, it will be much faster. So this looks good. But the problem is the database cache has a lot of limitations. What are those limitations? Let's check it out. First limitation is that database cache is not controlled by us. Database cache is decided by the database. It's not you that who will decide what all to cache. Okay? So it's not in your control. Now point number two, even if you're getting data from the database cache, you are still putting load on the database. Okay? So for example, 10 million users open the homepage. Even if you're getting data from the database cache, the database is still getting hit. Correct? server is making call to the database. So the database is getting load. **[27:05]:** So database will get all those 10 million calls. So overall you are putting the load on database. Doesn't matter if you are getting it from the database cache or the database disk. Overall all the load is coming to the database right and your database cannot sustain that much load. Now the next point is radius is much cheaper to scale. So earlier I told you you can imagine the cache as just like a hashmap or a map or a dictionary which contains the key value pairs right. So the structure of radius will be it is simple key value pair. So there are no joins in it. There is no SQL parsing there are no log. Now these concepts like joins logs transactions these are part of database. Again I've discussed about database in this video. If you want to understand this you can refer to this video. **[27:53]:** But overall radius is like much simpler thing that you can understand like key value pairs and again the good part is radius is a distributed system and it can handle millions of queries per second. Okay. So we said like the database won't be able to sustain that kind of load but but radius is built in such a way that it is good for this and apart from this as we saw in the first point you don't have the control over database cache but when you have radius as your cache when you have that additional component between the server and the database in which we will use radius that is under your control okay so you can tell it what to cache and what not to cache and That is why here even though database has its own cache we need this component. Okay. So now that you have the basic understanding of what is database internal cache. **[28:48]:** Coming back to the point should we focus on it or can we ignore it? You can ignore it. Okay. I just gave you the fundamentals here so that you know things. Okay. But you don't need to remember this. Okay. So you can ignore this and you don't even need to tell about database internal cache in the system design interview. Okay. So after this we come to our distributed cache where we will use radius. Now this is important one. So you need to remember this part. So overall I told you like five places where cache can be used but you need to focus or remember only these two the CDN and this distributed cache because both of these will be used when you are designing the big systems. Okay you will see a lot of usage of CDN as well as this distributed cache. So when I now talk about cache, you can generally assume I'm talking about the distributed cache because in this video we are going to talk about like the cache between server and the database right CDN I have already **[29:50]:** discussed in this video and it has different use cases. So this is different from this one that two I have discussed in this video. Okay but in summary if I have to tell you the role of CDN is to cache the static assets only like images and videos. Just think in that term. If user wants to watch a video or an image for that we have CDN. Okay. Whereas this one it has a different role. It sits in the backend kind of layer. You can say this is not part of the backend layer. Right? Because client is directly interacting with CDN. Whereas when we talk about this distributed cache, it is part of this backend system. It comes in between server and the database. Okay. So it can also cache some of the dynamic data because we will keep some expiry of that and it will keep on changing. **[30:39]:** We can keep on updating because it is just key value pair and each of the key can only have one value right. So you can keep on updating that that is fine but these have very different roles. Okay. So in this video we'll discuss or focus on this distributed cache in which we will use radius and it sits between server and the database. Now although I said like you can ignore this one and this one question that may come into your mind is like which one is more faster like the radius one or the in-memory cache. The answer is in-memory cache would be much faster. Why? Because say it is part of the server only. If server wants to get some data from the in-memory cache that will be kind of inside same process. So in the same component you are getting the data. Okay. **[31:27]:** But if server has to fetch data from radius, radius is not part of the server. Correct? It is a distributed system. If it was the part of server then it would not be a distributed system. Then it would be inmemory cache. Correct? But we want to make it distributed so that all of the servers can use it. So if server wants to fetch some data from radius, it would be a network call. Correct? And of course network call would take more time as compared to fetching data in the same process. Right? So in order to fetch data from in-memory cache you don't need any network call but if server wants to fetch data from the radius which is a distributed cache of course you would require a network call. So this would be faster but because we are talking about the distributed systems you can ignore this we will focus on the distributed cache because when we talk about system design we are designing systems which can handle huge **[32:19]:** amount of scale. We are talking about distributed systems. Okay. So that's why our focus will be on the distributed cache. And whenever you talk about using cache in the interviews, you'll talk about this distributed cache. Okay. So the use case of this distributed cache would be instead of server fetching data from the database, it can fetch data from the cache. Why? Because you can treat the data in database like your data in the hard drives or the SSDs. Whereas cache you can treat it as like your RAM memory which is very fast as compared to the SSDs. Okay. So if the server gets data from cache it will be very fast as compared to getting data from the database. So this component helps in reducing the latency. The queries which are made by the server you'll get the response very quickly if it is present in cache. **[33:13]:** So two terms here if server tries to get data from cache and if that data is present inside the cache cache will directly return data to the server without going to database and that case will be called as cache hit. Okay so cache hit is server tries to get data from cache and that data was present in cache and cache returned that data directly to server without even going to database that is cache hit. Let's talk about another scenario. Server tried to fetch data from cache but that data was not present in the cache. So that case is called as cache miss because there was a miss right that data was not present in the cache. So it was a miss. So good part is like you can remember it by the name basic English it says like it was a miss or hit. Hit means data was present in cache. **[34:06]:** Miss means it wasn't present in the cache. Right? So of course if it was a cache hit which means data was present in cache the response will be very quick because you are serving from the RAM but in case of a miss it will be slow why because now cache will fetch the data from database and then give it to the server. So in cache miss case query will take longer time right. What happens after cache miss because cache fetched the data from the database and gave it to the server at the same time it will now keep that data in the cache so that next time if you get the request for same data you can serve it directly from the cache. Okay. So just in case imagine server tried to fetch data from the cache it was not present right. So that was cache miss cache will fetch that data from the database. **[34:56]:** It will give it to the server and at the same time it will add that part in the cache. So now that data is part of the cache. Okay. Next time any server tries to get the same data. It will be a cache hit because now cache has that data. Okay. And cache will return that data directly to the server and it will not go to database. So because it was a cache hit now the response of the query will be very fast. Okay. So if you have data in the cache it is a cache hit. If you don't have required data in the cache it is a cache miss. And if it's a cache miss cache adds that data in the cache for the next time. Okay. So that next time you can serve that data directly from cache. **[35:38]:** So the overall flow here would be something like from the client we'll go to the server and from server in order to get the data we'll first try in the cache. In case data is present in cache that will be a cache hit and we can get the data directly from the cache and that will be very fast. Okay just in case data is not present in cache that would be cache miss and in that case cache will bring the data from the database and then provide it to the server. Okay. Now as we mentioned earlier that in cash we will keep limited stuff right because it is expensive and we don't want to keep a lot of things in it otherwise your search will become difficult it will not remain that quick right so now let's talk about cash eviction because if we have put something in the cash we also need to make it out of the cash as we will store limited things in cash only. **[36:26]:** So the first question comes why do we need cash eviction? So in cash we will have some fixed limited storage. Okay. So as we said earlier it is expensive. So we won't keep everything inside it. We will only keep those things inside the cache which will be frequently used or read. Okay. So now we have limited space in cash. We can't put everything inside it. So in case the cash is full, we need to evict some of the things from it so that new things can come inside the cash. So that's why we need cash eviction. Now the question comes what are the most common cash eviction policies? All right. The most famous one is LRU which means least recently used. So from the cache we will remove the least frequently used item first. For example, let's say the size of the cache was three. **[37:15]:** Okay. Now let's say we get a number two. Okay. Because the size was three and we have the space in cash. So we will put two here. Okay. So basically we are doing insertion at the beginning. Okay. Now let's say after this we get another number one. So our cache will look something like this. Instead of two being at the beginning it will come at the next position and in the beginning we will have one. So let's try to understand example of LRU. Okay. Let's say this is our cache which has fixed size of three. Okay. Now in beginning we have first number as two. Now we will do insertion at the beginning. So it will look something like this because we had the space. So it will look two in the beginning. Okay. Say next one comes. Okay. So again as I said the insertion will happen at the beginning. **[38:05]:** So it will look something like 1 2. And now say three comes. Okay. So it will look something like 3 1 and two. Okay. Now you can see the size of cache is full. Let's say one comes again. Okay. Now although the size was full but one was part of this cache. So instead of this what will happen is one will come at the beginning because it is now recently used. So one will come here, three will come here and two will stay here. Okay. Now say after this we have four. Okay. Now the size of the cash was full. So we need to do the eviction and here we are using LRU that is least recently used. So clearly two is least recently used here because it is present at the end. We are basically inserting at the beginning right and you can also see when one came again we changed its order. **[38:58]:** So one came in front and then three and then two. So if we have to remove now so two is the least recently used. So we will remove two and now because four is insertion and insertion needs to happen at the beginning. So it will look something like four 1 and then three. Okay. So this is how the least recently used will look like. Don't worry about this part like which data structure are we using and how are we doing it because that comes into low-level design kind of question and I'll cover that in a separate video. But for now just imagine like how things are happening. The main policy was like least recently used. So the thing which is least recently used should be evicted in case we need to evict something. We evicted only when the size was full. So the cache size was three and here three elements were present and now a new element was waiting to be inserted and that's why we did cash eviction using LRU here two was least recently used clearly you can **[39:59]:** see first we had two then one three so till now also two was least recently used then one appeared again okay so we changed the priority of one here so one came into beginning and even after this all two was least recently used because It came in the most beginning and after that it was never used. Just imagine it as like people are trying to read something from the cache. So first time there was nothing in the cache so it was a cache miss. Correct? So it picked from the DB and inserted in the cache. Next time also it was not present in the cache. So again a cache miss. So it was inserted in the cache then. Right. But how is it inserted? We are doing insertions at the beginning of the cache. So one came here, two came here. Okay. Then came three. **[40:45]:** Again a cash miss because three was not present here. We fetch it from DB and then put it in the cache. So 3 1 2. Now again one. Now it's a cache hit. Okay. Because one was already present in the cache. Now this is a cache hit. So we will return directly from the cache without going to the database. But at the same time because it is now more frequently used. So we will change its ordering. One will appear here. Now because it's a kind of you can say key value pair thing. the cache as I told earlier even in the map hashmaps or dictionary you have the unique keys correct so one will appear only once here we can't do it like one is coming twice here because it's representing the key and key will be unique so one was already present we just change the ordering here 1 3 2 becomes and then next time if four was read that was not present in the cache so it was a **[41:36]:** cache miss but it will fetch it from the database but now it needs to add that in the cache as well but you can clearly See the size was full. So we need to do cash eviction and we removed two using the least recently used cash eviction policy. Right? Now least recently used eviction policy is the most famous one. It is used in most of the places. But apart from this also there are many other like least frequently used. So the concept in least frequently used is that that you will keep the count of the frequency of the usage of the items like how many times the item was used. So basically you will maintain the frequency of the access and when you need to evict something you will remove that item which has the lowest frequency usage. Now apart from this we also have FIFO which is first in first out. **[42:24]:** So it will evict the oldest item first. Okay. And like this we have many more cash eviction policies but the most important most used and most asked in the interviews is this LRU one. So you need to remember this one least recently used and you will notice that in most of the places LRU is used as a cash eviction policy. So now that we are discussing about cash eviction let's discuss about one more important topic related to cash which is TTL. Okay. And now the question will come into your mind what is TTL? So first of all the full form of TTL is time to live. So how it will work is you will first of all set TTL. Say you have set TTL as X minutes. So it is acting as like time based expiry. So the cash entry will expire after x minutes because you have set TTL as X minutes. **[43:16]:** So kind of you are setting expiry of the entry in the cache. But why do you need to set the TTL? So let's try to understand that. Okay. So this was your server, right? And this was your cache and this is your database. Okay. So we said like when server wants to read something instead of going to the database it will first check with the cache right if the entry is present in the cache it will be a cache hit and the data will be returned directly via the cache correct but at the same time when we talk about the rights of course there are multiple strategies that we will discuss just after this but the generic strategy is like you will write in the database okay so write operations are happening at the database but the reads are happening from the cache correct So there can be something like for example for a key the value was present in cache **[44:08]:** as three right so this was present in cache but you updated the database with this value as five okay so next time the server tries to read this data it will read it from cache right because it will be faster to read and the value is present in cache so this is a cache hit hence cache will not go to the database so it will return value as three which is the stale data or the old data A. So you can clearly see the data inconsistency. The updated value is five but we are returning three because value is not currently updated in the cache. Right? So this kind of situation can happen because database is your actual source of truth where you have all the data. Cache is just acting as a temporary storage through which you'll get the fast response. Right? So there can be data inconsistency. **[44:56]:** So generally in order to get fresh data after certain while you set TTL. So of course there is a way to remove this kind of inconsistency that you can invalidate the cache right and if you will invalidate the cache this entry will go away and when the server tries to fetch the data from cache it will be a cache miss because you have invalidated the data from the cache. So cache will pick it from the database and it will get the new value which is five. So cache will return five to the server and at the same time it will update the value from 3 to 5. Right? But TTL is used so that even if you forget to invalidate the old data will expire or disappear after the duration that you have set in TTL. Okay. So here in this case if you have set TTL as like 10 minutes. **[45:44]:** Okay. So what will happen is like automatically this entry where three was present will be removed from the cache after 10 minutes as we have set TTL for this entry as 10 minutes and TTL acts as the expiry. So after this duration this entry will be removed from the cache. Okay. So TTL is acting as a automatic eviction but the duration is provided by us. So we can set the duration for TTL. So here the worst case scenario is server may get the old data for maximum 10 minutes because after 10 minutes this entry will be removed from the cache as a TTL for this entry was set to 10 minutes. Okay. So maximum 10 minutes you will see the old or stale data but after that this entry will be removed from the cache. So generally TTL is used everywhere as a safety net so that after a while you always fetch the fresh data. **[46:36]:** Now of course but if your data is dynamic you will keep smaller TTL. so that it get refreshed quickly but if your data is static you keep longer details because you don't need to refresh the data frequent as the data is static now here in this case I discussed about the right **[46:52]:** operation and I mentioned that there can be different different strategies right like will the right operation directly happen to the database or when will we write to the cache okay so in order to understand that let's discuss about the cache write strategies so the question comes what are the different cache write strategies ies. So basically the cache write strategy is define like how will you write things in the cache and database and that will define the consistency between the cache and the database. So basically what cache write strategies represent is like when a right operation happens how do we update cache and database and based on a strategy the consistency of data will happen between the cache and database. Okay. Now there are four cash right strategies but I'll tell you which one is the most important and which one to remember. Okay. All right. **[47:42]:** The first strategy is cash aside or this is also lazy loading and this is the most important one. This is most commonly used and you need to remember this one mainly as cash aside is used in most of the real systems. Now let's see how it works. So in case of a read we already know server will first check with the cache. If data is present in the cache it will be served directly from the cache which is the case of cache hit. But just in case the data is not present in the cache that will be cache miss. What cache will do is like it will fetch the data from database give it to the server and at the same time it will add that entry in the cache. So you can see here the server will check the cache. If data is found, you return the data directly from the cache. **[48:30]:** If data is not found, you read from database. So you are checking from database. Now put in the cache and return the data. Correct? So you are putting in the cache for the next time. So that next time if same data comes, you can directly return it from the cache. Now of course we know the case of read. The interesting case is of write. So in case of write, what happens is the server will update the database. Okay? It will not update the cache. Server will update the database. And then further what we will do is like invalidate or delete those keys from the cache. Whatever thing that we are writing, write operation can be the insert operation or the update operation. So say for example, we are updating the value of X. Okay. And X was present in cache as say it was present as 10. **[49:20]:** Okay. Earlier in database also it was 10. Now we updated the value to 20 right? So we have updated in the database but we saw earlier in this case if we don't update the data in cache then there will be data inconsistency. Correct? So in order to avoid that what we will do is like we will invalidate this key from the cache. We are not invalidating everything from the cache. We are simply invalidating that particular key on which the right operation has been performed. Okay. Because we are writing X here. So we have invalidated X from the cache and so next time when server tries to read from the cache that entry will not be present in the cache as we have already invalidated that. So that will be the case of cache miss. So what cache will do is like it will fetch it from the database and hence cache will now get the updated value. **[50:13]:** It will keep that value x equal to 20 in the cache and will return that to the server. Okay. And that is why it is also called lazy loading because when you get the read operation call from the server to cache at that time that value will not be present as we have already invalidated that or deleted that. So it will have to fetch it from the database. So it is lazy loading when the read operation call comes. So you will fetch it lazily when the read operation call comes and before that you are not getting that value in the cache. Okay. So the main point to remember here is when we have the write operation write will be performed in the database not cache and additionally what we will do is like we will invalidate or delete that entry from the cache and this is the most important one. **[50:59]:** So generally whenever you will use the distributed cache like radius and when you tell about that in the system design interview you will use this cache aside approach. You don't need to remember the name. Actually, it is fine if you don't remember the name, but you should remember the strategy. Okay? And strategy is very simple. You will update the database and you will invalidate or delete that key from the cache so that there is no data inconsistency. Now, let's see an example for cache aside. Okay. So, say we are updating the user profile. Okay. So when we are updating the user profile what we are doing is we are updating the database because our first step was to update the database and we are not updating the cache here. We are simply updating the database and the next step would be we will delete that we will delete that key from the cache. **[51:50]:** We will use radius as I told you earlier radius will be our distributed cache that we will be using. So we will do radius dot delete and we will pass the key. The key here is user with ID 1 2 3. Okay. So whatever id of the user is there you are specifically removing the data related to that user. As I told you earlier we are not deleting everything from cache because other data we need right. We are only removing that key from cache which has been newly written. Right? So any update that we have done to the database in order to avoid data inconsistency. So we will delete that specific key from the cache. So whenever next read operation happens there will be cache miss because server will go to the cache and that data will not be present in cache because we have deleted that from the cache right so that will be **[52:41]:** cache miss and then cache will fetch it from the database and it will have the new data and it will return the new data to the server. Okay now let's see some of the pros and cons of this strategy. So the pros are it is simple because we are writing to the database that is our strategy. We are not writing to the cache and we are simply invalidating those keys from the cache. So you can clearly see here it is quite simple because performing this operation in radius is very simple. It is very similar to as I told you like deleting or removing the key from a map hashmap or a dictionary. Right? In similar way we can delete it very easily. Here it is safe. Database is always source of truth because we are always writing to the database. So DB is acting as the source of truth. **[53:26]:** It is easy to scale and it works with multiple servers. Now talking about the cons, there will be a small inconsistency window. Now you will say why there will be inconsistency. We are removing the key from the cache. So why there will be inconsistency of data? So that's why we have mentioned small inconsistency window. So basically you can see we have two steps here. Correct? So first step is you will update the database and second step is you will delete the cache. So because when we talk about distributed system and huge amount of load there can be a case like in between this duration. Okay when we are performing these steps say first step is done and when we are performing the second step during that time you may get a read call and that may go to the cache that may fetch the old data. **[54:14]:** Okay. So that will be a very small window because we are performing these steps. But yeah, there can be a small inconsistency window. Okay, that if a read request comes while we are still in progress of this action, right? And second con, the first read after delete is slow. Yeah, which is understandable as this is lazy loading now that we have removed the key from the cache. So whenever a read request comes that will be a cache miss, right? So you can't get that data directly from cache and hence there will be some latency as the cache will fetch it from the database right but overall this is the most important one and you need to remember this one mainly I'll discuss about other strategies as well so that you get understanding okay how things work but this is the one that you need to remember okay now this one might still confuse you a bit like you **[55:07]:** might think like we have done these steps then how can there be small inconsistency in between. Right? So the reason for this is like these steps are not atomic. So there can be a time gap in between these. Okay? So when we talk about distributed systems when there are lots of load there can be multiple scenarios related to multi-threading. Like I have discussed about multi-threading in detail in these videos. So you can refer to these and you'll get some understanding about it. But the basic point is there can be a scenario that you have done this operation but before you complete this there can be a read request coming in between okay as we are talking about huge amount of load and multi-threaded cases. So there can be a scenario that a read request happens before you complete this task. Correct. And there can be small inconsistency window. **[55:57]:** Now of course when we talk about big systems there are approaches which we can use to handle this small inconsistency window. We can use short expiry or TTL. We can use versioning and we can also use double delete strategy which we are going to discuss later in this video. But overall you don't need to remember all that stuff. Basically I'm telling like there are approaches to handle this small inconsistency window. Okay. And by default we always use cash aside strategy. **[56:26]:** Now after this I'll also tell you some more strategies. Okay, you don't need to remember all of these. I'm just telling you so that you have idea about it. Okay. So the second one is write through cache strategy. So in this what we do is like whenever there is a case of write the application will write to the cache. So instead of writing to the database, we are writing to the cache. Okay. And what cache will do is like cache will synchronously write to the database. Okay. So basically this is you can say your server you have this cache here and here you have the database. Okay. So what we are saying is like whenever the write comes write will happen to the cache first and cache will write it to the database and this step will be synchronous. So overall there will be latency in this operation because we are writing in cache and we are also writing in the database. **[57:18]:** And when we say synchronous, it means that the response will need to wait till we complete our work. So until we have not written to both cache and database, we won't send kind of success for this operation. Okay. So this was the flow for write when we talk about read, we will always read from the cache and there will be no data inconsistency because we are also writing in the cache and the database. So you can always read it directly from the cache and you will get the fresh data every time. Okay. So this is a flow from the application or server you are writing directly to the cache and then cache is writing to the database. So that's why the name is write through cache. So your write operation is happening through the cache. So first the data is written in the cache and then in the database. **[58:02]:** So the right operation is happening through the cache. Okay. Now let's see some of the pros and cons for this strategy. First good point is that cache is always fresh. Why? Because we are writing to the cache first and then to the database. So every write operation also goes through the cache and that's why all the data in cache will be totally fresh. Okay. Next pro point is no cache miss after right because you have written that data in the cache. So there won't be cache miss for that particular key because you have just written it in the cache. Of course after the TTL there can be cash miss but when you write in database and when you read within the TTL duration there won't be any cash miss for that because you are writing first in cache and then in database. Okay. And there will be strong consistency. **[58:46]:** There is no inconsistency here because the data is same in cache and the database because we are writing in both in the cache and the database. So these are the pros but there are cons as well. First con is for the right operation latency increases. Why? Because you are doing the right operation first in cache and then in database. So the second step that we are doing we are doing it synchronously. So we are not returning the success just after writing in the cache. Okay. we are also updating the database at that time. Only when both of these are successful then we return the success for that operation. Okay. So there will be high latency for the right operation. Okay. Another big con of this is you can clearly see we are always writing in the cache. So there can be a case that we never read that data from the cache. **[59:33]:** Correct? But we are still writing it in the cache. So we are not utilizing the cache properly because we are also writing some of the things which might never be read from the cache. Correct. But when can you use this kind of system? When you need strong consistency. Okay. There are systems where we talk about strong consistency. Like you can refer to this video related to database. You'll get more clarity or idea about it. But this can be used as one of the strategy when you need strong consistency like banking transactions. But overall this is not that highly used. All right. After this we have one more strategy which is write behind or write back cache. So what we do in this is like app will write to the cache. Cache will respond immediately and cache will write to the database asynchronously. Okay. In the second approach we said that the cache will write to the database synchronously which means that the operation will not be complete till we have not written in cache as well as in the database. **[1:00:33]:** Okay. So overall the write operation took a huge amount of time. So this was write through. But when we talk about write behind we are again writing to the cache. What we are doing is like once the data is written in the cache we will respond immediately. We will not wait for updating that to the DB. That operation of updating to the DB will be happening asynchronously. Okay. So basically again if this is your server, this is your cache, this is your DB, right? So what will happen is like whenever a right operation comes the we will perform the right operation in the cache and we will return the success immediately once this part is done. Okay. And in background asynchronously what we will do is like we will also update the database. But this step will not impact the success that we have returned to the right operation. **[1:01:24]:** Okay. So the difference between write behind and the write through cache strategy is that in write through cache we waited for updating the database as well from the cache. Whereas here when we simply update the cache we will return the success for that operation and asynchronously we will update the database. Right? So you can clearly see in the right operation we will directly update the cache and we will return the success and asynchronously we will update the database. Now let's see some of the pros and cons related to write behind cache strategy. First pro is that we have extremely fast rights because we are not waiting for updating the database. So it will be fast. Okay. And this will be good for right heavy system. Right heavy system means you have a lot of write operations in your system. Right? For that part you're simply updating the cache and giving the success for that operation. **[1:02:16]:** So you have very low latency when we are talking about the right operation. Okay. So it is good for the right heavy systems and another good part is you are reducing the load on the database because we are updating the database asynchronously and what we can do is like we can do the batch rightes. What does that mean? It means like for example you had like five writes correct 1 2 3 and two more. Okay, in the first approach that we talked about like write through cache, what you will do is like for every write operation you will update the cache and you will also write it in the database. So you will perform the right operation five times because this happened five times, right? But here in this write behind approach because you are updating the database asynchronously. What you can also do is like you can batch all of these like five things. **[1:03:06]:** You can update the DB in just one update operation. So you are writing it once but you are doing like the batch write. Okay. So you are updating or writing in database in batch. Correct. So batch rights is a technique which is used in a lot of places. You can refer to this video of mine in database where you will get more idea about what are the batch rights and where is it used. Okay. So refer this video for more details. When we talk about cons for this strategy. So there is the risk of data loss if the cache crashes. Now because we were updating the database asynchronously and you know cache is not your permanent storage right. So there can be a scenario that you got a right operation request and you updated in the cache and you return success and before updating it in the database because you have to do that asynchronously your cache crashed **[1:04:01]:** and once and once it is crashed all the data that it had would be gone right. So that right operation that you had to perform on the database will be lost. So there is a risk of data loss. Correct? And in order to handle all those scenarios, we will have complex recovery logic which also comes as a con for this strategy. And one more con would be there can be eventual consistency. There will not be strong consistency because the data in cache might be different from the data in the database. Right? So there would be eventual consistency here. Okay. Now when would you use this kind of strategy? when your system is like right heavy. Correct? Because then you would do batch rights. Okay. But overall apart from these three also there are few more cache right strategies like you can clearly see we have different different ways to update the things right. **[1:04:51]:** In one way we are saying like we will simply update the database and we will evict the cache. Okay. In another way we are saying we will update the cache and from cache we will also update the DB but that would be synchronous. And in another strategy we are saying we will update the cache and we will update the DB asynchronously. In similar way you can have few more approaches as well. Like for example we discussed the first one where we were invalidating the cache. Right. There can be one more approach. You are simply writing it on the database but you are not doing anything for the cache. Okay. Let that old data remain in the cache and after TTL it will expire and you will get the new data. And similar to that you can have few more strategies as well. But the main point is you do not need to remember all of these like the names and everything because it would become difficult for you to remember or memorize everything. **[1:05:41]:** So you don't need to memorize all these things. The most important thing that you need to remember is the logic that we discussed about the cache aside strategy or the lazy loading. Basically what we are doing is like we are updating the database on every right and we are simply invalidating the cache or deleting the data for that specific key from the cache. Okay. So this is the most important one. This is most commonly used as well. **[1:06:08]:** And um even if you don't remember the name of this strategy like cash aside or you forget this name it's totally fine. If you remember this logic like what is the default logic that we generally use that is totally fine. You can always say we can use this kind of strategy. Okay. Now one question that may come into your mind is like when we talk about the cache between server and the database is this one cache or there are multiple cache. So the answer is as I mentioned earlier here we will use radius which is distributed cache. So logically you can consider it as one cache but physically it contains multiple machines which is also called as a cache cluster. So that's why we have mentioned here distributed cache. So there is no single point of failure here. So in the radius we have multiple nodes. **[1:06:56]:** So you will have node one, node 2, node 3 and much more because in system design we want like each and every component should avoid single point of failure. Right? And similarly when we talk about this cache between server and database here radius is the distributed cache which contains multiple machines. So as we have mentioned multiple times that we will use radius for this cache which is a distributed cache. So one question may come like how does that data look like? How do you keep data in cache? How do you fetch that data? So let's understand that. So we are taking an example. We are talking about the user profile and this is the user ID 1 2 3. And first we will try to see a case. We are trying to get the user profile and it's a cache miss case. So as we know in cache miss we don't have data present in the cache. **[1:07:43]:** Right? So let's see how do you try to fetch the data from radius. So here you can see we have written radius.get. So get is a simple method from which you can get or fetch the data. As we discussed earlier in radius we keep data in key value format. Right? So corresponding to key you have the value. So in the get method we pass the key and when we pass the key in the get method corresponding to that key we get the value from the radius. Okay. So here you can see the key is user col 1 2 3 and from this you can understand how can we generate the keys. This is a good key because it tells you that we are talking about user and then we are passing the ID. Okay. So like this we can create many other keys and it's important that we keep a good key because in radius as we said the data is stored in key value pair right. **[1:08:36]:** So it's important that you keep a good structure for the key. Okay. So using the get method you pass the key in it and you get the value. It is very similar to like we already discussed like a map or a dictionary or a hashmap right. You might have already worked in Java or Python or any other programming language on key value pair. So similarly for radius is quite similar to that. So here because we were talking about the case of cache miss. So hence for this key we won't have value present in the radius cache. So if data was present we would have returned the data directly from the radius and that would have been the case of cache hit but because we are talking about the cache miss this won't be executed and hence we will go here okay so we are trying to fetch data from the database now so we are doing db dotquery and we pass the query there so in the query we have written for example select star from **[1:09:31]:** users where id equal to 1 2 3 star means we want all the columns users here is representing the table name and using the where we are providing the condition. So if you want to know more about database you can refer to this video and you will get complete knowledge about the database. Okay. So here we are fetching the data from the database because this was a case of cache miss and once we get that data we will store that data in redisk cache. Why? So that if the same key is asked in new request we can return the data directly from the cache. Because this step is of course expensive. You are fetching data from database and the role of cache is that we want to return the data quickly, right? So we will add this data in the radius. How do you do that? **[1:10:19]:** You will simply do radius dot set. So earlier get was a method to get the data and set is the method to set the data. Quite simple, right? In the set we will pass like key and value. So the key was user col 1 23 and then value is the data which we have passed from the DB. So that is a value and apart from that you can also pass TTL which is time to live or expiry of this data. So for example here we have set TTL to 10 minutes. So after 10 minutes this data will be removed from the cache. We do that so that we get the fresh data after some time. That is also done to maintain the consistency because if you keep the data for long time in the cache it might be possible that you are returning the old or stale data to the end user right so that's **[1:11:10]:** why we said time to live or the expiry and after that we will return the data okay so this was a case of cache miss here we have seen in order to get the data from radius you simply use this method get and you pass the key inside that method when you want to add some data in the cache you simply use the method set. Inside that you put the key and the value and apart from that you can also set the TTL which is the time to live or expiry for that data. Here we were talking about cache miss which means data was not present in the cache and we fetched it from the database. But once we fetch the data from the database we also put that in the cache so that next time we can return the data from the cache itself. **[1:11:53]:** So after this say a new request comes for the same key which was user col 1 2 3. So next time what will happen is we will directly return the data from the cache. It will be the case of cache hit because now data was present in the cache. So here we saw two main methods related to radius. One was the get through which you will get the value and one was the set method through which you will add data to the cache. Okay. Apart from that there is one more method using which you can delete data related to some key from the cache. Of course you have also set the TTL which means after 10 minutes this data will be automatically removed from the cache. But just in case if you want to remove the data before this TTL you can use this method delete. **[1:12:42]:** Why would you do that? I'll tell you one example. So here in this example you can see we have added the data related to key user col 1 2 3 in the radis cache correct. So next time if the request comes for the same user key before 10 minutes so you will simply return the data from the cache and you won't go to the database correct but say during those 10 minutes we update the data per user id 1 2 3 okay so here you can see we have written the update query okay so we are doing db.tupdate update. So we are updating the database. We have written here update to update the database. Users is the table name. Okay. And then we are writing set name equal to any value where id equal to 1 2 3. Okay. So basically we are updating the name for the ID 1 2 3. **[1:13:32]:** Right? Again to know more about database refer to this video you'll get complete clarity. Okay. But just to give you a summary here basically we are updating the value in the database. So user id 1 2 3 we are updating the username. Okay. So instead of question mark any value can be present like you are updating the name for user ID 1 2 3 as muhit or you can update to any name right. So once you do that what will happen is like the information which was present in radius will become old or stale data because radius contains old name for user ID 1 2 3. But clearly we can see we have updated the name for user ID 1 2 3. So next time if the request comes within 10 minutes, we would be returning the old username which is wrong. Right? So should we now wait for 10 minutes? **[1:14:21]:** One better approach can be you can simply delete the data for that key from the cache. So here you can clearly see we have done radius.delete. We have used the delete method and inside delete we have passed the user key which was user col 1 2 3. So this will remove the data for this key from the redisk cache. It won't remove the complete data. It will simply remove the data related to this key from the cache. Okay? Because we have passed the key inside the delete method. So with this what will happen is next time if you get a request for user ID user col 1 2 3 as we have deleted the data so the data will not be present in the cache and that would be the case of cache miss. And again we will fetch it from the database and insert it in the radius cache and then return to the user. **[1:15:12]:** So using this whenever the get request will come you will get the updated or the fresh data from the database and then keep it in the cache. So this is cache side pattern or the lazy loading which we discussed earlier in this video. Right? This is done so that data remains consistent and you don't return the old data from the cache. **[1:15:32]:** So basically using this example we got to know how data is stored in radius. So the data is stored in key value format right for example the key can be video col 987 colon likes. So basically you are here trying to store the likes for video ID 987. Okay. So here you can clearly see using this way you can keep a good structure for the key. Okay. If you maintain a good key structure or key ID then it will be very useful because you have to fetch data from key right. So it's very important that you keep the right structure for the key. Here you can clearly see it gives you a good information that we are talking about video with ID 987. This is the ID. Okay. So you can replace it with whatever video ID we are talking about. And then we say likes. **[1:16:23]:** So it tells you for this video ID, we want to know the like count, right? So this is a good key structure. You should also make your key structure or key ID through which you get the good information what this key is about. Right? So clearly it says about the video with ID this and we are talking about the like count. So here we were talking about video 987 instead of this. If you were talking about video number say 800 then you would replace 800 with 987 here. Okay. But here as we said like video 987 likes. So it will give you the like count for the video ID 987. Okay. And the value which would be present to this key would be the total number of likes. And when you try to get the value for this key, you'll get this value from the radius. **[1:17:14]:** How do you get that? Using this get method. Inside the get method, you will pass the key which is this one here. Right? So data is stored in key value format in radius. You can get the value using the get method. You can set the value using set method. And you can delete data for particular key using the delete method. Quite simple right now. In this example, we have said like value as the like count which is integer you can say here. So a question comes into mind what all can be present in the value part for the radis cache. So in the value part you can keep integers, you can keep booleans and not only that you can also keep like JSON objects in that as well. For example, if we see these cases here, the first key is user col 1 2 3 corresponding to that this data is stored. **[1:18:03]:** We have like the name of the user, the age of the user, the photo of the user. Here photo would contain the URL. Okay, generally images and videos are stored in Amazon S3 which is the file storage. You can refer to this video to get more information about that. Here we would keep the photo URL. But you can clearly see in the value it's not like you are just keeping an integer or just a string. Here we are keeping a lot of information in the value part corresponding to this key. Similarly, this can be another key which the key here is video colon 99. So we are talking about video with ID 99. Right? So it contains information like what is the title of the video, what is the description of the video, what is the URL of the video. So clearly you can see in the value part we can have a lot of information. **[1:18:53]:** So we are keeping like a JSON object in the value. And another example is the key is feed colon 1 2 3. So we are talking about the user feed with ID 1 2 3. Okay. Or you can say the user feed of the user with ID 1 2 3. Right? And corresponding to it we have maintained an array in the value. So we have here list of videos 1 video 2 video 3. Just like in Instagram you generally see a lot of post which contains images of videos. So here you contain the list of videos means that we have the URL of the video and original video is stored in Amazon S3. You can refer to this video to get more clarity. Okay. So but the main point here is corresponding to the key in the value you can keep a lot of complex data as well like it's not just you can keep a string or integer boolean apart from that you **[1:19:48]:** can also store like JSON objects or arrays in the value so it is quite useful so here we mentioned about the data inconsistency right data can be different in cache versus in database so the question comes like how can we handle the inconsistency of data in the cache and when we talk about cache here we are talking about the cache between the server and DB so the radis cache we are talking about now there are multiple techniques or approaches through which we can handle the inconsistency of data between the database and the cache let's see that one by one so the first approach is using cache aside pattern we discussed about cache aside pattern earlier in this video but to revise it basically we treat the database as the source of truth here. So basically whenever we talk about the update flow whenever you update the database what we do is like we delete the data for that particular cache key because if the old data remains in the cache then there will **[1:20:49]:** be data inconsistency because you will be serving the old data from the cache whereas we have updated the new data in the database right so what we can do is like after updating the database simply remove that data from the cache key. We saw about the delete method earlier in this video using which we can delete the data for specific key. Okay. So what will happen is once we have deleted that data from the cache whenever we get the read request we will check the cache and data will not be present in the cache because we have deleted that data. So it will be a case of cache miss and we will put that new data or the fresh data in the cache and we will return it to the end user. So the end user is getting the fresh data. Okay. And hence we have handled the inconsistency using the cache aside button. **[1:21:34]:** So the basic panda is whenever you are updating the database you will simply delete the data for that key from the cache. So here we are treating database as a source of truth and we are not trusting the cache. So we are removing the data from cache whenever we are performing the update operation. The second approach is TTL or time to live or you can say we set the expiry of the data. For example, as we discussed earlier, there is a set method in radius in which you put the key, you also put the value and then you can also set the TTL which is the time to live or the expiry of this data in the cache. Right? For example, here we have set the TTL to 10 minutes. So even if the data is inconsistent in the radius cache, how TTL is helping us is like it is limiting that period. **[1:22:25]:** So the old data can be served from the cache for maximum 10 minutes and after 10 minutes because this data will automatically removed from the cache. So whenever the read request comes it will act as a cache miss and we will fetch the data from the database put that new data in the radius cache and then return that data to the end user. So basically using the time to live we are restricting the data in which you will send the inconsistent or the old data to the end user. So you're serving the old data for limited time now due to TTL which is time to live or expiry of that data. The third approach is using the versioning. Here the problem is related to multi-threading. For example the thread A reaches the old database. Thread B updates the DB and deletes the cache. But thread A sets old value in the cache right because multiple threads are involved. **[1:23:18]:** So if they are performing the operations at the same time then it can cause such kind of problem. So the solution is you can store the versions or the time stamp and only update the cache if the version matches. So we have talked about multi-threading in this video and we have talked about versioning in this video. So refer to these videos and through this you'll get some more clarity. If you are new to multi-threading or these concepts like versioning, you can ignore this for now. It is not that important. Okay, I'll just tell you all of the approaches and after that I'll tell you which all are important and you should keep those in mind. Okay. Now the fourth approach is also related to the multi-threaded case and the name is double delete strategy. So this approach is related to approach one but we are talking about multi-threaded case. **[1:24:08]:** So when we update the DB we will delete the cache right we talked about this in approach one as well but additionally what we are also doing is like after 500 millisecond which is after half a second we will delete the cache again why because we are talking about the multi-threaded case and we have to avoid the race condition window again we are not going to talk about multi-threading in this video you can refer to these videos and through this you'll get some clarity about multi-threading but Again, this is not that important. If you're new to multi-threading, you can ignore this case. By the way, the fun fact is like you might have seen this kind of approach when we talk about the singleton pattern. For example, in Java, when we talk about how to implement the singleton pattern, first we talk about the basic approach, then we talk about the double strategy in which we take care of the multi-threaded scenario, right? **[1:25:01]:** It is quite similar to that. Okay? But it was just a fun fact. If you're not worked on singleton patterns or you don't know about that, just ignore it. It's not that important for now. All right. The fifth strategy is read your right consistency. Which means after user has performed the update operation for a few seconds, force the read from database or the master database instead of the cache. So because we know the inconsistency will happen when you update the data in the database, right? because the cache might contain the old data and database will have the new data. So if there is something which is critical. So instead of reading that from the cache what you can apply in the logic is that for some time what you can do is like force the read from the master database. Okay. So that the user gets the latest data and during that time your cache can sync with the database. **[1:25:53]:** Right? So this will be used only where the reads are critical. You can say for example the password change right we have talked about master database replicas and reading from the database in this video. So if you want to know more about it refer to this video related to database and you will get the complete clarity. Okay but just to give the summary if the reads are critical if we have made the update operation what we can do is like apply the logic that we read from the master database instead of the cache for some time so that we get the fresh data which would be present in the database. Now comes the sixth strategy which is accept eventual consistency. Okay. And in a lot of cases in real life systems you have eventual consistency. So wherever it is acceptable you can take the case of eventual consistency. **[1:26:42]:** It means that for some duration which can be like a limited duration. During that time user can be displayed the old data. Okay. So you are serving the old data for a small window. So there is inconsistency for a small period of window and that is totally acceptable. Let me give you some examples and then you can relate more to it. For example, when we talk about YouTube videos, when we say the like count, it is generally eventual consistency. The number that you see as a like count on a video that is eventually consistent. Okay, say there were like 10,54 likes and you as a user saw like 10,000 likes. That is totally okay because as a user you actually don't know what is the actual like count and even if you get to the real number after few seconds or a minute also that is totally fine right because **[1:27:36]:** the end user doesn't know about the actual count it is just an estimation or you can say bit old data like that was a count actually before 10 seconds and it is visible to you now it is totally fine. So that is a case of eventual consistency that after a few seconds you will get the actual data right. So what I mean is like the actual count in DB might be 10,54 but currently you can see 10,000 but that is totally acceptable. Okay. And similar is the case for user feed as well. Say you are following a person and that person has posted a video. Okay. It is totally fine if that video appears on your user feed after few seconds. Okay. because most of the times you won't be even aware that other person has made a post. So eventual consistency is fine in this case. **[1:28:25]:** Similar is the case for analytics. Okay. So few seconds, minutes of stailness is totally okay. Now let's think of a scenario where eventual consistency is not acceptable. That is a case when we talk about message application like WhatsApp. Okay. Say user A has sent a message to user B. So eventual consistency is not okay in that case because chat applications are treated as like real life communication. Okay. If you see delay in that it won't be any fun. So there the eventual consistency is not okay. Similarly when we talk about the video calls right. So eventual consistency will break the experience. So even when we talk about the chat applications there are some features where eventual consistency is not okay. Like we talk about like the video calls, voice calls, the messages there you have to be strictly consistent. But when we talk about features like last scene of the user, whether user is currently online or not, in those cases, eventual consistency is okay because those are not the most important features of the chat application. **[1:29:33]:** Okay. So depending on your feature sometimes eventual consistency is totally okay and in that case we are okay with serving the inconsistent data for few seconds or minutes but there can be some scenarios like I talked about the video call the chats the banking application in those cases eventual consistency is not okay again you can refer to this video of database you'll get more clarity about all these concepts okay all right so these were the six ways using which we can handle the inconsistency of data in the cache versus the database. **[1:30:08]:** But as I said, you don't need to remember all of these. What you can keep in mind is this first the cache side pattern. We also discussed about it earlier, but this is an important pattern. If you remember this, I mean most of the times we are handling the data inconsistency using this. And then TTL is also an important feature related to the distributed cache. So yeah, you can remember about TTL as a concept and through this you will remember you can handle the inconsistency through TTL. Okay, you can ignore this versioning and double delete strategy for now if it is bit complicated to absorb for now. Okay, these are not that important. And when we talk about these, these are more about like concepts. Okay, this is basically you are saying eventual consistency is okay for your system, right? So you are not doing anything. **[1:30:56]:** It is just kind of like you are telling that your system is okay with some of the inconsistency. And when we talk about this one as well, you can refer to this video of database. You get more clarity. But if it is looking bit complex to you, you can ignore this as well. So most important this one and TTL this is a concept of radius. So using these two you will handle the inconsistency in a good manner. Now one question may come into your mind like we have a lot of servers right? So the cache is sitting between the servers and the DB but we have multiple servers. So how do we get the consistent data? Actually that case is already handled because we are using radius as the cache here and radius is a distributed cache. So it will already handle this scenario. All of the servers will refer to the radius which is logically acting as a single cache but physically it has a lot of machines. **[1:31:53]:** So it is a distributed system. Okay. So it avoids single point of failure and it can also take a lot of load. Okay. So that's why we use generally radius as the cache between the server and the database. Now next let's discuss about some of the problems that we face in the cache or some of the very important interview questions related to the cash. The first one is what is cash stampede or the thundering herd problem in cash. So let's try to understand this in a simple way. Okay. So say this was your server. Okay. And this is your database. We said between the server and database we have one more component which is the cache. So as we know in the cache we have like data in key value format right. So say there was a key which was quite popular or famous which means it gets a lot of hits right and because we have this data in the cache so the server will hit the cache and we will **[1:32:57]:** always serve this data from the cache itself we don't even need to go to the database right but this key is quite like famous okay a lot of hits comes for this key and say this have like TTL of say 1 minute right And what happens is like after 1 minute the data related to this key will be removed from the cache right because TTL is the expiry of the data. Now say you get like thousands of request per second for this specific key. Now what cache stamped means is like once this key expires. Okay say it was 1 minute and at that time this the data related to this key was deleted from cache. So next second from the server you will get thousands of requests specific to this key but now data will not be present in the cache. So what will happen now is like for all of these same request we will hit the database. **[1:33:55]:** So you can imagine we are making thousands of requests on the database on that second for the same data and that is what we called as cache stamped. So the name clearly depicts what does it mean? Okay. And why do we also say it like thundering herd problem? So imagine a case like you have one door here and you have many buffaloos. So when the door opens, all the buffaloos say 10,000 buffaloos run at the same time towards that open door. Right? In similar to this example, we have 10,000 request say coming to the cache and that data is not present in the cache at that second. So we make all those 10,000 requests to the database. Now it is similar to 10,000 buffaloos running towards the door which was opened. Correct? And that's why this cash stampede is also known as thundering herd problem in the cache. **[1:34:52]:** So although I explained it here but let's also try to understand this problem more with an technical example. So suppose in the cache we are keeping data related to the user feed so that we can show the data on the user feed quickly to the end user. So the key is homepage colon feed. Okay. And say that TTL was 5 minutes. So after 5 minutes the data for this key from the cache will be removed. And let's imagine like 1 million users open the application every minute. So our application is quite famous. Okay. Now let's imagine at this time our cache expires because the TTL has reached right. So for all the users which are hitting the cache there will be cash miss right and all the request will be sent to the database. So say there were like 100,000 user request for all of those request there will be a cache miss and the request will go to the database for the same request at the same time. **[1:35:52]:** Right? This is same as that we discussed here. A lot of requests come but that data was removed from the cache. So we will have cache miss for all of these because these are happening at the same time and for the same request we are hitting the database which is not a good thing because these are the same request like these contain data for the same request and we are hitting the database for all of these request at the same time. So what happens is like database is flooded with a lot of user request and the bad part is all of these are same like you are getting the same data but you are hitting the database. So what can happen is like your database can become slow or it can even crash and what can also happen is like your whole system can go down because you are putting too much load on the database. **[1:36:41]:** Now when does cash temp happen? It can happen like when TTL expires like the case that we saw here. Apart from that the cache is flushed, the radius restarts or the hotkey gets deleted. Here hotkey means the key for which you get a lot of request. Okay. Like for example, we said we have one key here for which we get like 10,000 requests per second which means like you get a lot of request for that particular key. So that is said as hotkey. We'll talk about hotkey after this concept. But yeah, all of these scenarios can be the reason for cash stamp. Now we have understood what is cash stampede. The question comes, how do we solve cash stamp? So we have multiple approaches to solve this problem. Let's see that one by one. So the first solution is using the mutx or the log. **[1:37:33]:** The main panda is like as we have like 10,000 requests but all of these requests are same, right? So what we will do here is like only one request will go to the database. Instead of all 10,000 requests going to database only one request will go to database because rest of the request are same right. So what we will do is like only one request should go to the database. How do we do that? Let's see that. So what will happen is say like you got like 10,000 requests. So when one of the request comes it will try to get the log. If it gets a lock, it will fetch data from the database and once it gets the data from database, it will set that data in the cache and after that it will release the log. Okay. So one of the request will get the log and once it gets a log, it will fetch the data from DB then set the data in cache and then release the lock. **[1:38:25]:** But for other request which came at the same time. Okay. All of those request will not get lock because only one request can get the lock at a time. Okay. So for all of the other request they will wait and after that they will retry from the cache and once the first request has performed all of these things the data will be present in the cache and then data can be served directly from the cache. So basically using this only one of the request is going to the database to fetch the data and all of the remaining request will simply wait. Now apart from logs the second solution is logical expiry. So what we are doing here is we will refresh the data in the background. So instead of deleting the cache at the TTL we won't delete it. What we will do is like we will mark it as expired. **[1:39:15]:** So from which we'll get to know this is old data but still we can serve the old data right when you delete the data you cannot even serve the old data but when the data is present at least you can serve the old data so you have just marked it as expired which shows like data is old but still you can use that data and what we will also do is like one of the background thread will refresh the data from the database and put that in the cache. So basically we are not directly hitting the cache. Earlier we talked about the case we will simply delete the cache at TTL and then of course it will be a cache miss and you will have to go to the database to fetch the new data. Right? So now this approach is basically instead of deleting the data you are simply marking it as expired so that you know that it is old data but you are not going to the database. **[1:40:06]:** What's happening in the background is one of the thread will fetch the data from DB and put that in the cache. So till that process happens, you will serve the old data. When you get the new data, you will start serving the new data. Again, because we are discussing about multiple approaches here. I'm just giving you all the options that are present to handle this kind of issue. You can keep an idea of all of these. But if some of these are like becoming difficult to understand, you can ignore some of these. I'll tell you which one is important and which one you can ignore. Okay. After this, the third approach is using the random TTL or you use the jitter to add the randomness. So earlier, let me try to explain you with simple example. Earlier we had like TTL like 10 minutes. So all of the keys would expire at some specific time like after 10 minutes, right? **[1:40:59]:** So instead of doing that we will add some random factor in the TTL. So some will expire at like 958 some expire at 101 some at 1005 because we have added some randomness in the TTL. Now instead of giving it as a fixed TTL we do TTL plus minus sum jitter here is you can simply say some random number you are adding or deleting so that there is some randomness instead of all of the request hitting at the same time you get different different timings so that you don't get all the load at same time right so jitter is a very fundamental concept which is used in a lot of places whenever you want to distribute some of the load so that the complete load doesn't come at the same time. You get like randomness here. So after this the solution four is pre-warm cache. **[1:41:51]:** So what it means is like you already know when the data related to cash is getting expired right. So what you can do is like before the expiry you can refresh the cache proactively. So for example if this was a key present in your cache it will expire in like 10 minutes right? So what you can do is like you can periodically like after every 9 minutes you can simply fetch the data from DB and set that in the cache and again you can start the timer of 10 minutes. Okay. So in this way you are never basically deleting the data from the cache because you're never reaching this threshold. Before this time only you will refresh the data in the cache. Okay? You'll fetch it from the database and put in the cache. So say like it was 10 minutes. So every 9 minutes you will fetch the data from DB and put the fresh or the latest data in the cache. **[1:42:43]:** So the data will never be deleted from the cache and you will always keep on getting the latest data after every few minutes like 9 minutes we have set here. Right? So basic point here is that you do not allow here to delete the data. So before that happen you put the new data in the cache and the timer begins again. It is something similar to when we talk about like paging in any of the application. Say for example when you see the user feed on Instagram you have infinite scroll there right. So what happens is like you have some data like 50 records you already have. So you show that because at the time on screen you can see only like 10 15 results right? So user scrolls but now when you see you are reaching the threshold like you have already seen like 35 records or 40 records and you know like after couple of swipes you will you don't have any data anymore. **[1:43:41]:** So what client does it like it sends a new request to the server to get the next 50 records. Right? So that user does not need to wait. Correct? If you reach to the end and then you fetch the data from server, it will be a bad user experience because user has to wait because once you have reached to the end then you are making a request. So basically now when the response will come it will take some time and during that time the user will have a bad experience because it will simply show loading or buffering right you don't want that kind of experience for the end user. What you want is like when you already know that user is reaching to the end of the page and you don't have data anymore, you simply send a request to the server before reaching to the end. **[1:44:25]:** So you have set some threshold like user has seen like 3/4 of the results. You send a request to the server and get more data. Okay? So that when user reaches the 50th record, you can show the next records because you already have that data. It's kind of like you are fetching data proactively. Similarly here you already know after 10 minutes the data will be deleted from cache. So why do you even want to have stampede kind of problem. You can set a threshold like 9 minutes or 8 minutes or whatever time zone. You can simply fetch the data from the DB and put it in the cache. So that you have the new data now. You have not allowed to delete the data because the timer has restarted now and you are refreshing the cache proactively. All right. So using these four solutions you can solve the case of cash stamp paid. **[1:45:12]:** Although I have explained all of these approaches but just in case if it is feeling like overwhelming for you difficult to understand what I can say is like you can keep this in mind. Basically it simply says only one of the request will go to DB and rest will wait. It is a good solution. If this solution is becoming bit difficult to understand because here we are not deleting the data. Earlier we talked about deleting the data. Here we are marking. So you can ignore it for now. I just told you so that you get okay we can do all such kind of things. But if it's like becoming difficult to remember you can forget this. Don't worry. Okay. The third one random TTL or jitter. Jitter is a very kind of fundamental concept which you might be using in multiple places. But again if it is the first time you are watching this or you have not worked on it or you find it difficult to understand again you can ignore this. **[1:46:07]:** And the fourth one this is a good solution I would say like it's kind of like you are proactively refreshing your cash. So the solution one and solution four these are the good solutions. You can keep those in mind and just in case you the interviewer asks you in any of the scenario and if you are able to tell these kind of solutions it really shows that you have great understanding of the deep concepts in system design like if you know all these things in related to cache it already means that you have good understanding of system design. Okay. All right. So above we mentioned something about the hotkey and I said like we will discuss about it later in this video. So now let's discuss about the hot keys problem. So the question comes into mind what is a hotkey problem. So a hotkey is a cache key that gets huge amount of traffic compared to others. **[1:46:58]:** So as we discussed earlier in radius we will keep data in key value pair right corresponding to key you will store the value and we also saw that you can read the value of some key using the get method where you pass the key in it. Right? So basically you get the request based on the key correct. So a hot key would be a key which gets huge amount of traffic. So imagine something like Messi or Ronaldo created a post right and we have kept that in cash. So that key will get huge amount of traffic and that key will be called as a hotkey. All right with this we understand like if a key gets huge amount of traffic that will be considered as a hotkey. But what's the problem with hotkey? So here are the problems which are caused by the hotkey. So as we discussed earlier radius is distributed cache and it will have multiple nodes and shards. **[1:47:47]:** So say a hotkey is present in one particular shard then that particular reddish shard will become overloaded because like you had multiple shards and here you had this hotkey. So you are getting a lot of traffic in this particular shard. Correct? So this particular shard is getting overloaded. Then other problems related to hot keys would be high latency, CPU spike, network bottleneck and that radius node will become the hotspot. So there is high probability this shard may go down. Now you may ask like how come all the requests are going to particular shard. Basically radius shards by the key because everything is related to the key, right? So it shards based on the key. So all the requests which are coming for the same key will be redirected to the particular shard. So it will always go to the same node. Now that we have understood what is a hotkey and what are the problems associated with hotkey. **[1:48:40]:** Now the question comes how do we solve the hotkey problem. So there are multiple techniques through which we can solve this hotkey problem. Let's check them out one by one in detail. So the first solution is using the local cache. As we discussed earlier there are multiple kinds of cache apart from the cache that we set between the server and the DB which we are talking about the radius cache. Right? Apart from this also we have multiple other cache. One of those was server level cache. Right? So using the server level cache we are solving the hotkey problem. So if the data is present locally in the server level cache then you don't need to fetch it from the radius cache. Right. So what we can do is like if we identify something as a hot key for that we can keep the data in the local cache which means at the server level. **[1:49:28]:** So a post related to Ronaldo, Messi or any Hollywood star you can keep it at the server level cache as well. In case it's not present in the server level then you can fetch it from the radius cache and even in case it's not present in the radius cache you can pick it from the database. So the flow is server keeps hotkey in its own memory. So this cache and because now we have this data in the cache itself most of the request will not even go to radius. So we will solve the hotkey problem using the local cache. The second solution is by replicating the hot key which is also known as key sharding or key splitting. So earlier we mentioned like as we have multiple shards in the radius say the hotkey was present in this shard all of the requests related to the hotkey will come to this shard only right. **[1:50:15]:** So instead of doing this what we can do is like we can replicate this hotkey into multiple shards. Why? Because we know that this key gets a lot of traffic right. So we are replicating or duplicating this key in multiple shards. So here we are keeping data for the homepage related to India. Now instead of keeping it in a particular shard what we can do is like we can replicate this key. We are creating the replica of the original key. Right? So we will have multiple keys. Now we have appended some ID at the end of the key to make it unique. Now if we talk about the read case earlier we had only one key and it used to go to the particular shard. But now what's happening is like because we have multiple keys we can pick any key randomly from these. So because now we have multiple keys here as we have replicated the key. **[1:51:04]:** This means that we have also distributed the traffic among the shards. So instead of all the requests going to one particular shard now the traffic will move to different different shards because the keys are different. So this was a read case. when we talk about the right case then of course we need to update all of the keys otherwise it will cause data inconsistency so we will update all the copies okay so this was the second approach because all of the traffic was going to one particular shard what we did was we replicated that key so after this the third approach is using the CDN or the edge cache we discussed earlier about the different kinds of cache this was your server this was your database Right. So we mentioned there can be this cache like I can say this as the radius cache because we will generally use radius here which is a distributed cache. **[1:51:56]:** So here we had the problem of hotkey. So one solution that we talked about is using the server level cache. Now apart from the server level cache we can also use CDN. Right? So if you want to know more about CDN you can refer to this video where I have talked about CDN in detail. And by watching this video you'll get complete clarity about the CDN. But just to give you the summary, CDN is used to cache the static assets. Mainly it is used for images and videos. And you know most of the applications or products that we use like YouTube, WhatsApp, Netflix, Amazon all of these deal with images and videos. Even in Instagram when you open you see the videos and images on your feed right. So in that CDN can help because CDN can cache those static assets and the client can directly fetch the data from CDN because CDN represents distributed set of servers which are present across the globe and which can cache **[1:52:56]:** the static assets. So if it's a case of cache hit which means client gets the data directly from the CDN then you are not even entering the backend layer. So radius was part of the backend layer correct. So you are not even entering this side not even server not even radius or the database correct client is getting the data directly from the CDN. Okay, but the main point to remember CDN is used to cache the static assets. Whereas when we talk about radius, we could also keep some dynamic data in it. Correct? But CDN is specific to the static assets like images, videos. Okay? Again, you can refer to this video, you'll get the complete clarity about the CDN. So the main point is if you get the data from CDN, you are not at all hitting the backend infra. You are not reaching radius at all. **[1:53:43]:** Right? So, so hotkey problem is solved by this. If you're talking about the static assets as those can be served directly by the CDN cache. All right. After this the next approach is using the logical expiry and background refresh. We talked about this earlier as well. Right. Instead of deleting the data we will just mark it as expired but we will keep on serving the old data and one background thread will refresh it. So because now data is never deleted due to TTL. We will always have that data. It might be old but we will still have that data but we will never face cash template problem which can increase huge amount of latency and cause a huge traffic on database right. So using this approach which we discussed earlier for cache template we can reduce the overall latency on the hotkey. Of course it is not solving the hotkey problem overall because you will keep on getting the complete traffic for that hotkey but at least you don't need to fetch it from database. **[1:54:38]:** So it is avoiding that cache template problem. So it's kind of a prevention because in case of hotkey you'll get a huge amount of traffic and if you don't do things like this then you will face the cash temp problem right the next approach is also related to cash stamped problem that we discussed earlier like you do not allow that data to be removed due to TTL before that threshold is reached you basically fetch the data again from the DB and refresh the cache with the new data. So before the TTL time is reached you proactively fetch the data from DB and refresh it in cache. This is the same thing that we discussed as our solution for cache stamped. So both of these like fourth and fifth these are actually not removing the hotkey problem but these are kind of helping preventing the cash stamp because you get a lot of traffic to a particular key and in case **[1:55:32]:** you don't follow these strategies it will create the situation of cash stamp. Right? So these two things we already discussed earlier but these three ones are the actual solutions for the hotkey problem. Okay, you can use the local cache, you can replicate the hotkey and you can use the CDN wherever possible for the static assets. **[1:55:51]:** Now because we have talked about multiple cache here I'll try to explain it to you with simple flowchart here. Okay. So let me draw first of all you have the client here, you have the server here, right? You have the DB here and you have this distributed cache which I can say as the radius cache. So inside this we had the problem of hotkey right. So talking about the different different cache you can have the server level cache and we can also have that CDN cache. Right? So this is the flow. First of all you have the client here. Right? So if we are talking about the static assets like images, videos those can be cached in the CDN cache and client can directly fetch those resources from the CDN. So if we are doing that we are not hitting the backend infra and hence we are solving the problem of the hotkey. **[1:56:45]:** So this is our first level of cache when we come from client side and then next when we come to the server in order to remove that hotkey problem we can keep server local cache. So if you have the data present in the server itself you don't need to fetch it from radius you can directly serve it from the server right. So this is the second level of cache that can appear and then of course we have this radius and then database. Okay. So if you are not hitting the radius of course you are already solving the hotkey problem. So using the CDN and server local cache you can solve this problem. Third way of course as we said we can replicate the hot keys so that we have data in multiple shards and radius nodes and the traffic can be balanced among those shards using the replicated keys. **[1:57:34]:** All right. So we have discussed a lot about cash now but there is one important question that we need to discuss and that is when to bring the cache component in the interviews. In interviews you would be asked the highle design question. So the important question is like when to bring this cache component in your highle design. So basically we can bring the cache component if our system is coming into any of these cases. The first one is our system is read heavy. Okay. Now the use case of cache was like there is some data in the DB which is getting called multiple times. Okay, there is some basically data which is getting a huge amount of traffic. people wants to know about that data, wants to fetch that data and because of that we keep that data in the cache and now cache is like much faster as compared to DB because cache is like your RAM **[1:58:24]:** storage whereas your DB is like the disk storage like the HDD or the SSD right and RAM is much much faster as compared to the disk storage right so that's why we want to keep that data in cache which is read very frequently so if your system is read heavy then of course it makes sense to bring cache component in the picture right so first case is like if your system is read heavy bring the cache component now similar to what we discussed because we said like we might be putting a lot of pressure on DB if you get a lot of request if your system is read heavy right in order to reduce the DB load we can use the cache component right because if the data is like read heavy or if you get a lot of read request right then it's a problem for DB DB may slow down or even die **[1:59:14]:** right so in order to reduce the load on the database we can introduce the cache component so this one is bit related to the first one that we discussed of course if your system is read heavy you need to reduce the load on DB and that's why you are using the cache bit interrelated you can say here now one more use case would be to reduce the latency as I said earlier cache is basically present in the RAM storage whereas database is in the disk storage like HDD or SSD. So SSD is much faster than HDD but RAM is even much much faster as compared to the SSD. So if you are getting data from cache it will be much faster as compared to getting it from the database. So if you're serving data from the cache you are reducing the latency and the end user will be able to see data much more quickly. **[2:00:05]:** Right? It's same as like the example we discussed like if you are the chef right and you have to make dishes right and if the salt and pepper are the things which you use in all of the dish preparation even if you're making pizza you add salt to it if you're making pasta noodles or anything you add salt to it. So instead of every time going back to the storage which is your kitchen main kitchen going and picking the salt and pepper from there what you can do is like then keep it in front of you near to you right whenever you are making something you can directly pick it because it's present in front of you you just use it and keep it again in front of you right so in this way you are reducing the latency so in similar way when you are serving the data from cache you are reducing the latency of the overall query you can say so sim Similar to this, one more could be cache can be used for the **[2:00:57]:** expensive queries and you can basically do the premputee for that. One example for this can be when we design the Instagram user feed or Twitter's news feed then basically we need to show that user feed or news feed to the user very quickly. You cannot take much time there because when user opens the application if it is loading then it's a really bad user experience. User won't use your application or product then. So what you need to do is like you need to premp compute the data so that when user opens that you already have that data you can directly show it to the end user. So you are doing like premputee on that. So if you want to know more about like Instagram design you can refer to this video because I have discussed about like highlevel design of Instagram in this video. You'll get more clarity about what I'm talking here. **[2:01:44]:** But yeah if you need to basically do the premputee for the expensive queries you can use the cache. So mainly you can see if your system is read heavy then in order to reduce the DB load and in order to reduce the latency and in order to premp compute the expensive queries all come in sense mainly to the read heavy system. So if your system is read heavy you can bring the cache component there. Now after this let me tell you one of the basic mistake that people make related to cache in the interviews. So as I said like when our system is read heavy this is say your server this is your database and now say your system is read heavy right so you will introduce the radius cache here correct to reduce the load on the database to reduce the latency of the query and much more right so you will say that that's totally fine but one mistake here is that people just say like we will use the cache here **[2:02:40]:** but they are not able to identify or tell like what to cache here like what are the caching here to basically reduce the load on DB or to reduce the latency what are you caching here like how is it helping in your system right if we just say that we will use the cache here it's a very generic kind of component in system design but you also need to tell how this component is helping in your design say for example you are asked a question related to design WhatsApp or design YouTube or design tiny URL anything like that right if you simply say I'll use a cache then it's a very generic thing to say but because you are designing a particular system you are designing a particular feature you also need to have in mind that what are you caching what data are you caching due to which you are basically reducing the load **[2:03:31]:** on DB or you are reducing the latency right so you need to identify that as well saying just that we will use a cache in between is not sufficient you also need to identify what Will you cache? And that will be specific to the feature that you are designing. Right? You already know that it is in form of key value pair. But what is it that you are caching? It's also important that you identify that, right? **[2:03:55]:** All right. So, we have discussed a lot of things about cache in this video. If you found this video helpful, like the video, subscribe to the channel and share it with your friends and family so that all of us can learn and grow together. If you have any doubts, put that in the comment section. If you want me to create video on some specific system design topic, put that in the comment section. I'll create a separate video for that. All right, then. This is Mohit Shabda signing off. Subscribe to the channel and thanks for watching.