Kafka was originally built for massive log processing. It retains messages until expiration and lets consumers pull messages at their own pace
Let's review the populat Kafka use cases.
Log processing and analysis
Data streaming in recommendations
System monitoring and alerting
CDC(Change data capture)
System migration
What is OSI Model ?
How is data sent over the internet ? What does that have to do with the OSI model ? How does TCP/IP fit into this
7 Layers the OSI model are:
Physical Layer
Data Link Layer
Network Layer
Transport Layer
Session Layer
Presentation Layer
Application Layer
How to Improve API Performance ?
Result Pagination:
This method is used to optimize larges results sets by streaming them back to the client , enhancing service responsiveness and user experience
Asynchronous Logging:
This approach involves sending logs to a lock-free buffer and returning immediately, dealing with the disk on every call. Logs are periodically flushed to the disk, significantly reducing the I/O overhead
Data Caching
Frequently accessed data can be stored in a cache to spped up retriecal. Clients check the cache before querying the database, with data storage solutions like Redis offering faster access due to in-memory storage
Payload Compression:
To reduce data transmission time, requests and responses can be compressed (eg:- making the upload and download process quicker)
Connection Pooling:
This technique invloves using a pool of open connections to manage database interaction which reduces the overhead associated with opening and closing connections each time needs to be loaded. The pool manages the lifecycle of connectins for efficient resources