Kafka, OSI Model & API's

Kafka, OSI Model & API's

·

2 min read

Kafka was originally built for massive log processing. It retains messages until expiration and lets consumers pull messages at their own pace

Let's review the populat Kafka use cases.

  • Log processing and analysis

  • Data streaming in recommendations

  • System monitoring and alerting

  • CDC(Change data capture)

  • System migration

What is OSI Model ?

How is data sent over the internet ? What does that have to do with the OSI model ? How does TCP/IP fit into this

7 Layers the OSI model are:

  • Physical Layer

  • Data Link Layer

  • Network Layer

  • Transport Layer

  • Session Layer

  • Presentation Layer

  • Application Layer

How to Improve API Performance ?

Result Pagination:

This method is used to optimize larges results sets by streaming them back to the client , enhancing service responsiveness and user experience

Asynchronous Logging:

This approach involves sending logs to a lock-free buffer and returning immediately, dealing with the disk on every call. Logs are periodically flushed to the disk, significantly reducing the I/O overhead

Data Caching

Frequently accessed data can be stored in a cache to spped up retriecal. Clients check the cache before querying the database, with data storage solutions like Redis offering faster access due to in-memory storage

Payload Compression:

To reduce data transmission time, requests and responses can be compressed (eg:- making the upload and download process quicker)

Connection Pooling:

This technique invloves using a pool of open connections to manage database interaction which reduces the overhead associated with opening and closing connections each time needs to be loaded. The pool manages the lifecycle of connectins for efficient resources

Did you find this article valuable?

Support Thirumalai by becoming a sponsor. Any amount is appreciated!