Skip to main content

Apache Zookeeper On Java On Ease

 Its an open source coordination service for distributed application.

It exposes simple set of primitive that a distributed application can built up on to implement higher level of services and synchronisation and also the configuration management.

Yahoo, twitter, Netflix, facebook

problem/solution :

Coordination services : The process of integrating or communicating between services in distributed env. They are prone to, Race condition and dead-lock.

Race condition : Two or more system trying to perform the same task.

Dead-lock : Two or more system waiting on each other.

Relieve distributed application from the responsibility of implementing the coordination service from scratch. 

Coordination challenges : 

Zookeeper is centralise coordination service is distribute and highly reliable running on cluster of service called zookeeper ensemble. 




2 types of Nodes

  1. Leader Node : 1 Node processing for write request. Elected internally. More votes becomes leader. Recommended to have odd number of nodes in cluster
  2. Follower Node : simply write calls delegates to leaders. Servicing read request. Plays role to elect leader.

Request processor : Only active in leader Node, responsible for processing write request originated from client or follower nodes. Once write request processed then it broadcast the changes to follower to update their state accordingly.

Atomic broadcast : Present in Both leader and follower Node. Responsible to broadcast changes in leader node and follower Node.

In Memo database/ replicated database : For Storing zookeeper data. Every Node contains it own DB that enables to serve the read request. Data is also written to file system to providing recoverability in case of any problems with cluster.


Client : Send message to the server to let the server know that the client is alive. Server sends the acknowledgement when a client connects, when no response from server then the client connects to another node.

Server : provide service to client

Ensemble : group of zookeeper servers, minimum has to be 3

ZNode : every node in zookeeper node is ZNode. It maintains the stat structure including

  • Version : if the data changes the version changes
  • ACL : An authentication mechanism for accessing the ZNode for read and write permission on ZNode
  • TImestamp : Time elapse from ZNode creation and data modification (In Millisecond)
  • Data Length : Total amount of data store in a ZNode (Max of 1 MB)

ZNode can store up-to 1 MB of data. 

Type of ZNodes :

  1. Persistence Node : Alive until explicitly deleted
  2. Ephemeral Node : Active client connection is alive. Dynamic Nodes
  3. Sequential : Persistence or ephemeral. Zookeeper sets path. Play an important role in locking and synchronisation. 

Session and Watches

Session :

  • Request in a sessions are executed in FIFO
  • Once session established, then session id is assigned to the client.
  • Client sends heatbeats to keep session valid.
  • Session timeout is usually represented in milliseconds.

Watches :

  • Watches are mechanism for client ot get notification of whatever changes happened in zookeeper ensemble.
  • Client can set watches while reading the ZNode.
  • Sends the notification to registered client for any changes in ZNode, changes are modification of data associated with ZNode.
  • Watches are trigger only once, if the client wants notification than it must be done through another read operation
  • When a connection session is expired the client will be disconnected from the server and associate watches also removed. 

Expirations happens when the cluster does not hear from the client within the specified session timeout period

 (i.e. no heartbeat)

Distributed Lock using Zookeeper





Description

The Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It adds many features that build on ZooKeeper and handles the complexity of managing connections to the ZooKeeper cluster and retrying operations. Some of the features are:

  • Automatic connection management:
    • There are potential error cases that require ZooKeeper clients to recreate a connection and/or retry operations. Curator automatically and transparently (mostly) handles these cases.
    • Watches for NodeDataChanged events and calls updateServerList() as needed.
    • Watches are automatically removed by Curator recipes
  • Cleaner API:
    • simplifies the raw ZooKeeper methods, events, etc.
    • provides a modern, fluent interface
  • Recipe implementations:
    • Leader election
    • Shared lock
    • Path cache and watcher
    • Distributed Queue
    • Distributed Priority Queue

Labels:

Comments

Post a Comment

Popular posts from this blog

How to create Annotation in Spring boot

 To create Custom Annotation in JAVA, @interface keyword is used. The annotation contains :  1. Retention :  @Retention ( RetentionPolicy . RUNTIME ) It specifies that annotation should be available at runtime. 2. Target :  @Target ({ ElementType . METHOD }) It specifies that the annotation can only be applied to method. The target cane be modified to:   @Target ({ ElementType . TYPE }) for class level annotation @Target ({ ElementType . FIELD }) for field level annotation @Retention ( RetentionPolicy . RUNTIME ) @Target ({ ElementType . FIELD }) public @ interface CustomAnnotation { String value () default "default value" ; } value attribute is defined with @ CustomAnnotation annotation. If you want to use the attribute in annotation. A single attribute value. Example : public class Books {           @CustomAnnotation(value = "myBook")     public void updateBookDetail() {         ...
Post a Comment
Read more

Kafka And Zookeeper SetUp

 Kafka And Zookeeper SetUp zookeeper download Link : https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.8.3/apache-zookeeper-3.8.3-bin.tar.gz Configuration: zoo.conf # The number of milliseconds of each tick tickTime =2000 # The number of ticks that the initial # synchronization phase can take initLimit =10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit =5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir =/tmp/zookeeper # the port at which the clients will connect clientPort =2181 4 char whitelist in command arguments 4lw.commands.whitelist =* Start ZooKeeper Server $ bin/zkServer.sh start Check zookeeper status dheeraj.kumar@Dheeraj-Kumar bin % echo stat | nc localhost 2181 stat is 4 character whitelisted argument  Check Kafka running status : echo dump | nc localhost 2181 | grep broker Responsibility of Leader in Zookeeper: 1. Distrib...
Post a Comment
Read more

Cache Policy

Cache policies determine how data is stored and retrieved from a cache, which is a small and fast storage area that holds frequently accessed data to reduce the latency of accessing that data from a slower, larger, and more distant storage location, such as main memory or disk. Different cache policies are designed to optimize various aspects of cache performance, including hit rate, latency, and consistency. Here are some common types of cache policies: Least Recently Used (LRU): LRU is a commonly used cache replacement policy. It evicts the least recently accessed item when the cache is full. LRU keeps track of the order in which items were accessed and removes the item that has not been accessed for the longest time. First-In-First-Out (FIFO): FIFO is a simple cache replacement policy. It removes the oldest item from the cache when new data needs to be stored, regardless of how frequently the items have been accessed. Most Recently Used (MRU): MRU removes the most recently accessed ...
Post a Comment
Read more