ZooKeeper for Search Applications
November 20, 2013
Looking for Google-style tech to speed up your search app? The AppScale Blog presents us with an affordable option in, “Emulating Google Megastore Using Open Source Technologies.” The article tells us why Apache’s ZooKeeper is even better than Google’s Bigtable (links in the quote are PDFs.):
“The BigTable model is not enough to fully emulate the Google App Engine Datastore API, as it is based on Megastore, which provides the added benefit of transactions on partitioned data. For this AppScale uses ZooKeeper, the open source implementation of Google’s Chubby. ZooKeeper provides a locking API using a variant of the Paxos algorithm.
“To emulate Megastore with open source software, AppScale automatically sets up a datastore for applications to use and provides the mappings from the Google App Engine Datastore API to the Cassandra and ZooKeeper APIs. With both ZooKeeper and Cassandra, whether its a one node, or an eight node deployment, AppScale will create the configuration files, and start the correct processes on each node. Optionally, the AppScalefile (the AppScale configuration file) can dictate the amount of replication the datastore does. This also makes AppScale a great tool to use to automatically set up a Cassandra or ZooKeeper cluster.”
The write-up goes on to address data layout in Cassandra, query types, and ZooKeeper locks. At the bottom are several helpful links for further investigation. Oh, and a brief, unexplained, lukewarm beer review that is apparently part 16 in a series. It is good to have diverse interests.
Cynthia Murrell, November 20, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Comments
One Response to “ZooKeeper for Search Applications”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
Zookeeper is also used in SolrCloud, the new(ish) distributed indexing and search features in Lucene/Solr 4.x.