Cloud Zone is brought to you in partnership with:

Software developer and frequent open-source contributor. Writing mostly for .NET, but also Java and C/C++. Really likes fiddling with data, texts especially, so he frequently finds himself working on databases or search engines, usually combining both. Itamar is a DZone MVB and is not an employee of DZone and has posted 31 posts at DZone. You can read more from them at their website. View Full User Profile

The Story of a Massive System Refactoring: Prologue

03.18.2013
| 1606 views |
  • submit to reddit

A couple of months ago I started working at Buzzilla, a company developing "cutting edge technologies and revolutionary analysis and research methodologies that combine to create advanced solutions aimed at harnessing the vast opportunities presented by online conversation". Or in short, full-text search and analytics on BigData.

When I started there the existing system was based on a home-brewed solution for distributing a Lucene index across dozens of nodes. Since we are looking to expand way beyond the number of a couple of dozen servers, we really needed to recreate the system using tools better suited for the job, which we could also take to the cloud. And it is much more than just about the search engine.

Refactoring an operational distributed system is really a great challenge. While keeping it operational, you need to replace parts one by one, but you also get the opportunity to experiment with new tools and bleeding edge technologies. It is also a great subject for a series of blog posts, this one being the first of many to come. Among the items we had to tackle on which I'll blog are:

  • Building a distributed and highly available search engine
  • Various topics on full-text search: relevance, scoring, multi-lingual search, best practices for analysis and more
  • Choosing the right web framework for a web UI
  • Migrating from MySQL to NoSQL - and selecting the right NoSQL for the job
  • Caching
  • Keeping tabs of logs in a distributed environment
  • Planning for and recovering from failures and crashes
  • CI, deployment, versioning and backups
  • Identifying and fixing performance bottlenecks
  • Generating and displaying system stats and performance metrics

Such a refactoring process could easily turn out to be a disaster. This is the story of a careful planning and a great team working together, which made this a success. We are now in the final stages of migrating to the new system, and we have many more challenges pending after we have gone completely live. This series is going to span months and hopefully have some great content which will spark great discussions.

 

Published at DZone with permission of Itamar Syn-hershko, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)