Cloud Zone is brought to you in partnership with:

Treasure Data's Big Data as-a-Service cloud platform enables data-driven businesses to focus their precious development resources on their applications, not on mundane, time-consuming integration and operational tasks. Our pre-built, multi-tenancy cloud platform is already in use by over 50 customers worldwide and is managing more than 200 billion rows of data and processing 130,000 jobs per day. Discover how Treasure Data can help you focus on your core business and benefit from the fastest time-to-answer service available. Sadayuki is a DZone MVB and is not an employee of DZone and has posted 27 posts at DZone. You can read more from them at their website. View Full User Profile

Treasure Data Working as an Alternative to Hadoop

12.16.2012
| 3855 views |
  • submit to reddit

Over the past year, we’ve done a lot to be proud of. We built a highly performant, Hadoop-based data analytics platform from scratch. For next-generation ETL and real-time data collection, we authored td-agent and open-sourced it as Fluentd, which now has ~800 stars on GitHub.

But our biggest achievement? Our happy customers. Today, we want to introduce one of them, Viki.com.

Viki.com: A global video platform

Viki is a global online video service that lets millions of people watch their favorite TV shows, movies, sports games, and more. What really sets it apart is its international viewership: Viki’s content is available in over 150 languages, all translated by its user community.

No Stranger to Hadoop

Viki.com is no stranger to Hadoop. They have very talented engineers, and before introducing Treasure Data, they used to run their own Hadoop cluster to analyze millions of pageview and user event data.

But several months in, Viki had an epiphany: Hadoop’s biggest cost is not initial setup but ongoing management and tuning. They noticed that they were spending more and more time just to ensure their cluster held up in the face of surging traffic.

BEFORE TREASURE DATA

Treasure Data: Move Fast, Add Value

Thus, when Viki engineers stumbled on Treasure Data, they immediately saw its value proposition: taking operation cost out of Hadoop and Big Data analytics, freeing up engineers’ time, and accelerating the core product’s development.

This is great for business, too. Good engineers are hard to find and even harder to hire. If you run your own Hadoop cluster, you need to dedicate some of your best engineers to its maintenance. Sure, Hadoop itself is free, but when you consider associated engineering cost, it quickly becomes very expensive.

AFTER TREASURE DATA

Support, Support, Support!

The other day, Huy, Viki’s software engineer who contributed a “success story” write-up earlier, had questions about the data transport between Treasure Data and Viki’s PostgreSQL (see the above diagram). Our engineers dug into his questions immediately, made a couple of technical suggestions, and within one week, Huy sped up the data transport by several fold.

In the end, there is only one question that matters: are we adding value for our customers? We still have a long journey ahead of us, but the encouraging feedback from our customers leads us to think we are on the right track =)

Published at DZone with permission of Sadayuki Furuhashi, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)