Welcome


Is there a future for Map/Reduce?

Posted on June 3rd, by Arnon Rotem-Gal-Oz in Big Data, Blog, Featured Posts. 3 comments

Google’s Jeffrey Dean and Sanjay Ghemawat filed the patent request and published the map/reduce paper  10 year ago (2004). According to WikiPedia Doug Cutting and Mike Cafarella created Hadoop, with its own implementation of Map/Reduce,  one year later at Yahoo – both these implementations were done for the same purpose – batch indexing of the web.

Back than, the web began its “web 2.0″ transition, pages became more dynamic , people began to create more content – so an efficient way to reprocess and build the web index was needed and map/reduce was it. Web Indexing was a great fit for map/reduce since the initial processing of each source (web page) is completely independent from any other – i.e.  a very convenient map phase and you need  to combine the results to build the reverse index. That said, even the core google algorithm –  the famous pagerank is … Read More »

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Buffer this pageShare on RedditShare on StumbleUponEmail this to someone


Hadoop YARN overview

Posted on May 27th, by Arnon Rotem-Gal-Oz in Big Data, Blog, Featured Posts. No Comments

I did a short overview of Hadoop YARN to our big data development team. The presentation covers the motivation for YARN, how it works and its major weaknesses

You can watch/download on slideshare

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Buffer this pageShare on RedditShare on StumbleUponEmail this to someone

Looking for developers to join my group

Posted on April 29th, by Arnon Rotem-Gal-Oz in Blog, Featured Posts. No Comments

With Amdocs TeraScale, my previous project, move into production, I moved from to a new role within Amdocs and took over the Technology Research group, which is part of the big data and strategic initiatives business unit.

Now it is time to expand the group and I am looking for a developer-architect and/or senior developer to join my group.

 

If you are a technologist at heart and like learning new stuff every day
If you can pick up a new a technology and be up and running with it in a day or two
If you want to tinker with the latest and greatest technologies (big data, in memory grids, cloud management systems, NFV, columnar database etc.)
If you want to help shape the technology roadmap of a large corporation

I am looking for you

The positions are located in Raanaa Israel. If you’re interested you can contact me … Read More »

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Buffer this pageShare on RedditShare on StumbleUponEmail this to someone