Killing the HBase zombie table

Posted on January 15th, by Arnon Rotem-Gal-Oz in Big Data, Blog. 3 comments

One of our team leaders approached me in the hall today and asked if I could land a hand in troubleshooting something. He and our QA lead were configuring one of our test Hadoop clusters after an upgrade and they had a problem with one table they were trying to set up:

When they tried to create the table in HBase shell they got an error that the table exists
When they tried to delete the table they got an error that the table does not exist
HBase ships with a health-check and fix util called hbck (use: hbase hbck to run. see here for details) – they’ve run hbase reports everything is fine and dandy

Hmm, The first thing I tied to do is to look at the .META. table. This is where HBase keeps the tables and the regions they use. I … Read More »

Introducing H-Rider

Posted on December 10th, by Arnon Rotem-Gal-Oz in Blog. 4 comments

In the last year and half or so (since I  joined Nice Systems ) we’ve been hard at work building our big data platform based on a  lot of open source technologies including Hadoop and HBase and quite a few others. Building on open source brings a lot of benefits and helps cut development time by building on the knowledge and effort of other.

I personally think that this has to be  two-way street and as a company benefits for open source it should also give something back. This is why I am very happy to introduce Nice’s first (hopefully first of many) contribution back to the open source community. A UI dev tool for working with HBase called h-rider. H-rider offers a convenient user interface to poke around data stored in HBase which our developers find very useful both for development and debugging  

h-rider … Read More »

Ignorance I tell you, it is all ignorance.

Posted on November 19th, by Arnon Rotem-Gal-Oz in Blog. 3 comments

I was poking around my old blog ( and I found this post from 2007 which I think is worth re-iterating:

In a post called “Ignorance vs. Negligence“, Ayende blows some steam off on some of the so called “professionals” that he met along the way. You know …those with a fancy title that don’t know jack and design some of the nightmares we see from time to time. I’ve seen this phenomena in a lot of projects I consulted/reviewed:

The senior security expert who recommended something which isn’t supported by the platform
The senior architect who throw the system down to hell by basing all the system on a clunky asynchronous solutions that should only be used by a tiny portion of the application.
The geniuses that built this wonderful code generator that generated code with so many dependencies and singletons that made … Read More »

The NoSQL landscape in diagrams

Posted on November 3rd, by Arnon Rotem-Gal-Oz in Big Data, Blog. 1 Comment

Here’s the NoSQL landscape in 3 slides (and hey, at least mine looks different :) )

451 research published their view of the NoSql/NewSql world in a unified diagram.

Infochimps published a similar diagram

And here’s mine from SOA Patterns chapter 10 (discussing “SOA & big data”)



SOA & Big Data

Posted on October 11th, by Arnon Rotem-Gal-Oz in Big Data, Blog, SOA Patterns. 2 comments

I gave a presentation of SOA and big data in IGTCloud forum

5 lessons big-data projects should learn from the iOS6 map debacle

Posted on September 21st, by Arnon Rotem-Gal-Oz in Big Data, Blog. No Comments

By now you’ve probably heard something about Apple’s new iOS6 maps app. In case you’ve been living under a rock, it turns out the new and shiny application that replaces Google maps in the new iOS release produces a lot of inaccuracies, mangled graphics, navigation errors and what not (just like the image you see on the left – for more examples you can see this site). Kidding (or gloating) aside, this debacle carries with it a few important lessons that anyone who is building a big data project should keep in mind.

Apple took data from various sources like Waze, Tomtom, yelp and others to build their database. thinking that it is all just geographical data using the same coordinate system so everything should be just fine. Well, it doesn’t work like that – out first and probably most important … Read More »

SOA patterns (finally) published

Posted on August 28th, by Arnon Rotem-Gal-Oz in Blog, Featured Posts, SOA Patterns. 8 comments

My book on Service Oriented Architecture patterns is finally published. You can get the ebook on manning’s site. The printed version will be available Sept 7 (can be preordered on Amazon) and the Kindle version/ePub versions will be available on Sept 20th.

I also prepared pages for all the pattern on my site (you can click on the pattern map) which holds a brief description of each pattern and is meant to use as a quick reference. There are a few thing I still need to do there like add the page number for each pattern, provide links to the first and last chapters (which are available for free on Manning’s site), add the anti-pattern etc. but most of the work is done.

What’s really amazing is that Gregor Hohpe agreed to write the forward to the book and what is even … Read More »

Distributed computing reading list

Posted on August 21st, by Arnon Rotem-Gal-Oz in Blog. No Comments

My twitter feed spewed a very good list of distributed computing related papers (compiled by Dan Creswell). There are links to a lot of papers there. Few of my favorites include The fallacies of distributed computing by Peter Deutsch – you may also want to check out the paper I wrote explaining them; Life beyond distributed transactions an apostate’s opinion by Pat Helland and also “The Byzantine generals problem” by Leslie Lamport, Robert Shostak and Marshall Pease, “A note of distributed computing” by Samuel C. Kendall, Jim Waldo, Ann Wollrath and Geoff Wyant and “Harvest, yield, and scalable tolerant systems” by Armando Fox, Eric A. Brewer which I mentioned before in “10 papers every architect should read”

There are a also a few additional papers that are not in that list and that I found illuminating:

“Architectural Styles and the Design … Read More »

Make technical debt explicit

Posted on August 7th, by Arnon Rotem-Gal-Oz in Blog. 6 comments

This is a rehash of a topic I posted about back in 2008 but it is just as relevant today.

When you work towards a release or some other looming milestone. There’s that  urge to leave stuff behind, cut some corners – you know, just a wee bit, no harm done – so we can ship the damn thing already. Mostly you’d fight that urge. But sometimes you’d want to make a conscious choice to make a shortcut because, well,  sometime delivery is more important

A little drop in quality in a single piece of code will not do much harm…for a while. There are, however, two problems with going this approach. One is that it is a slippery slope, i.e. it is probably not just one piece of code that gets this treatment in your code base. The second problem is … Read More »

Looking for an Architect to join our team

Posted on August 1st, by Arnon Rotem-Gal-Oz in Blog, Uncategorized. No Comments

As you may know last year I joined Nice systems where I lead the architecture team of our big-data multi-channel analytics platform. Well, we only have 4 architects in the team and we need at more. The current open position is for a senior architect for the application development group.  We’re looking for someone who is passionate about technology, a mentor and problem solver.  We need someone who is not afraid to write code ,make stuff work and pair with developers on one hand but can also  lead the system’s vision and design architectures on the other hand.

Please note that the position is in Raanana, Israel

Oh yes – here’s the official blurb:

Lead and design architecture for new enterprise applications and its infrastructure
Define, Lead POCs and choose technologies
Be a technical authority, design and document best practices, educate and support the development teams


Team player
Good English
At … Read More »