By now you’ve probably heard something about Apple’s new iOS6 maps app. In case you’ve been living under a rock, it turns out the new and shiny application that replaces Google maps in the new iOS release produces a lot of inaccuracies, mangled graphics, navigation errors and what not (just like the image you see on the left – for more examples you can see this site). Kidding (or gloating) aside, this debacle carries with it a few important lessons that anyone who is building a big data project should keep in mind.

Apple took data from various sources like Waze, Tomtom, yelp and others to build their database. thinking that it is all just geographical data using the same coordinate system so everything should be just fine. Well, it doesn’t work like that – out first and probably most important … Read More »

My book on Service Oriented Architecture patterns is finally published. You can get the ebook on manning’s site. The printed version will be available Sept 7 (can be preordered on Amazon) and the Kindle version/ePub versions will be available on Sept 20th.

I also prepared pages for all the pattern on my site (you can click on the pattern map) which holds a brief description of each pattern and is meant to use as a quick reference. There are a few thing I still need to do there like add the page number for each pattern, provide links to the first and last chapters (which are available for free on Manning’s site), add the anti-pattern etc. but most of the work is done.

What’s really amazing is that Gregor Hohpe agreed to write the forward to the book and what is even … Read More »

My twitter feed spewed a very good list of distributed computing related papers (compiled by Dan Creswell). There are links to a lot of papers there. Few of my favorites include The fallacies of distributed computing by Peter Deutsch – you may also want to check out the paper I wrote explaining them; Life beyond distributed transactions an apostate’s opinion by Pat Helland and also “The Byzantine generals problem” by Leslie Lamport, Robert Shostak and Marshall Pease, “A note of distributed computing” by Samuel C. Kendall, Jim Waldo, Ann Wollrath and Geoff Wyant and “Harvest, yield, and scalable tolerant systems” by Armando Fox, Eric A. Brewer which I mentioned before in “10 papers every architect should read”

There are a also a few additional papers that are not in that list and that I found illuminating:

“Architectural Styles and the Design … Read More »

This is a rehash of a topic I posted about back in 2008 but it is just as relevant today.

When you work towards a release or some other looming milestone. There’s that  urge to leave stuff behind, cut some corners – you know, just a wee bit, no harm done – so we can ship the damn thing already. Mostly you’d fight that urge. But sometimes you’d want to make a conscious choice to make a shortcut because, well,  sometime delivery is more important

A little drop in quality in a single piece of code will not do much harm…for a while. There are, however, two problems with going this approach. One is that it is a slippery slope, i.e. it is probably not just one piece of code that gets this treatment in your code base. The second problem is … Read More »

As you may know last year I joined Nice systems where I lead the architecture team of our big-data multi-channel analytics platform. Well, we only have 4 architects in the team and we need at more. The current open position is for a senior architect for the application development group.  We’re looking for someone who is passionate about technology, a mentor and problem solver.  We need someone who is not afraid to write code ,make stuff work and pair with developers on one hand but can also  lead the system’s vision and design architectures on the other hand.

Please note that the position is in Raanana, Israel

Oh yes – here’s the official blurb:
Responsibilities:

Lead and design architecture for new enterprise applications and its infrastructure
Define, Lead POCs and choose technologies
Be a technical authority, design and document best practices, educate and support the development teams

Requirements:

Team player
Good English
At … Read More »

I recently answered this question in Stackoverflow :

What is an utility tree and what is it’s purpose in case of Architecture tradeoff analysis method(ATAM)?

 I did answer the question there but here’s a better explanation with lots of examples base on the initial version for chapter 1 of SOA Patterns (which didn’t make it into the final version of the book).

There are two types of requirements for  software projects: functional and non-functional requirements. Functional requirements are the requirements for what the solution must do (which  are usually expressed as use cases or stories). The functional requirements are what the users (or systems) that interact with the system do with the system (fill in an order, update customer details, authorize a loan etc.).

Non-Functional requirements are attributes the system is expected to have or manifest. These usually include requirements in areas such as performance, … Read More »

It all started with this odd bug…

One of our teams is writing a service, that among other things, runs map/reduce jobs built as pig scripts with Java UDFs. The scripts accepts CSV files which have a text header followed by lines of data. It performs some grouping and then calls a UDF which essentially filter, enrich and transforms the data outputting another CSV with a new header followed by data – something like the following:

1
2
3
4
input = load ‘/data/INPUT2.dat’ using PigStorage(‘,’)…
grouped = GROUP input BY key;
results = FOREACH grouped GENERATE evaluateUDF(input) as output;
STORE output…

and all was well. The job spawns a few map tasks partitions, groups the data and runs a single reduce where the actual evaluation happens.

Then someone wanted it to run faster – We can do that by adding reducers we can do that by adding PARALLEL X to the … Read More »

I want to say I am delighted but I guess relieved is the right word here  - I’ve finally submitted all the book chapters to manning and I’m now doing an editing round on all the chapters (re-adding missing images, update pattern references, fixing typos etc.).

The current (and hopefully final) table of contents  is as follows:

Part I – SOA Patterns

Chapter 1 – Solving SOA pains with patterns – an introduction
Chapter 2 – Basic structural patterns
Chapter 3 – Performance, scalability and availability patterns
Chapter 4 – Security and manageability patterns
Chapter 5 – Message exchange patterns
Chapter 6 – Service consumer interaction patterns
Chapter 7 – Service integration patterns

part II – SOA in the real world

Chapter 8 – SOA anti-patterns
Chapter 9 – Putting it all together – a case study
Chapter 10 – SOA vs. the world – how SOA fits with other techs:  REST, Cloud and Big data

As part … Read More »