Self-learning is overrated (in programming) (October 10, 2010)
Few weeks ago I had series of interviews in two top-tier investment banks. I was applying for development position in similar departments, but the technology stacks those two banks were using varied wildly: from hardcore low-level multithreaded Java in one to Haskell/F# in another. The outcome of those interviews provoked some thoughts abouth whether self-education means that much when a well-paid position in a respectable company is at stake.
First of all, banks are always a good indicator of the position’s value in business terms. Startups may hire people to explore the previously unknown areas, to experiment with products that might have very little or no monetary value at the moment. To the contrary, banks are dealing with the established business models and are rarely tolerating even short-term decreases in the added value that every person brings to the company. They hire people to earn money for the bank, not to experiment, and they pay those people accordingly. This usually sets the interview direction from the very beginning.
I have around 10 years of industrial experience with Java, interruped by few years of enterprise C# development. So despite being exhaustive (two people on the other end of the phone line asking one question after another in the course of hours), interviews for the Java position went pretty smooth. How to implement a fair lock? What’s a GC generation and what are they used for? What would you do if you need soft-realtime Java system? How to avoid resource contention? Deadlock? Livelock? Can you implement locking without using synchronized keyword? Synchronized container that will use locking less often than a standard one? And so on and so forth.
Fortunately, most of those question constitute part of my day job, so it was nice to have this brain excercise. Not so with Haskell/F# interviews. I didn’t had a chance to gain any industrial experience with functional programming — I bet lots of people practically interested in FP didn’t. I was asked about type inference results for some specific function combinations, possibility of Haskell program to be as fast as C program, type covariance and contravariance in .NET, the way to control state and side effects with monads, existential types, a bit about the way Haskell supports multithreading, short “what is STM?”, and so on.
Shortly after the interview I found a polite “no” in my mailbox. I myself wasn’t satisfied with my performance, so this wasn’t a surprise. At this point I was happy that my experience with resolving all sorts of problems in the real industrial environment gave me a chance to learn what I know about Java. I felt what it is like to be in shoes of self-learner. One bright guy who fleed from the enterprise development to the AI-related work wrote this:
Learning AI (or any deep comp.sci for that matter) is not like learning J2EE or Ruby DSLs or whatever the fad du jour in the enterprise software world is — read a few chapters of the latest bestselling “pragmatic” book, write some crappy web site and hey presto you are the expert.
“Real” comp.sci doesn’t quite work like that
Real industry does not work like that too. I can’t imagine a self-learner to know the nuts and bolts of Java GC, just as I didn’t felt too confident about existential types. I simply never had an external pressure that would throw me into the necessity to know ins and outs of Haskell by heart, and pragmatic books (Haskell, as all you know, have one) no matter how good, do not help with that. Self-learner is like a fresh college graduate: he simply has to start his way from the lowest positions again.
No amount of self-learning can come near years of work in a mission-critical environment when company’s revenue is at stake.
PotD 2: Yago Hortal (September 6, 2010)
Pp32 by Yago Hortal (2009)
Getting to “bare metal” (September 2, 2010)
Have recently finished reading Peter Seibel’s “Coders at Work”. If you haven’t heard about this book yet, it’s a very well-written collection of interviews with some of the most influential people in our industry; the names of those author talks to vary from Crockford and Bloch to Norvig and Knuth. Apart from being a very inspiring read, the book gives a very interesting perspective on some questions Peter keeps asking over and over. “Do you consider yourself a scientist, an artist, an engineer, or a craftsman?” “Should all progammers be able to deal with low-level things even despite that this ability is rarely needed nowadays?” “Do you still write code?” The question on low-level programming is among the most interesting ones. Are there any differences in one’s performance with and without that knowledge?
I guess there are. Without having a big desire to fall back to the original topic of my blog, I want to point to a recent issue we had with our biggest SLEE-based project. This is the first project where we are using big branchy trees of SBBs in our application and the first one that showed such a bad results during the initial performance tests. What is happening? Why is it so slow? After several nights and weekends spent staring at the CPU profiler and thread dumps and a help from our vendor, the root cause of the problem became clear.
When the event flow is being delivered to the SBBs in the same tree, event router thread does not have any choice except for locking the whole tree. After all, there’s a SLEE transaction ongoing and the state of the SBB tree should be kept consistent. Other event router threads at that moment are waiting for the tree to be freed, even if they try to deliver the event to another SBB in the tree. More blocking time, higher latency, lower performance.
Interesting is that JAIN SLEE just like any similar event-driven framework tries to isolate the developer from threading almost completely. But it’s clear that a
developer can’t even diagnose a problem like ours without resorting to knowledge about thread states. So personally I have no doubts about the necessity of this knowledge.
And I think our playful age of Mongrel and Cucumber gives more opportunities to learn low-level things with less pain. Ruby folks seem to discover fibers lately (green
threads whose execution is controlled programmatically) so there will be a lucky few who will master the art of its scheduling. I myself had a chance to play with the
Java emulator of x86 PC — seriously, having the possibility to compile your C program, take the binary, and see how
Processor.processProtectedModeInterrupts() is executed in your Java emulator is astonishing.
I’m excited to read about other people thinking the same.
David Heinemeier Hansson in Jason Calacanis’ This Week in Startups (March 22, 2010)
A video of the most recent This Week in Startups episode with David Heinemeier Hansson started some heated discussion on the web lately. Even if you have not been a part of the debate, I strongly encourage you to watch this video (the interview starts after some 47 minutes of discussion; the video below skips them.)
The reader might be familiar with the David’s philosophy of doing business: it emphasizes profits (they are more important than market share, investments or anything else) and simplicity (minimalistic approach to products is the way the company could stay focused on delivering it and being frugal.) He sees the way chosen for 37signals as universal. When he evangelizing it, he is fastidious in his choice of words.
While it’s easy to fall for his “bad boy in the startups world” charisma, the interview gives a stark contrast between David’s tendency for overgeneralization and genuine experience of Jason. David is thirty and his interview host, Jason Calacanis, is ten years older. When placed together, they reveal the most attractive traits of each other’s personalities.
What I have enjoyed most in this video:
- How masterly Jason leads the discussion of some sorts of business models to its culmination, and how he ends this with a strong counter example like “Oracle might disagree”
- A discussion of work ethics. David: I always take Friday off. I prefer to work smarter, not harder. Jason: do you know what kind of schedule Steve Jobs enforced in Apple in the early days of PCs?
- How Jason constantly repeats “interesting” on almost all David’s remarks, and after somebody tweets about that in the middle of the show, and Jason reads the tweet aloud, he never repeats this word again. This is what I call a brilliant speaker.
This line is a signal to scroll up and click Play. Enjoy.
Wrapping-up the architects topic (March 17, 2010)
The previous post on software architects gained enormous amount of views and comments. I guess I hit the nerve here; the nerve being the career management for tech people. Except for the selected few in the programming world (like Guido van Rossum or Rob Pike or Joshua Bloch) there are almost no engineers widening their field of influence without becoming entrepreneurs, just by being employed by someone. Now I am certain that I am not the only one bothered by that.
Among other insightful things, the comments for that post brought a link to Who Needs an Architect?, a Martin Fowler’s article. I can’t resist the temptation to cite Ralph Johnson mentioned by Martin in the text
One of the differences between building architecture and software architecture is that a lot of decisions about a building are hard to change. It is hard to go back and change your basement, though it is possible.
There is no theoretical reason that anything is hard to change about software. If you pick any one aspect of software then you can make it easy to change, but we don’t know how to make everything easy to change.
Martin writes a bit earlier referencing the talk by economist Enrico Zaninotto
One aspect I found particularly interesting was his comment that irreversibility was one of the prime drivers of complexity. He saw agile methods, in manufacturing and software development, as a shift that seeks to contain complexity by reducing irreversibility — as opposed to tackling other complexity drivers. I think that one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs.
Most of the efforts behind the software architecture activity are focused on removing the need for it. I can’t be fascinated more about the extent how this fits into my vision.