To make a long story short, I'm building a trivial query engine over a dataset using XPath; for expressiveness purposes, I've allowed the users of said engine to query for attributes, and assumed that I can simply go up the hierarchy from there and access the elements directly.

I was somewhat dumbfounded to find that each node in the resulting node set had its parentNode property set to null; at first I was sure this has to do with an implementation detail of MSXML3 (perhaps it returns copies of the attributes for... I have no idea what possible gain could be derived from this.) A little more digging proved that, and I quote from MSDN:

In C/C++, IXMLDOMAttribute inherits IXMLDOMNode but are not actually child nodes of the element and are not considered part of the document tree. Attributes are considered members of their associated elements rather than independent and separate. Thus IXMLDOMAttributeparentNode, previousSibling, and nextSibling members have the value Null.

Working under the assumption that this is an MSXML3-only problem, I digged a bit into the W3C DOM specification only to find that it is, in fact, a specification issue:

Attr objects inherit the Node interface, but since they are not actually child nodes of the element they describe, the DOM does not consider them part of the document tree. Thus, the Node attributes parentNode, previousSibling, and nextSibling have a null value for Attr objects. The DOM takes the view that attributes are properties of elements rather than having a separate identity from the elements they are associated with; this should make it more efficient to implement such features as default attributes associated with all elements of a given type.

This seems to me a completely arbitrary design decision. It makes absolutely no sense to inherit from what at first glance is an interface for first-class citizens in the document (Node) and then castrate the interface with useless implementations. If, contractually speaking, attributes never have siblings or parents, then they shouldn't have those properties at all, otherwise this just frustrates the developers wishing to make use of those properties, and wastes their time by forcing them to dig through documentation to figure out what's wrong.

And on top of it all, the decision doesn't even make sense in light of the "efficiency" claim - how is it more efficient to partially implement an interface, instead of defining a new one that's more contractually sound?

Quick link: To download the wrapper classes click here

While working on a large application targetting the .NET Compact Framework 2.0 I realized that I'll need to feed some native code (specifically, the XSLT processor in MSXML 3.0 SP1) with an IStream implementation.

Articles about interoperating with unmanaged code in the CF are not exactly abundant; to save you the time I spent on incorrect and/or conflicting research, here's the bottom line:

• There is no Managed C/C++ for .NET Compact Framework 2.0. To some this may be old news, but you should really pay attention to this point if you're going to do any serious development against CF. 2.0 adds support for managed COM/ActiveX interop, but otherwise you're completely stuck with P/Invoke.
• Although it's not immediately obvious, CF 2.0 does support exposing managed classes via COM.
• The CF is missing some usually-minor classes from the BCL; in this case I was missing System.Runtime.InteropServices.ComTypes.IStream. Annoying but easy to work around.
• Finally, as an aside, MSXML 3.0 SP1 for Windows Mobile does not support the IXslTemplate and IXslProcessor interfaces, meaning that MSXML 3.0's already lackluster performance in XSLT transformations is further hindered by not being able to cache the interpreted stylesheets. This means that, if you use XSLT, your application will not scale. I was not initially aware of this issue, so I hope our data sets are small enough to handle this, or I may yet come to regret the decision to use XSLT in this project.

I managed to save quite a bit of time by leveraging Oliver Sturm's work, which was originally intended for the desktop. Since the CF is missing a whole bunch of minor classes, the managed definition of IStream included, I originally mucked about with midl trying to generate these definitions from the .IDL files. After this proved to be a genuine chore, I just ripped the definitions straight out of the .NET Framework 2.0 assemblies with the ever-useful Reflector.

You can download the class file here. If you use this it would really be cool if you could drop me an e-mail, and I bet Oliver would be equally appreciative. Enjoy!

God knows I give the guys at JetBrains a lot of credit, but I didn't see this coming: according tot he Omea News feed, Omea Pro is being open sourced!

For those of you not in the know, Omea Reader is JetBrains' RSS, newsgroup, e-mail etc. aggregator. It's a pretty amibitious application that I've been using for quite a while now instead of RSSOwl (Omea's interface is smoother, although RSSOwl definitely has its moments), although in reality I only use about 10% of its capabilities - newsgroups and RSS feeds.

I was about to buy Omea Pro this week and am now feeling really awkward that I can download it for free instead; the least I can do is spread the word. I suggest you take a serious look at it, because the non-Pro Omea Reader is already a very formidable product.

Update: Seems to work fine now, please let me know by e-mail if there are any problems (tomer at tomergabel dot com).

Apparently there is some sort of problem with the comment system, and it breaks in at least one case. I'm working on the problem, in the meantime feel free to use any of the other methods of contacting me if you need to.

As I mentioned before, I've had well over 300 trackback/pingback spam notifications from dasBlog. Since this was well beyond what I was willing to mess with by hand, I whipped up a quick Outlook macro to do the work for me:

Sub DeleteTrackback()
    Dim oSel As Outlook.Selection
    Dim oItem As Outlook.MailItem
    
    Dim oShell
    Set oShell = CreateObject("Shell.Application")

    Set oSel = Application.ActiveExplorer.Selection
    For x = 1 To oSel.Count
        Set oItem = oSel.Item(x)
        
        If (Left(oItem.Subject, 19) = "Weblog trackback by") Or _
           (Left(oItem.Subject, 18) = "Weblog pingback by") Then
            Index = InStr(1, oItem.Body, "Delete Trackback:")
            If (Index <> 0) Then
                URL = Mid(oItem.Body, Index + 18)
                URL = Left(URL, Len(URL) - 1)
                
                oShell.ShellExecute URL, "", "", "open", 1
            End If
        End If
        
    Next
End Sub

To use this macro, create a new macro and paste the source code; then select all the "Trackback/Pingback" notifications messages and run the macro. It could obviously be customized to work on entire folders or whatever, but that I leave to you. One final suggestion: if you (like me) keep a 15-tab Firefox window open at all times, you may want to open a new window (not tab, window!) so that you can then close all the URLs at once.

Ever since I upgraded to dasBlog 1.9 this blog was quite literally FLOODED with trackback spam. At first I tried to cut down on the spam using the various available techniques (IP-based blacklists, word-based blacklists, 404 responses to offenders), but unlike e-mail bayesian filtering, it's far more difficult to properly rate seemingly innocent URLs. Even if the current crop of spambots are relatively stupid and use blacklisted words in the page title, the next generation is bound to be more obscure.

At any rate I couldn't cope with the volume of spam (I have well over 200 spam trackbacks waiting to be deleted, and that's just from the last few days) and decided to turn off trackbacks and pingbacks. I wish I didn't have to do this - in many ways it feels like switching off what makes blogging special to begin with - but I don't have the time to deal with the impossible amounts of spam.

If anyone has any ideas on how to resolve this without turning off trackbacks, though, I'm definitely willing to give it a try...

I got a heapload of new music from a friend (thanks, Gil!) and wanted to share a couple of recommendations:

• The Flower Kings - a Swedish progresive rock band. I actually made the mistake of thinking them an American band (something about the style struck me as US-oriented), but regardless they really are very good at what they do. I've given serious listening time to Unfold the Future and it's straight at the top of my to-buy list.
• KBB - a relatively anonymous Japanese progressive band. Their first album "Lost and Found" (1999) is positively brilliant. I hope to catch one of their concerts at some point in Tokyo, but the site is completely out of date.

If anyone's interested in progressive rock and/or has recommendations to add, I'd be delighted to hear (just send me an e-mail or drop a comment).

The previous post in this series was an overview of the various communication hurdles developers and managers face. Here I will describe one of the methods we use at Monfort to tackle these issues.

We've been working very closely with one of our clients on a large variety of projects, most of which are based off of a shared technological foundation (I suppose you could call it a "framework," although it technically isn't). These projects generally fall into one of three categories:

1. Improvements to our technological infrastructure. These are either extremely large projects (measured in man-years) or fairly small projects (usually several weeks);
2. Product derivatives, demonstrations and platform ports that make use of the pre-existing technological infrastructure;
3. New products that make little or no use of the pre-existing infrastructure.

Over the years the volume of work we do for this client has increased considerably, creating the need for additional personnel in order to meet the demand. The caveat is, obviously, increased managerial overhead. This can take one of two forms:

1. Project management, in which a manager is directly related to all aspects of a particular project. As soon as an order is placed for the project, the manager is responsible to bring the project to fruition. More projects equate more project management;
2. Customer relations, in which a manager is responsible for maintaining contact with the client and providing the initial technical contact point. The person in charge has to have a very deep technical grip of the relevant technologies and be able to communicate effectively with both business and engineering personnel (the client's representatives as well as their customers). Finally, he has to maintain constant vigilance so that new projects will actually move beyond the initial "what if" stages.

Although good project managers are hard to find, the real problems became evident only when we added personnel to the customer relations position, which up until that point was (for the most part) filled by just one person. The first hurdle was in bringing additional people up to speed; the relevant knowledge was kept in e-mail archives and the heads of several people, which adds up to very inefficient indexing. A newly commissioned customer relations manager has to know a lot of non-trivial details: the representatives of the various clients he'll be working with, the work methodology against a variety of clients and similar information. The issue of knowledge sharing becomes a much bigger issue when the manager moves beyond that point and goes out to the field: he has to be kept up-to-date on all concurrent projects for those customers, prospects for future projects and the various proposals and discussions that had taken place, whether he was involved or not. When two or three different people all share in those responsibilities, they have to be constantly synchronized

This is a lot of information, and it became very obvious very quickly that we needed some system to manage and store it. After spending a lot of time thinking about this, I arrived at a fairly coherent list of requirements from the knowledge sharing platform we will employ:

• Any one of the customer relations managers should be able to freely access and update information;
• There should be no artificial limitations to the way information is written, presented or interlinked;
• The platform should be easy, if not trivial, to learn and use;
• The platform should be web-based and accessible from everywhere (we often require access to information for client sites).

Eventually, I came to the conclusion that the only knowledge/content management system that could actually work in our corporate environment is a wiki.

If you've been living under a rock for the past few years it's possible that you've managed not to hear about Wikipedia; the collaborative encyclopedia is one of the most ambitious and impressive projects to have attained a degree of success on the Internet. What distinguishes Wikipedia from similar, less successful projects is that the knowledge stored in Wikipedia is completely freeform: anyone can edit it, there are no predetermined schemas and no mandatory information fields - anyone can enter data in the way they deem fit. Here-in lies both the beauty and danger of a wiki: it doesn't brute-force the information into patterns. This means that knowledge can be shared in any conceivable way and content editors are free to interlink this knowledge in any way they deem appropriate. The danger here is that most people aren't disciplined enough to invent their own patterns. In our case this did not pose a problem, since only a small number of people were expected to be involved in the effort.

In the next post: installing, configuring and learning to effectively use the wiki.

Scenario: a couple friends and I want to go on vacation. So we go to a local travel agency's website and order a bunch of flight tickets. I give my credit card information, and a few seconds later I get a "transaction complete" notice and merrily call my friends to let them know that the deal is sealed.

Half an hour later, I get a phone call from said company. Oh they're so very sorry, but actually the flight is booked and they can't give me my tickets. Instead they offer a cheaper flight which either leaves or returns at a different time, or a much more expensive ($90 per person) flight instead of the one we were interested in. Bait and switch? Who knows, but working under the assumption that their ass is covered I took the liberty of examining their terms of service page (warning: Hebrew) and managed to find some very interesting bits (my own translation):

• The part of the document where you "authorize" the company to bill you also includes, at the very end, a statement in which you (the customer) say you are "interested in details regarding call-card benefits outside the country, and hereby agree to be contacted by a representative of [the travel agency] or [associated phone carrier]." I would think that this is the sort of opt-in they would at least provide a check-box for.
• Regarding hotels: "For your information, the supplier reserves the right to transfer you to an alternative hotel of similar rating, or higher for an extra payment of up to $100 per person... The hotel rating is according to the local Tourism Office and should not be used for comparing hotels in different countries." So basically, the carrier can (at their whim) screw you over, which may even incur additional charges.
• Still on hotels: "In the case where, as a result of changes to the flight schedule, the customer loses prepaid sleeping privileges at the hotel or additional costs are incurred, [the travel agency] will not be held responsible to said costs." I wonder who is responsible - the flight carrier? I seriously doubt it, their ass is probably just as well-covered.
• Otherwise, the document is basically full of "we're not responsible if" statements. Beautiful, I don't think the travel agency can be legally held responsible for just about anything.

Aside from being seriously pissed off at having my vacation ruined before it even started, it really annoys me that the various travel agencies in Israel are perfectly OK with screwing their customers over. The fact that it's legal (and maybe it damn well shouldn't be. Doesn't this fall under the definition of "false advertising?") doesn't make it any more reasonable. I hope the situation isn't quite as bad elsewhere.

(This is the first in a series of posts, in which I will try to articulate the various problems of communication in the software field and possible ideas on how to solve those problems.)

One of the most interesting aspects of being a software developer is communication. Perhaps the single most important trait of a good software developer is the ability to communicate with varying audiences:

• First and foremost, a software developer must be able to communicate effectively with the other developers on his team. The usual tools apply: e-mails, whiteboard, hall meetings, source code comments. For a developer this is relatively easy, since it is unnecessary to translate the technical abstractions into a coherent conceptual model: the abstractions themselves are familiar to all sides of the conversation.
• A software developer must also be able to communicate effectively with his team leader, project manager, boss - whatever you want to call it. This isn't quite as easy, as such communication can have serious repercussions on the project, the work environment and - potentially - the developer's career. Managers are usually not privy to the ultra-technical subculture of software developers, and although they almost always come from the same background they have just as often given up on subtle technicalities.
• Finally, a software developer must be able to communicate effectively with clients. At some point in almost every developer's career he is required to meet and at least converse with a client, sometimes in order to figure out a bug, sometimes in order to gather requirements for a new project. A lot of developers would rather avoid these encounters because they are forced to re-think everything they say and translate abstractions. A client is not, generally, heavily interested in technical matters, so these must be explained carefully and coherently; most developers I know find this process frustrating and draining.

A developer that delves into management is assumed to already be proficient in the above scenarios, but management brings a whole new slew of communication problems to the table:

• A manager is required to communicate with clients on a whole different level. When a developer is asked to make an estimate, the estimate forms a basis for the cost and schedule estimates his manager provides to the clients. The difference is that the manager is required to explain these estimates and is personally accountable to same. It is the manager's job to continuously communicate with the client, reassure, explain and assume responsibility for whatever pitfalls and hurdles the project encounters, and generally make sure the project is on track. A manager is also required to act as a filter between the client and the developers on the team for everyone's sake: on the one hand the manager is required to "protect" the developers and the project from ridiculous requests and whims on the client's side, and on the other hand he is required to make sure the client's requirements are fulfilled to his satisfaction.
• A manager must communicate effectively with the developers on his team. He must establish trust and confidence, so that he can constantly be aware of what goes on with his team while avoiding micromanagement and wasteful status meetings. This isn't easy because the manager and the developer are fundamentally on opposite sides: efficiency vs "doing the right thing", choosing wisely instead of following trends, cutting corners instead of continuous refactoring and improvement. The only glue they have is the desire for a successful project. A good manager must communicate his decisions to the developers on his team to constantly maintain the trust and confidence he has built; a frustrated or uptight developer is an inefficient developer.
• Finally, a manager must be able to communicate effectively with other managers. This is the least trivial form of communication for a variety of reasons: sometimes the corporate atmosphere requires managers to be ultra-competitive. It is hard working for a shared goal ("the good of the company") when one is constantly competing with one's peers. Even lack of competition can be an issue when two managers from different schools of management must collaborate, or when one manager is frustrated by the other's inefficiency. And even when all managers are equally efficient and equally dedicated to the shared goal, they must still share information between them and make sure the collective company process is on track.

You'll notice that the two lists are in opposite order. This is no coincidence; I ordered the lists so that the more difficult of the tasks are further down the list, so for developers the most difficult form of communication is usually with customers, whereas for managers it's communication with other managers.

The next post will deal with knowledge management and how we've used a wiki internally in our organization to alleviate the overhead of communication between managers.