I take pride in being one of the few people I know who actually buy their media: I have a sizable collection of CDs, DVDs, computer games and software that I've bought over the years, and I always feel good about having paid the people responsible for these efforts.

Until recently, that is.

It is commonly said that one of the most obvious traits of Israelis is that they hate to be screwed, and this is as true for me as it is for everyone else. It seems the media companies have taken upon themselves to screw me in every conceivable way, and paying for media is fast becoming an exercise in frustration for me. A most recent example of this is Valve's not-so-new-and-shiny content delivery network which goes by the name of Steam. I don't even know where to begin recounting what's wrong with this thing:

1. Content delivery speeds are abysmal. I recently downloaded Half Life 2 Episode Two and got 200K/sec maximum transfer rate (more common rates hovering around 50K/sec) on a dedicated line with 5Mb downstream. I consistently get 300K+ rates to even the most busy content delivery servers (Akamai, Microsoft etc.) and it's not like I can use a download manager to better tune the download to my connection.
2. The download manager is shit. Even ignoring the fact that the only controls it exposes are "pause" and "resume" doesn't help the fact that the error detection code is buggy as all hell: the first time I tried downloading the game it got stuck on 99% without any type of diagnostic or error message, and wouldn't resume. Reading piles of angry forum threads led me to the conclusion that the downloaded content files are simply corrupt; deleting and re-downloading the game solved the problem.
3. Terminology is all screwed up: telling the game manager not to automatically download updates for a certain game will pause any pending download for that game, including the game content itself.
4. Although there is no apparent reason for this, playing a game pauses the downloads for all other games. That, at least, has been my observation (Episode Two was downloading when I started on Episode One, and hasn't progressed a single per cent when I quit the game).
5. The application itself is completely opaque. At no point does it give any indication of what it's doing; you can start the client, nothing happens for two minutes until it finally shows you an "updating Steam client" window. There are no visible clues when it's attempting to access a server (e.g. when clicking on Show News) or when a downloaded upgrade is being installed.
6. I don't want to connect to a server to play a locally installed, legally bought game. That's just unforgivable, even if it didn't mean I sometimes have to wait for several minutes before the server actually logs me in instead of timing out.
7. It might shock you, but I still play old games. Sometimes very old games (think Master of Magic). Will Half Life 2 be playable in five- or ten-year's time when the Steam servers have long been cold? I doubt it.

I know Steam probably works well for a lot of people, but for me it's a god-damned affront: I'm a paying customer, there's no reason why I should have so little control over a game that takes up gigabytes on my hard drive. To add insult to injury, the pirated versions often work better: the pirated version of Half Life 2 itself had considerably lower loading times, didn't suffer from the audio stuttering issues that plagued the original, and didn't waste hours of your CPU time on decrypting the game content once it was finally downloaded. If Valve wants to keep my business, here's what they should do:

1. Switch to an open distribution model (HTTP or, preferably, BitTorrent) so I can use my own software to download their games if I so wish;
2. Get rid of the dependency on Steam for their games. When I click on the HL2E2 icon I want the game to come up, and I don't give a rat's ass about Steam;
3. Move to an asynchronous, transparent update mechanism for their games, preferably one that allows me to download game updates and install them on my own.

With the original versions becoming increasingly irritating and pirated versions becoming better than the originals (not to mention less costly), does paying for media still make sense? Remember, that's just one example, I could give a great many more.

The Java implementation for generics is radically different from the C# equivalent; I won't reiterate issues that have been thoroughly discussed before, but suffice to say that Java generics are implemented as a backwards-compatible compiler extension that works on unmodified VMs.

The implications of this are considerable, and I'd like to present one of them. Lets fast forward a bit and consider a relatively new language feature in Java (introduced, I believe, with J2SE 5.0): autoboxing. A thoroughly overdue language feature, autoboxing allows the seamless transition from regular value types (e.g. the ubiquitous int) to object references (Integer); before autoboxing you couldn't simply add a value to an untyped ArrayList, you had to box (wrap) it in a reference type:

ArrayList list = new ArrayList();
list.add( 3 );			// Compile-time error
list.add( new Integer( 3 ) );	// OK

Eventually Java caught up with C# (which introduced autoboxing in 2002), and with a modern compiler the above code would be valid.

With the introductions out of the way, here's a pop-quiz: what does the following code print?

HashMap<Integer, Integer> map = new HashMap<Integer, Integer>();
map.put( 4, 2 );
short n = 4;
System.out.println( Integer.toString( map.get( n ) ) );

As a long-time C# programmer I was completely befuddled when the code resulted in a NullPointerException. Huh? Exception? What? Why?

It took me a while to figure it out: because Java generics are compile-time constructs and are not directly supported by the VM, what actually happens is that the underlying container class accepts regular Object instances (reference types); the compile-time check merely asserts that n can be promoted from short to int, whereas the actual object passed to the container class (via autoboxing) is a Short! Since the container doesn't doesn't actually have a runtime generic type per se, the collection merely looks up the reference object in the map, fails to find it (I guess the Object.hashCode implementation for value types simply returns the reference value as the hash code as in C#) and returns null. Doh! *slaps forehead*

A few ReSharper tips for your viewing pleasure (according to rumors it can extend your penis too!):

Alt+Shift+L locates the current file in the Solution Explorer (at least with the IDEA-style keyboard bindings we all know and love). This is particularly useful for extremely file-heavy solutions where you often want to Get Latest (or Undo Check Out) the current file.

Note: you can get this behavior automatically through Tools->Options->Projects and Solutions->Track Active Item in Solution Explorer, but in my opinion it's annoying as hell.

Easier unit testing: you can bind shortcuts to run or debug unit tests by context. Context means that the behavior is exactly as expected: using the shortcut with the cursor on a test will run just that test, using it on a class will run it as a whole fixture, and using it on a directory in the Solution Explorer will run all tests that reside in that directory. This is easily accomplished by going to the Tools->Options->Keyboard menu and binding:

• ReSharper.ReSharper_UnitTest_ContextDebug to whichever shortcut you wish for debugging purposes (my favourite is Ctrl+T, Ctrl+D)
• ReSharper.ReSharper_UnitTest_ContextRun to whichever shortcut you'd like to use in order to just run tests (my favourite is Ctrl+T, Ctrl+T)

Note: You can get the same behavior with the ubiquitous TestDriven.net, but it makes no sense to me to pay twice for the same functionality.

Bits and pieces:

• Apparently everything is closed on Christmas day, which I suppose is obvious to you unless you live in a predominantly Jewish or Islamic country in which Christmas isn't really celebrated.
• Spamalot is brilliant.
• Manchester is a really cool city with terrific pubs and shopping districts.
• The dominant ethnic group in London is not, in fact, cockney brits, but rather Indians (i.e. immigrants from India).
• I'll post some pictures as soon as I upload them to Flickr.

Bottom line: As promised. Would buy again.

If you use Executor (a freeware launcher utility), check this out. There's even a video showing it in action!

I'm flying to England tomorrow for a week's vacation, which will hopefully give me a bunch of ideas what to write about (it's quite difficult for me not to focus on my current area of work, which I doubt would be of much interest to readers...)

If you happen to be in London or Manchester some time within the next week, get in touch and maybe we'll get together for a beer

Quick link: download

In my work at Semingo I often encounter situations where it's impossible to unit- or integration-test a component without accessing the web. This happens in one of two cases: either the component itself is web-centric and makes no sense in any other context, or I simply require an actual web server to test the components against.

Since I firmly believe that tests should be self-contained and rely on external resources as little as possible, a belief which also extends to integration tests, I wrote a quick-and-dirty pluggable web server based on the .NET HttpListener class. The unit-tests for the class itself serve best to demonstrate how it's used; for instance, the built-in HttpNotFoundHandler returns 404 on all requests:

    [Test]
    [ExpectedException( typeof( WebException ) )]
    [Description( "Instantiates an HTTP server that returns 404 on all " +
                  "requests, and validates that behavior." )]
    public void VerifyThatHttpNotFoundHandlerBehavesAsExpected()
    {
        using ( LightweightWebServer webserver = 
            new LightweightWebServer( LightweightWebServer.HttpNotFoundHandler ) )
        {
            WebRequest.Create( webserver.Uri ).GetResponse().Close();
        }
    }

The web server randomizes a listener port (in the range of 40000-41000, although that is easily configurable) and exposes its own URI via the LightweightWebServer.Uri property. By implementing IDisposable the scope in which the server operates is easily defined. Exceptions thrown from within the handler are forwarded to the caller when the server is disposed:

    [Test]
    [ExpectedException( typeof( AssertionException ) )]
    public void VerifyThatExceptionsAreForwardedToTestMethod()
    {
        using ( LightweightWebServer webserver = new LightweightWebServer(
            delegate { Assert.Fail( "Works!" ); } ) )
        {
            WebRequest.Create( webserver.Uri ).GetResponse().Close();
        }
    }

The handlers themselves receive an HttpListenerContext, from which both request and response objects are accessible. This makes anything from asserting on query parameters to serving content trivial:

    [Test]
    public void VerifyThatContentHandlerReturnsValidContent()
    {
        string content = "The quick brown fox jumps over the lazy dog";

        using ( LightweightWebServer webserver = new LightweightWebServer(
            delegate( HttpListenerContext context )
                {
                    using ( StreamWriter sw = new StreamWriter( context.Response.OutputStream ) )
                        sw.Write( content );
                } ) )
        {
            string returned;
            using ( WebResponse resp = WebRequest.Create( webserver.Uri ).GetResponse() )
                returned = new StreamReader( resp.GetResponseStream() ).ReadToEnd();

            Assert.AreEqual( content, returned );
        }
    }

We use this class internally to mock anything from web services to proxy servers. You can grab the class sources here -- it's distributed under a Creative Commons Public Domain license, so you can basically do anything you want with it. If it's useful to anyone, I'd love to hear comments and suggestions!

Well the title is actually a semi-private joke, but the point of the post is to draw attention to long-time friend, coworker and Mentor cofounder Shlomo Priymak's new blog. Shlomo is our sharp-but-misanthropic DBA, precisely the kind of person you'd want to pay attention to for hardcore MySQL (and other) problems and solutions.

Speaking of Semingo, we're gearing up to a relatively close alpha launch and have a new corporate website. Now would still be a good time to hop on the bandwagon and join a fast-growing company made up entirely of crazy-ass people out to do the implausible. If this sounds right to you, get in touch! (keywords: web 2.0 startup search .net java developers qa algorithms nlp and others)

With the web host giving me trouble, just to add insult to injury the comment captcha generator stopped working. I don't know how long it's been this way and I sincerely hope it's a new problem; at any rate I disabled captchas and added Akismet spam filtering in the hopes that it'll keep everyone comfortable and the blog clear of spam...

I'm getting a little tired of all these issues with dasBlog (I still haven't been successful in configuring it on the new webhost) and am seriously considering replacing it; I'm basically really happy with the application, but configuration and installation issues are taking a little too much of my time. If anyone has an easy-to-use platform in mind, preferably one with a straightforward migration path, I'd appreciate the suggestion.

So you want a web project, a build system and a reasonable IDE to take care of the annoying details for you, right? The good news are that it's actually quite possible and there're many ways to do this. The bad news are that it's nigh impossible to get them to play along if you don't already know how to do that. It took me days to find a solution that finally seems to work, and I'd like to share it with you. I'm probably missing a few important details or did something really really stupid along the way (I'd appreciate comments!), but this process does seem to work. I'm not going into essentials of Java-based web development here -- if you want a more basic explanation of the terminology, post a comment and I'll see what I can do...

1. Goal

I want to:

• Use a common, standard and powerful IDE to edit and debug my Java code, and preferably provide a usable GUI interface for dependency and project management;
• Use a common, standard servlet container to host my servlet and still be able to control and debug everything from the same IDE;
• Have a convenient way to handle internal-and-external dependencies without worrying too much about the details;
• Be able to quickly compile, test and package my servlet for deployment;
• Understand as little as possible about the dependency stack of the tools involved

I'm going to tell you how to achieve most of these goals, with two glaring omissions: I won't show you how to do testing (I haven't successfully managed servlet unit testing so far -- different post on that) and I can't help but delve into some of the more annoying details involved with these tools and their dependencies. Sorry about that. Additionally, some of the information here applies even if you use different tools, but you're bound to face issues not covered here; don't assume I know more than you do -- seek the answers, post them somewhere, and maybe the next person will actually find what they're looking for!

2. Tools

The tools used are:

• J2SE JDK is an obvious must-have. Version used: JDK 6 update 3;
• Eclipse (but please don't download it just yet) for code editing, debugging and project management;
• Web Tools Platform: this is the Eclipse plug-in that adds web development capabilities to the IDE, including J2EE dependency management, hosting and running servlets from within the Eclipse workspace etc. This would be a good time to run along to the WTP web-site and download the Web Tools Platform All-In-One package. I only used the release version (2.0.1 at the time of writing this), so if you use another version your mileage may vary;
• Apache Maven is the newfangled build system from Apache slated to replace ant. I've used it for the last few days and so far it appears to be quite robust and even fairly well-integrated into Eclipse (see next item). Version used: 2.0.8;
• M2eclipse is the Eclipse plug-in for Maven integration. I've only found one problem with it so far, which I'll detail later on;
• Apache Tomcat is a solid choice in servlet containers. It's robust, fast and open-source, and has terrific Eclipse integration. I haven't given any of the other containers a serious whirl yet though.

3. Preparations

Unlike Visual Studio, with the tools mentioned above there's no straightforward installation procedure. You'll have to designate at least a workspace directory (where your Eclipse projects, settings etc. go) and some location where the tools themselves go. For me, it's C:\Dev\Eclipse and C:\Tools respectively.

• Setting up Java:
  • Install the JDK and remember where it was installed (nominally in %PROGRAMFILES%\Java\jdk1.6.0_03)
  • Set up a system-wide JAVA_HOME environment variable pointing to the same directory
• Setting up Maven and Tomcat:
  • Extract both archives to your designated directory (e.g. for Maven it would be C:\Tools\apache-maven-2.0.8)
  • Add the Maven bin directory to your PATH environment variable (user- or system-wide, depending on your preference)
  • Add whichever J2EE libraries you desire from the Tomcat installation to your class-path. If you have no idea what I'm talking about, you'll probably just want to set the CLASSPATH environment variable to your equivalent of c:\tools\apache-tomcat-6.0.14\lib\servlet-api.jar;c:\tools\apache-tomcat-6.0.14\lib\jsp-api.jar
• Setting up Eclipse:
  • Extract the WTP all-in-one package (which contains Eclipse itself) to your designated directory (e.g. C:\Tools\Eclipse)
  • Load Eclipse and point it to your designated workspace location
  • Install M2Eclipse:
    • Go to Help->Software Updates->Find and Install..., select "Search for new features to install" and click Next
    • Click on New Remote Site..., use M2eclipse or whatever for the name and http://m2eclipse.codehaus.org/update/ for the URL
    • Click on Finish and let Eclipse install the M2Eclipse plug-in
  • Set up a web server runtime for Eclipse to host your servlets in:
    • Open Window->Preferences...
    • Under Server select Installed Runtimes and click on Add...
    • Choose (from Apache) the Apache Tomcat v6.0 runtime and click Next
    • Enter the Apache installation directory (e.g. C:\Tools\apache-tomcat-6.0.14) in the appropriate location and click Finish

4. Creating a new web project

First off, you must create the actual project, directory structure etc. To do this:

• Open a command prompt, go to your Eclipse workspace directory
• Decide on your Maven group and artifact IDs; it's worth noting that the artifact ID is also the directory name for the project
• Type in mvn archetype:create -DgroupId=your.group.id -DartifactId=your.artifact.id -DarchetypeArtifactId=maven-archetype-webapp
• You'll notice that a new directory was created under the workspace root
• Edit the Maven project descriptor POM.XML in the newly created directory:
  • Add (after the <url> tag, although I'm not sure the order matters) the following section:

    <profiles>
        <profile>
            <id>servlet</id>
            <activation>
                <activeByDefault>false</activeByDefault>
            </activation>
            <dependencies>
                <dependency>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                    <version>2.5</version>
                    <scope>provided</scope>
                </dependency>
                <dependency>
                    <groupId>javax.servlet.jsp</groupId>
                    <artifactId>jsp-api</artifactId>
                    <version>2.1</version>
                    <scope>provided</scope>
                </dependency>
            </dependencies>
        </profile>
    </profiles>

  • Under <build>, add the following section:

    <plugins>
        <plugin>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>maven-jetty-plugin</artifactId>
        </plugin>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.5</source>
                <target>1.5</target>
            </configuration>
        </plugin>
    </plugins>

• You can now thank my colleague Aviran Mordo for finding out this bit of Voodoo.
• In the command prompt, now enter the project directory
• Type in mvn eclipse:m2eclipse -Dwtpversion=1.5 to create an Eclipse project
• Run Eclipse if it's not already started, then from the package explorer right click anywhere and click on Import...
• Choose General->Existing Projects into Workspace. For root directory pick the workspace directory
• Choose the new project and click on Finish
• At this point you may encounter a "Java compiler level does not match the version of the installed Java project facet" error. If that's the case, just right-click on the error (in the Problems view) and select Quick Fix, which will allow you to change the Java project facet version to 6.0. If this isn't what you want, you probably know enough to resolve the issue on your own...
• You'll need a src/main/java directory as a root source folder (as per the Maven convention). Right-click on the project, select New->Source Folder and type in src/main/java.
• Finally, in order to execute or debug the project on an actual running server, right-click on the new project and select Properties. From there go to the Server tab and select the runtime you created in the previous chapter.

At this point you have a Maven web project with a corresponding Eclipse project ready for editing in your workspace. In practice you will have to do several things to have any meaningful results.

1. Add your own code into the mix, such as a servlet. When adding a new servlet (via right-clicking the project, New->Other and choosing Web->Servlet) your WEB.XML file is automatically updated with the new servlet.
2. Add your own dependencies. Maven handles dependencies quite well; for instance, in order to actually create a servlet you're going to need servlet-api.jar in your classpath; the easiest way to do this is to right-click the project, select Maven->Add Dependency and then simply type in servlet and choose javax.servlet servlet-api.
3. When you wish to run or debug your servlet, right-click on its Java file and select Run As->Run on Server (or Debug, as appropriate). Your applet should be happily up and running.

5. Converting an existing project to Maven

I'm not sure how to go about doing this for web projects, but converting regular projects to use Maven is actually pretty straightforward; move your sources to the appropriate directories according to the Maven conventions, right-click the project in Eclipse and choose Maven->Enable Dependency Management; this will implicitly create a Maven project descriptor for you (POM.XML) and that's pretty much it. From that point on your Eclipse and Maven projects should peacefully coexist, allowing you to leverage both tools for your purposes.