All I can say is: YES! Breakpoint 2007 was amazing. I went with Mickey (my brother) and we had an absolute blast! On the first day of the event we happened to spot an open bag one of the tables that had a newly-bought cellphone package sticking out of it, and were astonished to see that it had a sticker in Hebrew on it. We left a note saying "contact us!" and the anonymous Israeli guy turned out to be no other than Bacter!

 
Left to right: Mickey, Itzik (a.k.a Bacter) and myself

The entire 3-day event was an absolute blast. I got to hang out with really great guys like Havoc (one of the Outline organizers), who was cool enough to give Mickey and me a ride to the Netherlands after the party; Jeenio who also hosted the party prize-giving ceremony; Andy Voss (a.k.a Phoenix/Hornet and MindCandy fame) and a whole bunch of others. I also went to a couple of intereting seminars (one on new optimization strategies for realtime raytracing and one on moving from demos to the gaming industry, both of which you can find here), spoke to quite a few demoscene legens (including Chaos and KB of Farbrausch) and had a really awesome time.

 
People dancing to a live cover of Bubble Bobble? You bet!

Press Play on Tape gave a really good concert in the main hall, and you have to see it to believe it - people were literally dancing to live covers of C64 classics (Bubble Bobble and Commando, to name but a few). They also re-did their classic console-controlled Cannon Fodder, which was even cooler in real life...

One of the Scene.org Awards Amiga demo nominees on the big screen 

For the first time I also got to watch the annual Scene.org Awards ceremony play out, and it was really impressive - the level of crowd involvement was utterly fantastic, and the whole hour-long event was amazingly well-received.

Some of the compos were really quite funny - a "speech coding" compo was held, in which someone had a piece of original code and had to transliterate it using the built-in Windows Vista voice recognition. That feature, apparently, sucks eggs, but the wide range of mistakes it made gave the audience a very good laugh for nearly an hour.

The video compo had a wicked-cool entry by Jakob Bienenhalm called LOL, Internet - see it, spread it, it got a standing goddamn ovation!

fr-041: debris by farbrausch, the winner of the Breakpoint 2007 demo compo

By far the highlight of the party was the PC demo-compo. In this somewhat daunting, several-hour event, no less than 23 demos were shown, and there were some really astounding entries: Andromeda (the oldskool Amiga group) made a huge comeback with Noumenon (2nd place), which was not only cool fanservice but also a really impressive demo; Synesthetics won a very well-deserved 3rd place with the excellent STS-01: Lucy in the Sky with Deities; and Traction and Brainstorm collaborated on a very impressive demo called Fairytale. Of the lesser-appreciated entries I particularly favored Kikumoto by Vovoid.

Farbrausch really rocked the house with fr-041: debris though - this demo had at least three standing ovations while it was still playing, and as much as I loved the other demos... you just have to see this. As a regular demo it's impressive. On the big screen it's bigger than life. And when you take into account that it's only 177KB... it easily becomes the new Second Reality. Only one other time in my life have I felt this exhilirated to be a part of an audience.

Whew. There are more pictures and anecdotes I may share on occasion, but for now I'm spent

I've mentioned System Shock 2 before; to make a long story short, it's one of the finest computer games ever made.

The random Gnome's random Lair published an article on how to run System Shock 2 on modern PCs, plus links to the various (extremely mature) mods that improve the sound, texture and mesh quality in-game; one of these mods also subtly balances the game, making certain items more powerful while normalizing others.

I've been playing System Shock 2 once a year or so since 2003 (four years after it came out!) and I can tell you that it's an experience not easily topped. Go, play!

With my project finally nearing completion, it's nigh time for Microsoft to release yet another update to the .NET Compact Framework 2.0. Service Pack 2 ought to bring it to about "beta 2" level.

Check out these gems:

By far one of the most annoying aspects of the .NET Compact Framework is how heavily it relies on P/Invoke to fill in the gaps. The framework itself is missing huge pieces of functionality, and with the lack of C++/CLI for the Compact Framework a developer is often time left with no choice but to hack around the missing functionality with P/Invoke (or COM interop, if you have the patience to muck about with ATL).

The problem is that, on occasion, a P/Invoke call would result in a MissingMethodException (in fact, if you're really unlucky, the type loader will throw the same exception on loading the method actually making the P/Invoke call). Although a lot of the scenarios have been thoroughly ironed out by now and can be resolved through a Google search or some hacking on the developer's part, there is one scario that is esoteric enough that I couldn't find any references to it on the internet: you can get a MissingMethodException when you are out of memory.

We're working on an extremely large (in mobile proportions) and complex project, involving massive amounts of .NET logic combined with a very large, performance-concious and memory-hungry native codebase. We've had to hack around a lot of missing capabilities in the .NET Compact Framework (as well as some bugs and/or shortcomings in other parts of the OS), and one of our native calls would inconsistently throw a MisingMethodException; having reearched the problem for a day or two I was convinced that the problem was an incorrect function prototype for the exported function and added explicit calling convention declarations. This seemed to have resolved and I was content for a couple of days, until the problem resurfaced.

The exception content itself is next-to-useless, and since the same P/Invoke call would work intermittently I was hoping that enabling the loader log might supply some further information. Alas, all the log would provide was the following line:

Failed to find/load [SomeDll.dll] (even in [\Program Files\Somewhere\])

Not good. I then dug up another article on the subject and proceeded to enable the interop log, which provided some additional information:

JIT ERROR FOR PINVOKE METHOD (Managed -> Native):
[pinvokeimpl][preservesig]
int Workarounds::GetSinkWrapper(out IImageSink , IImageEncoder );
int (I4_VAL) GetSinkWrapper(IImageSink& ** (INTF_REF) , IImageEncoder *(INTF_VAL) );

I was originally interested in the bizarre native signature generation (specifically the IImageSink& ** parameter - where'd the reference come from?), but upon reading some valid log files for working methods I was convinced that it's a dead duck. I then set my attention to the JIT message: what has the JIT to do with a native function? I theorized that the JIT is responsible for the native signature generation for native functions and kept working under that assumption. That narrowed the question down to, "what went wrong with the JIT compiler?"

Eventually it occured to me that this might be yet another manifestation of the memory limitation issue (processes under Windows CE 5.0 are limited to a 32MB address space). The P/Invoke call is made fairly late into the application; I added a dummy function to the library which I called on application initialization. This forced the JIT to take care of the native library before anything was sucking up memory, and the issue was resolved.

The moral of the story? If you get MissingMethodExceptions on your P/Invoke calls, fisrt make sure your DLL is actually deployed; then check the DllImport signature (you can find a lot of useful resources on this FAQ). Finally, make sure you're not out of memory.

I was screwing around with XmlSerializer and manual recompilations (you can enable raw code output from the XmlSerializer) and somehow this happened:

.NET Framework reinstall time, I guess...

Also had some personal issues to take care of, and my family are all out of the country, which means I have a dog to take care of, which means less free time. All of this results in a lower posting frequency, which I intend to remedy in the coming weeks.

In the pipeline (in case you're wondering, this is also a placeholder so I know what's left):

• More posts about .NET Compact Framework (memory and string handling, nonmodal dialog boxes, bizarre method overloads in the BCL, issues with CAB generation)
• Issues with Windows Media Player Mobile
• Some articles I want to link to and discuss
• A couple additional articles on using wikis in the workplace (with additional insight on real-world scalability and usability)

I use Trillian for my IM needs, and a few days ago ICQ abruptly stopped working with an "invalid password" error. Attempting to log in using Icq2Go resulted in the same error; logging in on ICQ's site also indicated that the password is wrong. Oddly enough, the security question remained the same but the system would accept none of my answers; furthermore it wouldn't accept any of my e-mails (current or previous) for resetting my password.

I searched the help system. ICQ's answer?

Please note: If you are unable to get a new password using the password assistance system, then you will have to register a new ICQ number.

So basically, my ICQ number since 1999 or so is useless. I'll update my contact information and try to get in touch with my older ICQ contacts, but if you're looking to contact me please don't use ICQ anymore (I'm not going to register a new number, not much of a point with everyone having an MSN account anyway). I'll be changing passwords on all my accounts in the meantime.

Update (1 July 2007): The newer versions no longer exhibit this problem (tested on beta 12.0.1183.516). Just make sure you chose "None" for the image borders.

Looking back at my post on Windows Live Writer dating back to August, it seems that I either missed a very significant fact about the product or perhaps something has changed in one of the betas. The short story is that Windows Live Writer sucks at handling images; whenever I embed an image it goes on to severely reduce the image quality:

You be the judge: on the left, the original. On the right, the "improved" version

Not only does WLW upscale my image for some unknown reason, it also does this via an extremely low quality scaler. In the case of simple images such as the one above, this will also result in larger file sizes (this example  demonstrates a 400% increase in file size - from under 21KB to a little over 104KB).

This reminds me of the old image pasting problem with Microsoft Outlook, and what kills me is that I can't think of any conceivable reason why anyone would develop this "feature". To add insult to injury, the only feedback link I could find (via this post in the Windows Live Writer blog) is an MSN Groups page that requires registration. The registration process itself asks you about a hundred completely personal questions that Microsoft has absolutely no business asking.

I'll try and get in contact with the WLW team since I do want to keep using the tool, but this particular issue is starting to cost me a lot of time and effort.

Footnote: This is fast becoming a series of posts on the woes of using the .NET Compact Framework. In fact, I've added a subcategory for this exact purpose; you can access the category and its RSS feed from the Categories list on the right-hand side of the screen.

Here's another one of those "bang your head repeatedly against the wall and look for a sharp object" issues: closing a stream derived from HttpWebResponse.GetResponseStream takes an abnormally long time. In fact, the (blocking) call to Stream.Close takes approximately the same amount of time as it would to read to the end of the stream. I've only encountered one other reference to this in a newsgroup post from 2004, which (according to Todd, the original author of the post) was never actually answered.

Seeing as I was trying to use HttpWebRequest to create a minimalistic download manager for one of our applications, I had to come up with a solution: one option was to close the stream asynchronously and just let the download run its course, wasting valuable memory, CPU time and bandwidth; another was to never close the stream and make do with a resource leak. Not content with these solutions I decided to dig into the framework code with the ever-useful Reflector. The first hurdle was locating the assemblies; they're not trivial to find, but if you look hard enough you can find actually useful versions in your Visual Studio 2005 installation folder, under SmartDevices\SDK\CompactFramework\2.0\v2.0\Debugger\BCL. These are regular .NET Assembly PEs so it's trivial to load them up in Reflector.

Some digging into the BCL sources proved that HttpWebRequest does, in fact, read until the end of the stream when closing; this is the relevant code excerpt, from HttpWebRequest.doClose:

if (!this.m_doneReading)
{
      byte[] buffer1 = new byte[0x100];
      while (this.ReadInternal(buffer1, 0, buffer1.Length, true) != 0)
      {
      }
}

It only started to make sense when I did some reading on HTTP (a protocol I'm not deeply familiar with). Apparently, HTTP 1.1 connections are persistent by default, which means the connection is maintained even after the request is completed, and further requests are served from the same connection. Technically, this means that the KeepAlive property of HttpWebRequest is true by default, and a Connection="Keep-alive" header is added to the HTTP request. I can only surmise that, with persistent connections, the response must be read in full in order to allow future requests (if the response was cut off, some concurrency issues may apply). Unfortunately, setting the KeepAlive property to false did not resolve the issue and the connection was maintained until closed.

Since I could find no way to resolve the problem, I decided to hack around it:

stream.GetType().BaseType
    .GetField( "m_doneReading", BindingFlags.Instance | BindingFlags.NonPublic )
    .SetValue( stream, true );

Screwing around with BCL internals is obviously NOT a recommended practise and it's something I tried very hard to avoid, however this problem had to be resolved in one way or another. I traced the code a bit deeper to see if this would actually have the expected results; apparently HttpWebRequest keeps an internal "reference counter" (actually a bitfield, but same principle) on the connection. According to the implementation for HttpWebRequest.connectionRemoveRef, if the reference counter reaches 0 the private m_connection field of the request becomes null, and the connection is released. I started out by testing this:

Before and after - you can see that the m_connection field becomes null

Unfortunately I have no idea how to test whether ot not the connection is actually disposed (conceptually, if KeepAlive is set to false then the connection should be immediately closed); when I find the time I'll try and track the HTTP session using Wireshark, but currently this horrible hack will just have to do.

I've been hacking away quite a bit at an application that contains large managed and unmanaged portions, and a necessarily complex interop layer in between. It appears that interop marshalling behaves somewhat differently between the full and Compact framework. Here's a somewhat laconic list of the issues I've encountered and how to resolve them:

1. You may encounter NotSupportedException on calls to Marshall.SizeOf. Although the documentation does not specify this as a possible exception, experience shows that this is a result of wrong marshalling attributes: for example, although trying to marshal a bool to UnmanagedType.I4 makes sense from a C programmer's perspective, it results in the behavior described above. The article describing this is called "Using the MarshalAsAttribute Attribute," but the version in my locally installed copy of MSDN Library (August 2006) does not contain this information.
2. Annoyingly, the default marshalling, or even an explicit UnmanagedType.Bool, results in corrupt values (probably some minor framework bug). I worked around this by defining the member as int and manually giving it the value 0 or -1.
3. It's not obvious, but you can't use UnmanagedType.LPArray from within structures - it only works on P/Invoke method declaration parameters. The only way to do this is to manually call Marshal.StructureToPtr and do some pointer arithmetic with IntPtr (annoying, but at least it's safe code).
4. The marshaller always frees up memory; although this makes a lot of sense from the .NET perspective, it probably means that you'll have to code in some sort of deep copying mechanism in your native code if you want any of that information in your state. This also has performance repercussions you should consider when architecting your interop layer.

I'll post updates to this as I come across more issues.