NotZed's (Software?) Diary

I've been looking into versioning data in databases, and just wrote an article about it. For some reason I started thinking about writing a blog engine/cms/online documentation system and it's been consuming me for the last week or so, and along the way I got a bit side-tracked and looked into the revision thing.

I wont even be using SQL - but that's the easiest way to explain some of the ideas, and is also a bit quicker to test them.

I don't know why I'm so absorbed by the CMS thing - I don't really like web programming. Well if I get anywhere i'll let you know.

Been a bit quiet around here. For a few reasons - i've been busy outside of work with some changes around the house, and i've been busy with stuff inside work which has just been sucking my brain of skills and any will to learn. Yes, i'm still working on a windows box writing .NET code. It's just mostly unpleasant, and isn't at all invigorating either.

Anyway I started looking into AROS as something else to engage my time. Not sure where i'm going there, but I've written an AVL tree implementation for it so far. It got me into other data types and other ideas.

I wrote some analysis of list merge sorting implementations a couple of weeks ago for instance. Partially to provide a nice base implementation for an AROS List sort routine, and also to compare it to GList - both algorithmically and implementation wise.

I looked into literate programming as well. I couldn't get around all the tex in cweb, so I grabbed texiweb from the libavl distribution and played with that for a while. I'll upload something when it's in a bit better shape. I was investigating hash tables this time. Just a simple implementation and one that provides incremental rehashing. I ran into a bit of a roadblock trying to get accurate timing of short events - e.g. peak/average/miniumum times for hash inserts (e.g. incremental rehashing should be much better than bulk rehashing). gettimeofday() is a bit rough, and the rtdsc instruction didn't seem to give very reliable results either.

I joined the Free Software Foundation. I've been considering it for years and just did it at last.

I investigated memory allocation - this was partially to play with ideas in AROS. AROS has a simple first-fit memory allocator which has the nice property of having zero bytes of overhead for any allocation (not even the size), but it scales linearly with the number of free blocks. And all processes share the same address space - and same memory allocator. I came up with a couple of different allocators, one that used the AVL tree to provide a best-fit implementation, and boundary tags to speed up coalescing, and borrowed some ideas from dlmalloc() to speed up small allocations. Then I played with another that used super-blocks aligned on memory addresses which don't require any overhead for size for small blocks, etc. I was surprised to later find something like it is in glib's slice allocator already. I've put the ideas I had on the backburner for now, but I guess they may resurface.

I keep reading about the CELL and SPU's, but I can't think of anything to do on my PS3 - something that isn't going to require weeks of time anyway. Maybe one day, hopefully before too long - it just looks too interesting to not try something.

Well, I set a new personal best time today. Help took a full 3 minutes to show up after I hit F1. And I've only got fickle studio, firefox and a console running irc-ii. This useless system is really getting to me. I can't understand how people can use this rubbish all the time, and even like it, promote it, argue how great it is. I must repeat, vs2005 is simply the worst piece of software i've ever had the misfortune of having to use in my 20+ years of using computers (oh shit - it really is that long too).

And run that on top of XP and all your nightmares become reality.

Another widget done for work, together with one of my workmates. A 'location selector'. It lets you select locations from around the world. Database backend, various helpers, and re-usable for other tasks - e.g. embedded in a route planner.

Basic selector, with a pre-selected location.

With the search 'window' activated. For Management-Effect, it fades in and zooms out from the Search button when you click on it.

And finally, one of the other uses of the location selector within our application, with some added functionality written by Shane.

The world map coastal outline is based on the "CIA World DataBank", with some simple processing applied to cull it down to around 14K line segments. Although we recently added a texture map satellite picture which negates the need for the outline at all, the outline adds a bit more contrast - particularly when you zoom in.

I also fixed some picture links - too lazy to check last time.

Not much to report - coding WPF is basically pretty boring, its like a bloated over-complicated toolkit. And even then it doesn't come with basic widgets you'd expect.

Anyway, a couple of days ago I wrote a colour requester. It's got some problems - the HSV colourspace is a bit wrong, but it's good enough, and probably overkill for people used to the awful excel colour selector.

Ahh dear oh dear. Today I thought i'd look into multi-threading in .NET, and specifically, WPF. I've been busy working on writing a data-base backed implementation of our application, and I've started to hit some performance problems - not major - that could benefit from threading. 6 long hard years on Evolution taught me many things, but one was that multi-threading is an absolute must in any modern GUI application, and is a much simpler solution than synchronous cooperative multi-tasking using idle handlers and recursive mainloop callbacks. By a long way.

So, it was with some dismay that the main 'threading model' documentation that I found regarding WPF in the help files had this to say about threading application, in its opening paragraph:

When we created Windows Presentation Foundation, we worked very hard to save developers from the difficulties of threading. As a result, the vast majority of Windows Presentation Foundation programmers will never have to write an interface that uses more than one thread. This is great since multi threaded programs are complex and difficult to debug. They should be avoided when single threaded solutions exist.

Right. Idiots.

Fortunately the actual mechanisms available aren't all that bad. Something like the ability to invoke functions on a specific GMainLoop, so you can easily call things 'in the gui thread', etc. And you can have a thread-per-window if you wish (although this application uses a single window, so this isn't so useful).

Well, obviously the guys writing the documentation had little idea about threading - otherwise they wouldn't have bagged it so much. But i'll leave it up to them to summarise their conclusions:

If nothing else this paper probably taught you that multi threaded Windows Presentation Foundation applications are complicated. Avoid them when you can! If you program with Windows Presentation Foundation long enough you’ll undoubtedly run into situations where multiple threads are needed to keep the quality of your application high. Hopefully, the explanations and examples in this paper will help you build the clear, efficient solutions you’re looking for.

But then again, they're probably right. .NET is really just a re-vamped 'visual basic' and the new coding-fad-of-the-week and not really a serious enterprise-quality framework.

Oh, I feel so dirty. I spotted a show-stopper bug for vista/wpf last week - a massive memory leak in the multi-column list-view widget. Even letting my m$-fanboi workmate get all of the kudos for 'finding the bug' has left me emotionally scarred!

Another day wasted working away infront of a couple of big screens. Nothing like a positive attitude, but I guess negativity is permeating the office lately. Maybe its all the hayfever in the air - i've got it this year, and it normally just doesn't affect me like this.

At home I worked a bit on the MIME parser last night. So it took me a while, but I managed to get most of the state engine working how I wanted it too. Now to do more testing.

I was using a technique you might call XLP - extremely lazy programming, or code by intuition (CBI?). Basically you have a high-level idea of how you want something to work, and can't be bothered working out the fiddly details. So you code it up, and then just debug it and poke it until it does what you want, maybe cleaning it up a little later on. Sometimes your intuition (well, experience isn't it) is right, and you save all the hassles and time of having to consciously design something, and still end up with an excellent design. Other times you end up with a mess which never gets close to working.

Fortunately I had a good known implementation in CamelMimeParser to work from, although I came up with something less complex (if due only to fewer lines of code), with a different state sequence, so didn't just copy it. I keep looking back at code i've done in the past and wonder how I ever did it - my mind seems a little dulled these days. I still need to work on some features like scanning Berkeley Mailbox files, and handling all of the corrupt/truncated cases, but the foundation is there - and there's no rush.

*SNEEZE*

Hmm, another fine spring day - pity the weekend wasn't like this. Its such a shame to have to work in weather like this (or really, to have to work at all). Ho hum.

Well over the weekend I sat on my fat arse too much, drank too much, and noticed how grey my hair is getting. I've been being poked lately about doing something again on Camel, but although I can't really bring myself to start on it again, it has got me thinking a little bit about mail again. However, I did play around with a MIME parser implementation in c-hash on Saturday. Basically the same algorithms as the parser in Camel, but I think I can simplify some of the logic a bit, now I'm older and more experienced (or lazier). I guess I'll see how it goes, I couldn't get my head out of first gear (in the fog of a hangover?), so I didn't get too far. Needing to keep things in 'raw' format, i.e. using byte[] rather than string's, is a bit of a pain, although I guess it was fun to start building new interfaces and so on from scratch, as things came together slowly.

I also thought about doing something just in straight C again. But I don't really want to work on Camel. Maybe I could take bits of it, re-work it a lot. Cut down the interfaces plenty, strip out the glib stuff, play with some new tech like SCons ... Or maybe I need a more social hobby!

Hmm, September already - the year is vanishing too quickly again, and I don't seem to have much to show for it. I seem to be relatively busy at work, but it never seems to get anywhere! We had some spring-y weather last week, but the last couple of days are right back to the heart of winter - my shoes got soaked riding home last night. Pretty awful end to a shitty day at work (no reason, just felt like crap).

I started looking into databases this week; i'm not sure I really want to go that route, but we have some data which probably fits a database better than doing everything manually using custom objects - and certainly a world better than the randomly-formatted EXCEL files that are being used for the data (!!!!) ... but unless the rest of the app goes toward a database, at least partially, it doesn't make much sense. But i'm short on other ideas right now, and its about time i had a bit of a refresher, I haven't played with rdms's for 10 odd years. And we need a data-loader to try to extract some computer-usable data from those excel files anyway.

I started playing with access at first, but that quickly became an exercise in total frustration, and we're only talking half a dozen relatively simple tables (funny, last time i played with it years ago it didn't seem such a mess). Besides, my installation seems to be partially screwed, so the drivers aren't working properly, or don't work in 64-bit processes, that I could tell. So I installed MySql and started playing with that. Ahh a much more comfortable environment to play with - even if the install wizard didn't really work. Nothing wrong with the command line though - although the console client doesn't work inside emacs' shell or cygwin's xterm, which sucks, the windows console is awful. The mysql gui tools are ok at least, although a little buggy, and the db isn't terribly verbose with it's error messages, but the documentation is pretty good, and there's always google.

Then once I had some working tables and data loaded, I came to the 'ADO.NET' stuff to bind to the db so I could display more bloody lists and tables in some UI. Ugh, clunky stuff. Well, i guess nothing was meant to be easy, but all these 'frameworks' don't seem to add much in the way of ease, and it's more shitty interfaces to learn about. Just as well I don't have to use it in anger i guess.

Still, my main frustration with windows is the shitty interface! Now excel has decided to hide half my menu's all the time (I suppose since i've been using it more lately), which gets on my nerves, everything locks up while its busy, or crashes when you'd rather it didn't, vs is awful as ever, and having everything 'raise' when you click on it drives me right up the wall. Really unproductive environment to work in, if like any normal user since about 1990, you expect to access multiple applications simultaneously.

Hmm, BEER DAY at last, although there's still the whole day to suffer through before I get there. Well, unless I shirk the afternoon stretch and spend it down the pub instead - although its a cold miserable day here, so maybe I should save that up for summer.

Well, not a whole lot happening in WPF land. Its got its bugs, its got its problems, but it mostly just does the job. Fortunately we've got machines that are fast enough that it just feels like my old celery 500 w/ 128MB that I used to develop evolution on in the first year - i.e. adequate but not lightening fast.

I occasionally run into this specific bug where you cannot bind to elements in XAML which cross namespaces e.g.

  <bob:Block x:Name="foo"/>
  <joe:Bland Foo="{Binding ElementName=foo, Path=Foo}"/>

I'm sure i'm not doing it all 'properly' - whatever that is - because everything is so poorly documented. I have a mix of ways of solving problems, some are driven by indirect data-type based binding, some are more direct, whatever works for the moment. Some are based on direct rendering, others use structured building blocks.

But yeah, apart from some relatively interesting things in the product, which I can't talk about, the UI stuff is mostly lots of lists, trees, and forms. Basic, boring, bread and butter stuff ...

The windoze 'window manager' is giving me the shits though. 2x24" screens and it still seems to make it hard to use applications properly. Things keep popping up when I don't want them to, focus jumps all over the place, and alt-tabbing around is a pita. Not to mention things freezing while they're busy. Feels very archaic - I miss my amiwm big-time.

I've spent most of the last couple of weeks doing some pretty boring frontend->backend interfacing code. It took so long because I didn't really understand the code, the comments/documentation wasn't very useful, and the guy who wrote it was off sick. Well, I got something working anyway.

Today I worked on something a bit more 'interesting', although it's still pretty simple, just a curve editor. We might need an input widget that lets you define time-variant quantities, which may also be cyclic, so I made a simple little curve editor and a non-ui class to drive it. The pics should suffice to explain it I think:

The blue circles are the calculated values at the specific months, more there just to test the code works than anything. Although the UI part of the code is a pretty messy hack, at least it's a pretty simple messy hack. I guess i'm getting lazy in my old days - these machines are so stupidly fast (even handicapped so much by winblows), that good enough will do. Extrapolate probably isn't particularly useful, but I added it for completeness anyway.

I got the basic idea from the IPO curves editor in Blender, although it is a bit simpler, since we don't need smooth curves (for now, anyway).

I've mostly been too busy or distracted to do much in the way of 'interesting' work lately. Mostly just hard slog stuff, lots of data structures, and trying to present them in usable ways. The boring grind. Dealing with the shiteness that is visual studio - it crashed about 5 times a day this week. Definitely the single buggiest piece of software i've ever regularly used in nearly 20 years of using computers regularly.

Despite many claims to the contrary - which I never believed anyway - c-hash, .NET, and so forth is no more productive than any other environment i've used. Arguments could be made to suggest infact that all the overheads and buggy development enviroment reduce productivity (my old 1.5Ghz laptop feels a lot snappier in general than this 2x3Ghz box running windows). I have a thought on why some people claim that such modern enviroments are 'orders of magnitude' more productive than more time-honoured systems. They basically can't code for shit to start with, so the time they'd have spent fixing their own bugs in utility code is saved by using the framework/toolkit/whatever's utility code - but that is a long way from actually getting any work done.

Yeah so I ran into a problem with the method suggested in the last post to colour alternate lines in the listview. When i moved from using a GridViewRowPresenter directly, to using a ListView with a GridView 'View', suddenly selection fucked up. i.e. the text would go white, but the background stayed the background set below. Ho hum, well I guess more work is required - I didn't really need it anyway so I just disabled it for now.

Well, i've been pretty busy lately - which in the wpf world means lots of swearing. My poor workmates.

I've found some shittiness/bugginess in some databinding stuff today. If you try to define a DataTemplate resource in a ContentPresenter inside another resource defined in a given object - well, it doesn't work. Either it - perhaps - tries to put it into the outer resource dictionary (sounds related to/the same as a known bug), or it just doesn't seem to like a ContentPresenter as a DataTemplate (for a GridViewColumn?). Ho hum.

I had to write my own tree control a few weeks ago because I needed a multi-column tree view. It's kinda messy to use, has a few bugs, but it somehow seems to work most of the time. Just a 'simple' wrapper which takes a tree structure and maps it to a ListView. So that way you can just add whatever columns you want using a GridView ListView.View. One of the columns is setup to display the tree expander/offset 'title', etc. The example of a multi-columned tree from the ATC team didn't really seem that useful to me at the time - although maybe I just didn't understand it enough. The main difference is that rows and structure is explicitly defined using special tree nodes - no indirect content-presenter data binding going on. But since I needed such structure anyway, separate from the data, this was no extra pain, and let me easily encapsulate the logic anyway.

I also wanted to colour alternate rows in alternate colours - and the examples I could find on the net seemed pretty confusing, so I discovered a simpler - (if perhaps less efficient?) - way to do it.

    {
        ...
        listview.ItemContainerGenerator.StatusChanged +=
	    new EventHandler(ItemContainerGenerator_StatusChanged);
        ...
    }

    void ItemContainerGenerator_StatusChanged(object sender, EventArgs e)
    {
        if (ItemContainerGenerator.Status ==
	    System.Windows.Controls.Primitives.GeneratorStatus.ContainersGenerated) {
            for (int i = 0; i < Items.Count; i++) {
                ListViewItem lvi = ItemContainerGenerator.ContainerFromIndex(i)
		    as ListViewItem;

                if (lvi != null) {
                    if ((i & 1) == 0)
                        lvi.Background = Brushes.White;
                    else
                        lvi.Background = Brushes.AliceBlue;
                }
            }
        }
    }

It iterates all items because there was no simply-obvious way to find out what items might be visible right now. There probably is a way to find that information, and there are probably dozens of other ways to do the same thing, but this works for small lists just fine.

My workmate is shitty that his blog wont stick around on google's search result for his name - well I guess thats what you get for being a total M$ groupie!! Particularly since this page jumped to the top of the search result for mine in a matter of days ...

Hmm, for some inane reason this stupid pile of shit, so-called "operating system" has decided it wont open ftp folders in explorer anymore, only in internet explorer. Groan.

Well, I stand corrected on the missing zip ... it's there, just in an unintuitive place, and I didn't have the time yesterday to search for it (i'm used to a shell which searches for me).

I got a strange mail about yesterday's entry though:

"Megabytes of bloat" - I thought it was only 5Meg in size.

Last time I coded against Avalon, it didn't even have a tree/outline control
- so lacking a spinbutton is minor.

And Windows has come with a zip utility for the last 5 years. I just flicked
to my Parallels XP VM and zipped a file to prove it - and my XP VM is a
virgin XP with security patches and Firefox only.

I think you need comments enabled on your blog so when you say inaccurate
things, people can correct you.

Isn't 5 a plural number? Sure is here. Avalon != WPF - WPF is meant to be near release-ready, and this isn't a blog either! But the comment is now here anyway.

Oh well, back to the hacking. Less mundane things to worry about than individual widgets - more complex design (UI unfortunately) problems too.

I already have a 'blog' on planet.gnome.org, however i'm currently working with decidedly non-GNOME technology - 'Windows Presentation Foundation', and I feel the need - being the free software kinda guy I am - to talk about some of it in more detail.

Well to kick it off, i've written a little 'spinbutton' 'control' for WPF - apparently known as a 'UpDownNumericControl' in the winblow$ world. Because WPF, despite its megabytes of bloat, apparently didn't think it needed such a thing. Don't worry, more rants about windoze to come, but for now I should get back to work.

One thing that seems pretty dissapointing with WPF is it's lack of 'standard images', or even 'standard controls', things like up/down arrow buttons, and the like. Or they're just impossible to find in that shitty documentation it comes with. So you end up having to do a lot of stuff 'by hand'. And what's even worse, it doesn't have a very simple mechanism for automatic scaling either. Sure you can embed things inside of a ViewBox, but that seems terrible overkill. So you'll note that the buttons are hard-coded in size. I use a 3 row grid so at least they'll space out to fit the available space, should for example, the font size increase, so the buttons don't fill the vertical space anymore.

Umm, dunno what else - the source code is available here, under a GNU GPL licence. This stupid windows, so-called 'desktop' doesn't come with many normal utilities - like zip, so i've stored it as a compressed tar file (thanks cygwin).

Copyright 2006 Michael Zucchi