Friday, December 01, 2006

My Dream File Manager

What follows is a collection of points for my dream file manager. It was collected from notes I have been taking over the past two years so I am sorry if it doesn't flow that well together. So grab a beer and get ready for some chuckles as you read the key points for what would makes my dream file manager.

1) Hide the details from a user on how to access their files

Files are stored on volumes. Users should be able to just point and click and have access to these volumes. "Volumes" is a generic term referring to a number of things such as cameras, ipods, nfs shares, ftp, fish, zip, and more. Hardware volumes that are removable should just appear in the file manager with no effort on the part of the user by default. Software volumes must be discoverable (through a network explorer) and if possible accessible through an assistant such as in mockup on the right.

Currently in Konqueror you need to be taught about the useful ioslaves by someone else before you discover them yourself. This would not have been a problem if there was the most basic assistant to help users discover what ioslaves were available.

To accomplish this first task the file manager should have tight integration with solid and HAL to get all new hardware volumes and possibly provide an assistant to make accessing software volumes discoverable. For network browsing the mdnsResponder is also available for discovering local shares.

2) View & Edit meta information

Files contain meta information that is for the most part only relevant for a file manager. This can be broken down into two categories, the first being for volumes and the second being for files.

Volumes:
  • Volume Name
  • Free space
  • Size
  • Health status
  • etc

Files:
  • Size
  • Permissions
  • Ownership
  • What opens this file?
  • Icon
  • etc

The above two lists are very small and will change depending upon the file system, hardware, OS, etc. A modular approach is best.

Beyond the basic file system information file manager often have a plugin system for extended attributes editing such as the comment, title, and keywords on a JPEG or the rating of a mp3. This should not be part of the core file manager, but through an addon script system. The extended attributes should not provide means to edit metadata that would not normally be accessible to a type specific manager. For example in iTunes they don't let you edit the play count even though this is just another field in the id3 metadata. Extended attributes also by default shouldn't let you edit every possible metadata (image exif has quite a lot for example), but only the most common and useful.

3) Export and Import

A file manager can be expected to move files in and out of the current environment. Two common cases includes audio cd import and data DVD burning. This builds upon #1, but typically has a intermediate step that needs to be performed before a file can actually be copied/moved. In #1 you could see a CD and could copy the CDA files off the disk, but now you could see mp3's and when you copy them it will transparently encode the CDA files. Also creating a virtual folder that would then be burned to a CD or DVD. The export and import will typically be plugins that are well integrated with the interface (and not klugy addon hacks). Check out the Finder's Burn folders for an example. Another example is an iPod who when mounted is just a bunch of garbled files, but with a import layer the user could simply see a list of files with the correct file names and could drop new files in and copy old files off. Finally Trash is a nice little virtual folder that has the extra feature of emptying or restoring.

To accomplish the above today you can use dbus with k3b for the ripping and burning.

4) Basic information, but not a full blown viewer

The simplest example of this is showing a image thumbnail rather then the png icon. Other examples of this include showing a specific application icon rather then a generic binary icon. Depending upon the interface if there is a preview zone it can include extra basic information such a image dimensions and for audio basic playback.

5) Full search capability

File managers much have full search abilities, but by default only present a single line edit in the upper right of the interface. Through an assistant the user should be able to make virtual folders that contain rich search results. Some common useful virtual folders should be present on the system such as recently launched applications or files created in the last week.

Today Konqueror can integrate with strigi for search results, but Konqueror can even present the slocate ioslave until strigi is up and working.

6) Manual Grouping

To the user manual grouping is very similar to search folders. They are virtual folders that users can drop files into. The files are not moved there, but only sym linked. I hate to mention it because of the one and only one way that browsers have implemented it, but a common way that this is seen today in application is a bookmark system. To help you to see it differently think about how in iTunes you can have playlists that you build up yourself by dragging files from your library. You can have smart playlists (all songs I rank 5) and plain old playlists which you manually manage (the Norwegian audio lessons I am currently working on). A even simpler implementation of this is the OS X Finder sidebar which lets you drop folders and files on the sidebar for quick access later.

7) External interface to access the "stuff" in the manager

The first aspect of this is to be able to communicate with the file manager to perform operations such as a search, viewing recent folders, icons for files, etc. dbus is currently the solution for this. The second part is once you get the results being able to access the data which KIO/QFile do fine.

An basic example of this is a screensaver that queries the file manager for a list of the fifty most recently created images.

8) Extensible

A file manager should be extendable. Because of the wide use cases that this application is going to be used it needs to be modifiable. This can be as simple as providing a way for people to add right click actions to replacing the search system entirely.

Almost all of the above requirements for a file manager would benefit from plugin type systems and in fact many of them could be implemented in with them.

Incorporating the ability to write portions of the application in PyQt, kjs, QScript, or other scripting language should be thought about from day one.

Stressing this even more it is often time the community who when provided access to an extensible scriptable system will come up with features and capabilities that were not on your roadmap. A perfect example of this is FireFox that has countless addons. The features provided by the community will satisfy user itches without forcing the core application to become bloated while at the same time increasing the value of the application. Another benefit is that the interface can start out simple and only the most successful features can be added at a later date helping to prevent bloat.

9) Addon/intigrated tools/actions that build upon the above features

A file manager should provide by default a set of basic set of core features that enhance file management tasks.
  • History of everywhere that you have browsed, i.e. back/forward buttons.
  • compress folders & expand compressed files.
  • Wildcard / RegExp selecting.
  • .hidden file support that hides files/directories in the file system.

Some extra addon module/scripts can be provided for more advanced users
  • Drop Stack (drop your file(s) in this space, change directories and then drag it to its final destination)
  • console at the current location for advanced users.
  • renaming assistant.
  • Some sort of built in automatic backup system.
  • Ability to selectively mark folders for sharing.

10) Interface


Finally, after all that we get to the interface. The simpler the better, it should have tabs, but not on by default, there should be a search field, but only a single line edit in the top right. There Should be a single sidebar on the left that contains all mounted volumes (virtual or not), bookmarks, and virtual folders. Somewhere always present it should list the free space for volumes. As for the files there is merits to having only an icon view and a details view, but some would argue that a tree view and column view do very well. It would be fascinating to see several completely different approaches that focus on either a list, tree, or both. I will leave that up to a usability study, but no matter what the result it should be fully accessible from the keyboard. I personally love the column view in the finder for its quick keyboard access and final preview window. Also the first nine requirements generate a basic list of features that the interface should have, but they are not that interesting to debate.

Sunday, October 29, 2006

Netflix Prize Contest

I was very excited when the Netflix contest was first announced. I have been toying with that problem for a while, but I never had enough data to do anything really with so it was always a minor project of mine. As I watched people get interested in the contest many of them were spending the first week figuring out how to manage all the data that was given to them. The worst part was that after a week of doing this it wasn't very fun and they would stop, but yet they hadn't even worked on the real problem yet! I have been staying up late this week, but having a blast and created a little present for you all, the NetflixRecommenderFramework. It lets you very quickly start hacking on your algorithm ideas without having to bootstrap everything else first. It even includes autotests. Maybe you can win the million dollars.

On the Qt side of things one of the included tools is a small GUI application that can be used to explore the database. At over 100 million rows I am happy to say that Qt QTableView can handle it just fine and there is no startup delay. This is a followup to last years blog entry 66 million rows in QTreeView where I got it up to 66 million rows without any work. This time around I have actual data behind each row and column and not just a model that generates the data based upon the row and column. The bugs found before were still there if I set the height of the rows too large at which point their total height was greater then max int, but I have fixed those issues and they will be in the next Qt release. Anyone know if Gtk can do this?



Update: fixed link to project and source.

Tuesday, October 10, 2006

Dynamic Models

When creating a custom tree model in Qt one common request is to lazily populate the tree. Starting in 4.2 there is a nice way to do this using QAbstractitemModel's fetchMore(). In your model overload hasChildren(), canFetchMore() and fetchMore(). hasChildren() should return true for those index's that have children even if the rowCount() is currently 0. The default implementation will return true if rowCount() is greater then 0. QTreeView uses hasChildren() and not rowCount() to determine if it should draw the little triangle/plus symbol to expand that index. When the user clicks to expand the index QTreeView will call "if (canFetchMore(parent)) fetchMore(parent);" Taking advantage of this inside the model's fetchMore() implementation, the node can now be populated.

I have created a small example FtpModel which wraps QFtp. It will only list the directories that you have opened. It doesn't implement renaming, drag/drop, or any convenience for uploading/downloading files, but those can be added without much work if you want to take this and use it. Feel free to download the source and check out the model code. It is released under the BSD license.

On the right is a screenshot of the example application with a dirmodel on the left showing my home dir and on the right the ftp model connected back into my local box and a open directory.

Monday, September 25, 2006

Drill Down View

When dealing with data that is in a hierarchy, one way to present the data is inside of a drill down view. Some examples of what a drill down view are include the recent Start menu redesign put out by SuSe, the TiVo menus, and the iPod interface. It has the fantastic features of taking up almost no space and being very intuative. For more in depth discussion on this view check out One-Window Drilldown on the UI Patters website. Creating this in Qt 4 was very simple, here is a animated gif screengrab of the results.



The implimention was done by subclassing QListView and its moveCursor function. Before changing the root index I would store the current view image. When the current index changes I start the animation. One feature found in drill down views is animations. In Qt 4.2 there is a new class called QTimeLine. This class is perfect for helping you add animations to widgets such as the above one. Below you will find the code that you can use to have a drill down view in your application. The only code I didn't show is a item delegate that I wrote to draw the triangles on the right hand side, but i'll leave that part as an exercise for the reader.


#include "QtGui/QtGui"

class DrillDownView : public QListView {
Q_OBJECT

public:
DrillDownView(QWidget *parent = 0) : QListView(parent) {
connect(&animation, SIGNAL(frameChanged(int)), this, SLOT(slide(int)));
animation.setDuration(200);
};
QModelIndex moveCursor(CursorAction cursorAction, Qt::KeyboardModifiers modifiers);

public slots:
void currentChanged( const QModelIndex ¤t, const QModelIndex &previous );
void slide(int x);

protected:
void paintEvent(QPaintEvent * event);

private:
QTimeLine animation;
QPixmap oldView;
QPixmap newView;
int lastPosition;
};

void DrillDownView::paintEvent(QPaintEvent *event) {
if (animation.state() == QTimeLine::Running) {
QPainter painter(viewport());
if (animation.direction() == QTimeLine::Backward) {
painter.drawPixmap(-animation.currentFrame(), 0, newView);
painter.drawPixmap(-animation.currentFrame() + animation.endFrame(), 0, oldView);
} else {
painter.drawPixmap(-animation.currentFrame(), 0, oldView);
painter.drawPixmap(-animation.currentFrame() + animation.endFrame(), 0, newView);
}
} else {
QListView::paintEvent(event);
}
}

void DrillDownView::slide(int x){
viewport()->scroll(lastPosition - x, 0);
lastPosition = x;
}

QModelIndex DrillDownView::moveCursor(CursorAction cursorAction, Qt::KeyboardModifiers modifiers)
{
if (animation.state() == QTimeLine::Running)
return QListView::moveCursor(cursorAction, modifiers);

QModelIndex current = currentIndex();
if (cursorAction == MoveLeft && current.parent().isValid()) {
oldView = QPixmap::grabWidget(viewport());
return current.parent();
}

if (cursorAction == MoveRight && model() && model()->hasChildren(current)) {
oldView = QPixmap::grabWidget(viewport());
return model()->index(0, 0, current);
}

return QListView::moveCursor(cursorAction, modifiers);
}

void DrillDownView::currentChanged(const QModelIndex ¤t, const QModelIndex &previous) {
if ((current.isValid() && previous.isValid())
&& (current.parent() == previous || previous.parent() == current)) {
setUpdatesEnabled(false);
setRootIndex(currentIndex().parent());
setCurrentIndex(currentIndex());
executeDelayedItemsLayout();
// Force the hiding/showing of scrollbars
setVerticalScrollBarPolicy(verticalScrollBarPolicy());
newView = QPixmap::grabWidget(viewport());
setUpdatesEnabled(true);

int length = qMax(oldView.width(), newView.width());
lastPosition = (previous.parent() == current) ? length : 0;
animation.setFrameRange(0, length);
animation.stop();
animation.setDirection(previous.parent() == current ? QTimeLine::Backward : QTimeLine::Forward);
animation.start();
} else {
QListView::currentChanged(current, previous);
}
}

int main(int argc, char *argv[])
{
QApplication app(argc, argv);
DrillDownView view;
view.setWindowTitle("Use arrow keys");
QDirModel dirmodel;
view.setModel(&dirmodel);
view.show();
return app.exec();
}

#include "main.moc"

Monday, August 14, 2006

Transformers


After years of collecting I have finally put together a little Transformers generation one site, ToyBin.org. The site is a complete gallery of every toy, its modes and accessories in 1984, 1985, and 1986. This includes such old favorites as Optimus Prime, Megatron, Soundwave, and Devastator. It also has a simple search system which lets you do things like search for Green Truck and see all of the photos that are that. On more then one occation I have had people tell me the color of a transformer they remember having as a kid, but they remember anything more specific. There is a rss feed for the Transformer of the day which has been quite a lot of fun being surprised by old toys you forgot about such as Astrotrain or Shockwave.

Saturday, May 27, 2006

Network

Like a lot of people these days I own more then one computer. Back in the mid ninities along with many others I would list my computers down to which hard drive was in what box, but that type of information changes quite often and provides almost nothing of value to the reader. Instead I will highlight a few items that I feel are important and why.

KVM Switch

Even back in 1997 I knew I needed something similar to what this product does. Once I started to have more then one computer I knew that having three monitors, three keyboards, and three mice wouldn't work out. It would be too expensive and take up way too much room. After doing a little research I purchased an OmniCube which is made by Belkin and have used it ever since. It allows me to have one monitor, one keyboard and one mouse attached to up to four computers simultaneously. To switch between computers I hit "Scroll Lock" twice in a row and then the keys 1 through 4. I would recommend a kvm to anyone that is looking to own multiple computers while keeping costs and space down.

Kinesis Keyboard

I tried out many different and exotic keyboards, but in the end I settled on a Kinesis Classic keyboard. This keyboard which has the most frequent keys such as enter, space, delete etc in the middle of the keyboard reduce the amount of work on the weakest finger, the pinkie. Due to its curved design my wrists don't bend reducing any possibility of injury. I also found that after a week I was able to type faster with this keyboard.

Herman Miller Aeron Chair

I code up to eight hours a day, seven days a week, and spend another twenty hours a week working on other projects in front of my computer. If my butt is to be plastered for that long in something it should be comfortable and wont break. What can I say? This chair isn't the best just because of hype. When I first got the Aeron chair everyone around the office came and took a quick spin in it. Sitting in the chair for a minute they would all say: "Ok this is comfortable, but is it really worth that much?" One day After working all day I attended a meeting and noticed just how good my chair was and how bad all of the other chairs were. You don't know how good something is until you have to use something much worse.

Quiet Computing

As computers get faster and hotter the fans get louder. When upgrading my Tivo's hard drive it was recommended to get one of the ultra quiet Seagate hard drives. When I installed it I found to my suprise that even though it was faster then the old Tivo drive it was dramatically quieter, and the old drive was quiet by any standard. Impressed and hearing of other quiet computing projects I started looking them up online. One of the items I discovered was the Antec Sonata case. With hard drive noise dampners, a very large (thus spinning slowly) rear fan that was temperature controlled and a power supply fan that was self controlled it was the prefect case for reducing noise. When I needed a new hard drive I got another Seagate and also picked up the Antec Sonata case. When my other computer is on I can't hear anything from the Sanota computer over the noise from the old computer (before it was upgraded). With a little research a few dollars more you can pick up very quiet computer components. You do not need to have a jet engine under your desk.

The router appliance

One of the big reasons why I got into Linux back in 1997 was that I was able to take an extra computer and with Linux's built in capability turn the computer into a router. Then all my computers could be online at once. Back then the other option was to either use wingate (a primitive proxy) or some overly expensive windows program. I learned a lot about Linux just by setting up all the different servers such as iptables, dhcp, and samba. But these days rather then having to run a whole separate computer running all the time and needing to know a lot about the inside of Linux anyone can walk into a computer store and pick up a low power, noiseless box that frankly does more then I had ever set up manually, all for less then $100. And the best part? It still runs Linux.

Monday, April 17, 2006

Mini's

Over on slashdot there is a review of the Nokia 770 Internet Tablet. Maybe Nokia has big plans, maybe they are going to re-use some of the software (cough only the browser cough) for their phones, who knows, but I feel like a time warp back five years with the Zaurus. You could almost read slashdot and swap the name of the product and it would be the same. People complaining about the software, but saying "it doesn't matter because the community will quickly make it better/fill X nitch/create whole new solutions/create new better rom/fix X problems. It is really sad too because the hardware itself isn't even that great from what people are saying. So in a year or two this device will be dead. Maybe a whole new better device will come out or more likely three more revisions will limp out, only time will tell.

In a similar vain I saw a review of PyQt and BlackAdder and thought I would check it out because I am interested in what they would have to say about BlackAdder. I have never used it myself, just curious about the product I have heard about from time to time. Suffice it to say the reviewer found it less then stellar, with this choice quote summing it up:

TheKompany is going to have to bend over backwards to show they've mended their ways before I would ever consider buying anything from them again.

Not to end on such a unhappy note I recently found a LISP and Qt4 integration attempt. It is pretty cool and I got it working on my box without too much trouble. Good for quickly hacking up some test apps.

Sunday, March 12, 2006

Documentation on the web

There is something very interesting about web applications that desktop applications lack. That is integrated help. When you go to amazons page for Pragmatic Programmer (good book btw) notice on the right hand side in the blue box there is a link to learn more about how A9.com users save 1.57%. Or if you go to a random product on OldNavy.com you will notice that there is a link for a size chart. Head over to Slashdot and make a comment or most every online blog and you will often find a list of html tags you are allowed to insert, hints, and tricks on how to make your blog entry 'rich'. Sites like pbwiki.com provide an easy way to learn the wiki syntax. And if you go to wikipedia you will find a whole toolbox of links to documentation about thing you might want to do with that particular page. And to use 30boxes again they provide right on the main page an example of how to add an event.

Because the interface for a page in a web application takes up the full size of a web browser and the web application is mostly text (rather then buttons, lists, and menus) it is easy to think about adding help into the page. Not only can you integrate help into the webpage, but you can have the help be determined upon where you just were and what your current task is. And to top it all off because the help is right there users will actually read it unlike most application documentation which is read by so few users. Part of the reason might be that with web applications you can review the logs and see that everyone stops using your site on page X. Adding a tiny bit of docs to that page then gets everyone past it. As a developer having the docs embedded within your application I bet you would find yourself caring about it a lot more too.

KDE actually has a built in mini-documentation features, called WhatsThis. Of course I think you all will agree that WhatsThis is used probably less then someone reading the actual documentation. As sad as it is tooltips provide the majority of documentation for users. Some widgets do provide more such as the OS X search widget actually has some text inside it telling you what it does. If you goto digg.com you will notice that same thing in their search box. Browsing around in applications I actually had a hard time finding much embedded documentation. The closest thing is toolbars that have text under them. Maybe clippy has something to do with this. There was such a backlash against clippy that no one has dared try anything similar. Today many believe that there is some magical interface that is so user friendly anyone can use it without any documentation no matter how complicated the task.

Here are just a few simply ideas on how documentation could be more integrated with the application:
- Now that we have opengl windows you could flip a window over and on the back would be the documentation for it.
- A drawer that would contains mini paragraphs about the current operation and links into the docs for more information.
- Clickable lables with links into the documentation or even just pulls up the documentation in a drawer.

Of course these are just some ideas off the top of my head. User interface testing would be a key aspect to finding out what would work best. Suffice it so say when you use online applications that has embedded documentation it makes the entire experience easier and better.

Lets have a meeting

It is amusing to watch the new internet calendaring companies as they fumble around. Yahoo has one, Google now has something called C2 and there are many other sites out there such as 30boxes which all seem to do a good job as a calendar. Honestly though, most people don't really care to have a calendar. For those people that want to write out their schedule they will use a calendar on a piece of paper in their back pocket or a PalmV (which still rocks btw). You might convince a few of these people to be your customer, but after that you are dead in the water and have no revenue.

People keep creating calendaring applications because you typically use calendar apps to schedule to meet with other people in the office. That is where the tide of people are hiding (and money). The main reason you want your calendar online is so that others can find out when it is best to meet with you. Once you get one person using your site to schedule meetings then others might use it more and soon it will be a little virus sucking more and more users into using it because it provides a valuable service of having everyone else's calendar and a easy way to schedule. If you made the front page of your "calendar" site be four lines listing your e-mail/im, the person you want to meet with, date, reason and a big schedule button you probably would do better then a lot of calendar sites out there today. Google can no doubt leverage gmail and gtalk to make an easy scheduling system (and oh yah have a calendar too). If you go to 30boxes you can tell that scheduling a meeting is much more of just another feature and not the central part of the application.

What is the most surprising thing about meetings is that it is an amazingly simple way to grow your calendaring service exponentially. And if you get a manager that uses your calendar he might just tell everyone in a company to use your stuff and bam a lot of new users all ready to schedule weekend stuff with their friends, continuing the cycle of more users. Creating a community around your application is a good thing and is something that can get you laid.

If this seems familiar remember about a year ago now when Nat announced Hula. JWZ created a often referenced blog entry called Groupware bad telling how he was going to fail. The current set of calendars are better then Hula, but they still are missing the mark.

Of course I could be wrong and most people really do just want to have a calendar they themselves fill with things they are doing to view only by themselves, but I think that a much more likely answer is that calendaring sounds like a simple web2.0 company you could make and flip (how else would you come up with a horrible company name like 30boxes? And did you notice they have a gmail theme?). If that is your goal you don't think about growing users as fast as possible, but think about making a feature set that someone else would want to buy.

Monday, March 06, 2006

How much free space do you have?

How much free space do you have is a pretty basic file manager issue that most linux file managers don't easily tell you. That is until today when I ran across Thunar today. Pretty slick.

Monday, February 13, 2006

Simplicity in design: an example

On my personal website like many people I have a photo gallery. My web hosting provides 2GB of space which was a large sum when I first signed up. I was already pushing the limit when last spring I got a Digital Rebel XT camera. At eight megapixel the file size it produces is conciderably larger and by the end of 2005 not only had I vastly surpassed the 2GB limit for the minimum that I wanted to put up, but my monthly backup for my full photo collection took up several DVD-R's. When I first started hitting the 2GB limit I would deleted older photos (or ones I didn't like) from the site, but I realized that this was not a long term solution and either I would have to pay for more space or do something about the photos.

Part of the problem stemmed from the fact that my photos were stored at the highest quality the camera could give me. I regret not doing this with my earlier digital photos that will forever be the low quality and low resolution that they currently are. The other problem is from the image size. The Cannon produces images with a resolution of 3456x2304 and my older Casio is respectable at 2048x1536. It was cool to offer them at that size online, but really 1600 or 1280 will probably satisfy almost everyone, and the few that want the higher resolution can always e-mail me for the file. Armed with those two bits of information I knew what needed to be done. I set out to write a script that would 1) go through and make a copy of every photo from directory A into directory B (keeping all sub-folders) and 2) resize the images to 1600 if they were larger and lower the quality to 75 for those that were jpg's.

With those specs I quickly stubbed out a recursive function that would take a directory (A) and a destination (B) for the duplication. It would read everything in A and if it was a directory make it in B and then call itself to parse B. If it was a file it would copy the file and if it was a jpg and larger than 1280 then resize it.

After writing it I got the same feeling that something wasn't right. This bash script took more then a hour to write, 95% of that time was wasted fixing issues caused by file names and directories with spaces and was an bulky 55 lines. This was not a "hard task", a dialog in digikam that implemented this feature would probably have less lines of code and taken a fraction of the time to code. The code seemed way to brittle with a lot of edge cases. I set aside the project and went to bed. The next day (like it always happens) I realized my mistake. My problem had been with how I had worded my original design. The original design had directed my implementation down the wrong path. I quickly realized what my design should have been:

Make a copy of all my photos and lower the resolution and quality of the jpgs.

With a new awareness of the problem I quickly wrote a script that did the job in just two lines. Once you added in error checking for the argument, you get a total of nine lines.


#/bin/bash
to=$1_web
if [ -d $to ] ; then
echo "$to already exists";
exit
fi

cp -ar "${1}" "${to}" || exit
find "${to}" -regex '.*jpg$\|.*JPG' -exec mogrify -quality 70 -resize '1280x>' '{}' \;


Not only is it simpler, but it does away with several issues that I had come up with in the fifteen hours sense I had written the first version. Also due to its simplicity there is no chance of any issues due to the file names having spaces. Best of all it took ten minutes to write and test which is what I had expected the original task to take.

Many of you have probably heard the following story:
A manager gives a task to his three employees. The first employee, fresh out of school goes right to his desks and starts coding a solution. Eight o'clock that evening he finally finishes, having to only change 2000 lines. The second employee who had worked there for a few years sketches up some stuff on his whiteboard, creates his solution at 1000 lines and leaves at five. The third employee goes for a walk around the campus with a notepad. When she returns she deletes 200 lines and changes 100 before heading home at three. Who was the most productive?

Simplicity in design is often spoken about in books and so I thought I might write about a real world example that is small enough for a blog. After coding my first solution I find a bit of elegance and pride in my final solution even though really it was my folly in my first design that was the problem. When designing code listen to your gut. If something seems to ugly, too convoluted take a minute to step back. Take a walk with a notepad, (don't check your e-mail!), even explaining your design to someone else might be the perfect way to discover the real solution simply by trying to articulate the full problem verbilly. Your friend will probably be able to poke holes in your design or suggest new avenues to explore.

Coding can be a lot of fun, I especially enjoy moments this the above while walking to work I realized that my difficult problem wasn't difficult at all and just like I had originally hypothesized, was decidedly simple.

Saturday, February 11, 2006

Massif for memory profiling

So you might have noticed that your application is using a bit of memory, but it wasn't *that* much so your didn't worry about it too much. Then one day maybe you noticed your app was using more memory then Konq and you don't have a leak. What do you do? Use a heap profiler.

You have probably used KCachegrind to improve performance, but did you know that Valgrind includes a heap profiler too? Check out the massif documentation which should get you started. I have found a Gnome massif tutorial that provides a walkthrough of a gnome application and most everything applies to KDE applications too.

KDE 3.5.1 is better then ever, but there are still applications that use way to much memory. On my system I regularly find Konversation and Akregator using more memory then KMail and Konq combined. I profiled them both and found some interesting results while exploring what massif can do.

Applications that use a little amount of memory matter too. I profiled a small Qt4 application I have today and was able to save a few MB after just a few small changes. Small savings really do add up and are important. Recently in Jim Getty's blog he has talked a number of times about how the $100 laptop will have 128MB of ram total for the system, desktop *and* running application. Recently he discussed how the gnome-clock-applet was using more then 60 megabytes. With the conversion to Qt4 KDE applications are in the perfect opportunity to show that KDE can be really light. Running the most basic KDE3 application uses more then twice the memory (total including libs) of some Qt4 example application's I tried. Just migrating to use Qt4, KDE applications should get a nice drop in memory size. But that is just the beginning, just like you take time to speed up your application with KCachegrind so should you profile your memory usage from time to time. And if you refractor your application while porting it to Qt4 take a minute to see if you can decrease the memory requirement. Many systems today still ship with only 256MB. KDE applications are in fantastic shape and we will be getting better in the next year and with tools like massif we will have no problem providing a desktop and applications with a low memory footprint.

Sunday, January 22, 2006

Artificial Intelligence for Computer Games Book Review

This week I read "Artificial Intelligence for Computer Games" by John David Funge. The first thing I noticed when I got it up was how thin it was. It reminded me of the small reference O'reilly books. Props for having a hardcover though. I think that it is really called "Artificial Intelligence for Computer Game An Introduction", but you would only know that by seeing it on the first page as that isn't on the cover, side or back. Reading through the preface I found that he hopes his book will inspire new academic research for AI in computer games and that he is working for a startup that is developing AI technology. This first I doubt will happen and the second clearly showed me where the next hundred pages was headed.

Before getting into the book I have to mention the code. You get your first glimpse of code on page seventeen where a class header is shown. The class name is tgGameState. Any guess what "tg" stands for? Neither do I. He tries to save on space by having functions with partial words like "inline getNumCharacters()", but the follows it with a pointless comment // Get the number of characters. In appendix B (Programming) it says that code is written to be as easy to understand as possible and is therefore not that efficient. If he had wanted to go for readability he would have expanded the function names, removed the pointless comment, and ditched all the inlines and not of even mentioned the constructor, deconstructor (which aren't defined in the book anyway) etc. Poor variable names, missing error checking, it would have been much better to use sudo code. I noticed a few bugs even. Overall this was the worst part of the book and I wouldn't want someone in the "entertainment industry" reading this if I was trying to sell them my code.

Onto the actual book. My mention of the reference O'reilly books wasn't just to point out the size. This book really does feel like a jumping off point for AI in computer games. topics are briefly mentioned, but never really gone into depth and to make it sound complicated greek symbols are used when showing a formula. I would have appreciated five or six footnotes per pages telling where to get more information, but most of the time there wasn't. The first two chapters where more of a crash course in game design. So by the time I was on chapter three and on page 33 you can tell that was nervous that i was 1/3 through the book and really hadn't gotten into any sort of real AI stuff. but it picks up from there. There are a lot of hints for how to integrate AI into games. For example a Non-player controller (NPC) could have an arrow drawn on its chest (where it thinks the player is) and other visuals indicating its internal state. Of course these are more likely just basic game tips and nothing to do with AI. One neat idea was that your NPC could have several decision making units that could be swapped out. When really close to the player the most CPU intensive one would be used and when far away in the locked room the "stand still" one could be used. Perception, Mood, Remembering, Searching, some basic physics were all touched upon. In chapter 7 it gets very close to mentioning/talking about genetic algorithms, but alas it was not to be. The last part of the book was very boring and I found myself putting down the book a lot before finally finishing it.

Overall I feet like I was cheated because of the poor cover title. The cover really should have had "an introduction" on it. I expected it to be bigger with more in-depth explanations that didn't leave me hanging, and not such sub-par code for a coding book. On the plus side I found out the name of the orc on the cover is named "Fluffy".

I have previously listed some books I highly recommend and am interested in hearing what you recommend is good.

Friday, January 20, 2006

66 million rows in QTreeView

Inspired by the recent work with gtktreeview I decided to see how far I could push the item view framework. Modifing the interview demo from 1000 rows by 100 columns I was able to get around sixty million rows by twenty million columns. Once I would go past that (somewhere around ~75 rows and ~25 columns) I started hitting some int overflows, but even with showing that model in the tree view, list view and table view took about 1 second (including showing the icon, text etc) for the application to run. Not bad I must say.

Sunday, January 15, 2006

Job Trends

On reddit there was a link comparing job postings for lisp, python and ruby.

So of course I had to put in for kde and gnome. There were both near zero and gnome even did have 0 at one point. Either way they are both so small that the difference is inconsequential. But when you put in some different keyboards you will find out some more interesting things.


  • There are about as many jobs for khtml as gtk+

  • Companies don't seem to hire gtk programmers, but mono (now the future of gnome with it in fc4?)

  • The mono and gtk graphs seem a little in sync again suggesting that if you program in gtk you should know mono too.

  • There is about twice as many people hiring with qt experience as mono

  • At least right now ajax really does pay to learn

  • Companies want more "developers" then "programmers"

  • More companies are looking for qt experience than coco (even apple had a job posting looking for Qt experience)




Oh course this is just a graph off some web page so take it with a grain of salt.


On monster.com you can subscribe to keywords and they will e-mail you when a new job is posted with that keyword. It was fun watching as the Qt job postings grew from one every few months to sometimes more then once in a day ( at which point I unsubscribed). It was very easy to see what kind of companies were hiring qt developers. I bet you there is someone out there who is exploiting similar information on the stock market for tools companies. Knowing what a company is hiring often shows where a company is and where it is going.
"

Friday, January 13, 2006

Emotional Design Book Review

I read "Emotional Design" by Donald A. Norman this evening. After reading his other book "The Design Of Everyday things a few months ago I had high hopes for this book, but after reading it I felt like it was just a collection of notes put together. The main idea took up the first half of the book teaching you something that could just as well have been in a well written 12 page essay. The gist of it is that emotional design is broken down into three things:

  • Visceral - Appearance - Look at that, it is shiny and so smooth!

  • Behavioral - The pleasure and effectiveness of use - My ipod is a pleasure to use because it just *works*<.li>
  • Reflective - Self image, personal satisfaction, memories - I remember the first time I saw someone else wearing white headphones too, the ipod is fun to show off to people who hadn't seen it.


Now those three things mean very different things to different people. A good example is given in the first chapter:

Go outside. Get some air.
Watch a sunset.
Boy, does that get old fast.
-XBox advertisement

This is a very well designed add, but really only for guys 18-35. Everyone else will get different results for the three sections.

Take Qt (at least for me)
  • Visceral - Eye Candy, Nice themes, Easy to read GUI code

  • Behavioral - I enjoy using it to create my applications, the design is consistent and works with me

  • Reflective - Learning Qt was a breeze due to the well written documentation, I really enjoyed showing off what I was able to do and accomplish in a short amount of time.

You learned all this in the first chapter and the rest of the first half of the book fleshed it out. For example you can have something that is annoying to use and doesn't look that great, but has a really high Reflective level (such as an expensive car you can show off) that makes it worth while to have.

The second half of the book seems to jump from topic to topic vaguely related. There are two sections all about robots and giving them emotions, incorporating them into our lives etc, but this just seemed like filler and didn't contain anything new that isn't covered in many other books. The author spends some time dissecting movies, video games, and many other thing too.

I was expecting much more a continuation of "The Design Of Everyday things" (which is very good) and so I was very disappointed. Overall an interesting book to barrow from the library or pick up used off Amazon.

Sunday, January 01, 2006

Fireworks in Norway

Went to a new years party this evening and at twenty of twelve we left the apartment and headed a few blocks away to the local park which has hill and a very nice view. As we were walking there it was mesmerizing to watch as more and more people were appearing from every direction heading to the park creating a large stream of people up the hill. All day long I have heard fireworks going off and even from our window we watched a father and his kids set off some large fireworks in the parks across the street earlier in the evening. While we walked down to the party we heard (but couldn't see most) a firework or two every minute. When we turned the corner and could see the park we could also see many fireworks also going off all over the hill. We walked up the hill and onto the first flat area. There were people everywhere and in the middle there was an unofficial area where people were walking up and launching their fireworks from. A number of other similar spots could be seen elsewhere in the park. The fireworks being set off were not little fireworks, but every size, many very large (that wouldn't be at all legal in the US). Now I have seen big fireworks (mostly at forth of july in Boston from the river bank), but not fifty feet above me. Because there was no government run show (let alone a synchronized radio firework show) for about a hour people where continuously setting off fireworks everywhere you looked. And up on the hill you could see the same thing was happening all across the town at other parks. Look left right, behind you, in front of you everywhere another massive firework was going off. Occasionally the fireworks would explode on the ground or just a few feet off the ground spitting out into the crowds. I realized that I probably had a better chance of getting hurt then anytime in my recent past. Words really can not describe the shear chaos, noise and marvel. So of course next year I will definitely be going out again.

Popular Posts