Ben Shoemate » Information Visualiation

A primer to communication – 1953 film

Ben Shoemate — Tue, 17 Feb 2009 00:42:00 +0000

The basics of communication presented in this 1953 introduction to "the era of communication" (aka the information age) are still true in 2003. Transmission, noise, redundancy, distortion… misunderstanding. Charles and Ray Eames were husband and wife, not brothers. They were not initially architects. Ray was a visual artist. This is a classic Eames film and a great introduction to their creativity that spanned many industries.

Insert Map feature of Windows Live Writer…not bad..not good either

Ben Shoemate — Sat, 29 Nov 2008 21:58:53 +0000

This is a short test to see how the insert map feature of Windows Live Writer works. I think I am ready to give this product my endorsement. The push pin is a bit ugly though…I mean come on.

Information Visualized – all the water in the world

Google Reader — Fri, 27 Jun 2008 10:50:39 +0000

all the water in the world (or 1.4087 billion cubic kilometres of it) including sea water, ice, lakes, rivers, ground water, clouds, etc. (left). all the air in the atmosphere (or 5140 trillion tonnes of it) gathered into a ball at sea-level density (right). both shown on the same scale as the Earth.

[link: phiffer.com|via boingboing.net & benfry.com]

Justification of Ockham’s Razor as a principle of reasoning

Ben Shoemate — Tue, 06 May 2008 17:07:15 +0000

Scientists use the principle of Ockham’s Razor as their guide. Ockham’s Razor states that when there are multiple consistent theories are being considered, the choice should be the simplest one. Simple theories have an intuitive appeal, but that is not a justification of Ockham’s Razor as a principle of reasoning. A justification should demonstrate that preferring the simplest compatible theory is better at finding the truth than any other competing strategy.

This article does a good job with great cartoons of explaining why this principle is valid. Very insightful. Here’s the link.

Update: here is an animated older version. The link above goes to the version 2 story board which has not been animated.

Powerpoint on Powerpoint – rules we should all use

Ben Shoemate — Thu, 27 Dec 2007 22:52:32 +0000

Here is a great presentation on how to use powerpoint.

7 Email Visualizations for Thunderbird

Ben Shoemate — Wed, 14 Nov 2007 00:20:42 +0000

Now that I have all my old email in one place, my next project will be to create some visualizations to start analyzing it. What do I mean? I’m talking about graphs, charts, tables, heatmaps, grids, networks, etc that illustrate patterns in the email. For example – I have roughly 40,000 emails (closer to 20,000 or so were sent by an actual human, and fewer still were sent only to me – but I don’t know for sure and thats part of the problem). My goal is to have a set of Thunderbird plug ins that will allow me see patterns like these. These are 7 visualizations I think Thunderbird (or Gmail) needs. If I get a lot of interest, I might actually build them so let me know what you think in the comments (no registration needed but I do moderate so it might not show up right away).

Report 1: social network – How many people are in my social network? – Unique names are pulled from the To, From, CC, and BCC fields – the total counts for each are shown. A tag cloud is created showing all the names. Overall, the names are evenly spaced out – but when 2 names appear in the same email, they are more related and grouped closer together. The more often names appear, the more larger and bolder they get. The cloud can be filtered to show only data from the to, from, cc, or bcc feilds or to show activity for a given period of time. Lines with arrows can be turned on to show the origin, destination and direction of email.
Report 2: A River of email – OK – so I know you are all familar with NameVoyager. No? Its a dynamic graph of Baby names (see screenshot below). Imagine the same thing for your email – years on the bottom, blue is received, pink is sent, you type a letter and the list is filtered (second screen)
.
(All Email – with count per year, per person – you can zoom in to see by month, or filter to see a given name)

This activity over time (a graph showing the peaks and valleys of overall emails sent and received). It will be interesting to see email drop off at the holidays, peak when I’m on projects with large teams, and drop off when I am working with small teams or not on a project)
Visual 3: Company wide graphs – Once I can do this, it is a small step to analyze not only mine, but a whole company’s email. This lets me build social networks. Who knows who? How well connected are they? Need to get introduced to someone – did you know your friend worked with him two years ago?
Visual 4: Integrate with your contacts – zoom out to see all your contact at once with a color indicating your activity with that person. Or just be able to sort by most popular.
Visual 5: Build a tag cloud of words you use in all your messages (like manyeyes.com). Show topic popularity this way.
Visual 6: Building on the data from visual 5, build a tool like google trends that uses words as they appear in email. This would show the topic trends.
Visual 7: World map of email sources – using server source show the geographic location of people who have sent you mail.

Of course many other visuals are possible using other attributes over time (word count, attachment size, tags, etc).

Does anyone out there have something like this already? Working on it? What other plugins are needed? And most importantly, who wants to help

How to organize your hard drive

Ben Shoemate — Sat, 10 Nov 2007 18:59:34 +0000

This is a problem every one struggles with: what is the best file structure for old files? The problem is actually compounded by the need to do routine back-ups. As I was going through all my old hard drives as part of my email project to upload all my email into Gmail. I took note of the various ways I organized my files over the years, the various folder structures I adopted. My digital life is dominated by projects – personal projects (most half-baked and half-finished), client projects (with thousands of files each with multiple versions), and team projects (like personal projects only with multiple people). Then, like everyone, I have downloaded files (in the form of mp3s, images, videos, and saved web pages), personal photos, and miscellaneous documents.

Here is my strategy…

First, ALL my documents go in the “My Documents” folder that windows provides. The first thing I do when I re-image my computer is right click on that My Documents folder and change the location to by my secondary hard drive. So the C: Drive in my laptop is for installed programs, and the D drive (I replaced my cd-rom with a second drive) is for all the files I care about. If I have to check my laptop at the airport (which I never, ever do – electronics are not covered by the airline policy) or leave it in a shady area – I pop out the D drive and put it in my pocket.

So, lets look at my D drive (My Documents folder):
As you can see, the one that is most out of place here is the “Camtasia Studio”. A lot of software when you install it, creates a folder in My Documents. The one I hate the most is Adobe, which creates a completely empty folder for NO APPARENT REASON. I delete it all the time. The other trouble maker is Mircosoft OneNote….a nice program, but…I use Google Notebook.

I also have divided my Projects at the root. Prior to this I had one projects folder with sub-folders. You have already seen the Email folder in my previous post, so lets get to the interesting one and look at Clients. This is by far the elephant in the room with 8.19 GB, 6,197 sub folders, and 91,743 files.

Inside the Project (Clients) folder, I have organized things by year, and client name (sorry – I thought I better blur out the client names – that’s not important here anyway we are talking about the structure itself). Alright now, stop trying to read it! If you want to know where I’ve worked just ask me, I’m not ashamed.

Before this I did not have the year as part of the title, which was good because I often go back to work for the same clients over and over and all their projects grouped together, it also allowed me to press the first letter of the client to jump to that folder. So why add the year, because it adds consistency with how I organize my other folders such as Photos and helps with Back-up.

Next lets look inside one of the client folders.

Reference Folder Structure
In 2002 I started to notice a growing problem, although my projects have similarities, I was difficult to back and find files from old projects because I was not following a naming convention. To be honest, since I don’t work with the same team from project to project, naming conventions are hard to follow. I come with mine, others come with theirs and we end up compromising – taking the best of each and doing what is right for the project and the client.

The idea was to start each project with a set of empty folders that map roughly to a pattern that had been established on many projects. I then retro-fitted the structure to old projects.
Here it is (you can also download a zip file that creates this empty structure)

It is still a good structure, but it has a price – first of all, when you start a project, all the folders are empty, so you spend time clicking through them, looking for files that don’t exist. So you get annoyed and delete the empty folders. At this point, you start creating folders on demand again. This would be solved if windows had a better file management system using tags or at least showing file count at the folder level.

As for Music, I let itunes orgainize it, I don’t like how it does it, and I hate how when I reinstall itunes I loss at lot of info like playlists and rankings.

What else? If you have a question, our found this useful, tell me in the comments.

Google more popular than sex (at least according to google)

Ben Shoemate — Tue, 16 Oct 2007 16:55:52 +0000

I was playing with Google Trends yesterday. With a few keystrokes you can satisfy your need for both useless trivia interesting facts and the data to back it up. Where else can you discover that basset hounds are more popular in Hungary, that Hillary Clinton is finally more popular than her husband, or that in spring of 2007, google became more popular than sex:
Here are some more fasinating trends (at least to me).

People search for terms like “full moon” with the clockwork regularity of celestial events like, well, like full moons.

The holidays also show a steady heartbeat in search. I thought it was strange that “Earth day” would be more popular than Easter.

But then i remembered two things. First people are still working on Earth day, so they are at their computers, and second, google skews the results with those cartoon logos they put on their main search page. When you click it, it searches for that term. So bottom line, this just means more people are using google on Earth day than Easter. It makes sense but also reminds us that we have to take these graphs with a grain of salt.

Other things that follow a pattern: American Idol, which is apparently very popular in the Philippines.
IBM’s star is falling (its as popular now as summer is in the winter) and Lenovo’s is rising.

Point A in the graph is where Lenovo bought IBM’s PC business. One last graph, this one ties into both IBM and portals. Since I spent a lot of time helping clients get value from their portal investments, lets look at where the market interest lies.

This may be apples and oranges since Sharepoint is used for collaboration more than portals (and I’ve always felt it creates silos for information more than it tears them down, but thats the subject of a later blog.) Tell me about the interesting trends you discover in the comments.