Discussing the average man

Today we celebrate the birthday of Adolphe Quetelet (22 Feb 1796 – 17 Feb 1874), a Belgian mathematician, statistician, astronomer and sociologist. Quetelet made significant contributions in the application of statistics to sociology. He was a pioneer in the field of probability theory, applying it to social phenomena, and crime in particular.

Quetelet developed a unique method to statistically profile people. He defined the concept of the ‘average man’ – a theoretical construct that represented the average value for a wide range of human characteristics. In other words, Quetelet conceptualised a person who was average height, average weight, average age, average intelligence, etc.  Real individuals would therefore be grouped around this average man according to a normal bell curve. Quetelet’s average man was useful in people profiling, as real people could be defined in terms of how much they differed from the average man.

If you walked past the average man, would you recognise him?(© All Rights Reserved)
If you walked past the average man, would you recognise him?
(© All Rights Reserved)

I find this ‘average man theory’ fascinating. Imagine meeting a real version of the absolutely average man. Would he seem average? Or would his incredible averageness actually make him stand out? How will the average people from different nationalities compare? Imagine putting the average American, Aussie, Kiwi, Englishman, German, Frenchman, Indian, Chinese, Japanese, South African, Italian, Russian, etc in a room together (each equipped with Douglas Adams’ Babel Fish, so they could communicate). How do you think they would get along? Who would be the smartest? The strongest? The most obnoxious? The most aggressive? The fattest? Who will be best able to survive in a jungle, Bear Grylls style?

And how has the average man evolved over the past 200 odd years, since Quetelet first came up with the idea? Imagine putting the average men (from a specific nationality of your choice) from 1813, 1913 and 2013 next to each other – how would they differ?  For one thing, Mr Average 2013 would probably be older than his predecessors, given how populations are aging. Depending on the population we are operating in, he may also be more overweight. Will he be more intelligent than those before? Wil he be happier or more prone to depression? Who will be physically the strongest? The fittest? Who will have the best posture?

Of course, given enough statistical data on the different populations, the above answers should be available. I just don’t know what they are, so I can only wonder. And the most interesting thing is that we all have our own preconceptions about different nationality stereotypes. I am sure if you could select Mr Average from each of the world’s nationalities, and had to pick the most intelligent, the strongest, the most obnoxious etc, you would most likely already have someone in mind. The fun bit will be to see, given hard statistical data, just how wrong our preconceptions and stereotyping may be!

Regarding programmers, monkeys and probability

Today, 7 January, we celebrate International Programmers Day. According to some sources, at least – there is also a Programmer’s Day happening on the 256th day of the year, 12/13 September, depending on whether it is a leap year. In a way the 256th day option makes more sense, as 256 is 2 to the power 8, which is the number of distinct values that can be represented with an eight-bit byte – something sure to amuse the programmers among us.

What makes 7 January an amusing date for a day dedicated to programmers, is that this is also the day that the French mathematician Emile Borel (7 Jan 1871 – 3 Feb 1956) was born. Borel, a pioneer in the field of probability theory, is the man who proposed the famous thought experiment that if you allow a monkey to randomly hit a typewriter’s keys, it will, with absolute statistical certainty, eventually type every book in the French National Library (known as the ‘infinite monkey theorem’).

The infinite monkey theorem, as applied to programming. (© All Rights Reserved)
The infinite monkey theorem, as applied to programming. Considering the quizzical look on his face, I’d say he’s just created some low-level assembly code…
(© All Rights Reserved)

Applied to programmers, does this mean that, if we let a monkey loose on a computer keyboard it will, given enough time, eventually write the code for every operating system and every computer programme ever developed?

I guess so… Which would support the often held opinion that people in IT get paid way too much… 🙂

Just kidding – of course I have the utmost appreciation for the computer whizzes who keep all the systems around us going. And when you think about it, there is almost no aspect of our daily lives that does not, in some way or another, have an aspect of programming involved in it. We certainly live in an age where IT is super-pervasive, and as such it makes perfect sense dedicating a day (or two) to celebrating the contributions programmers make in our lives. Good on you, each and every one!

Enjoy some mental gymnastics on Card Playing Day

Today, 28 December, is Card Playing Day – the day to celebrate all games involving your classic deck of cards.

When you think about it, a deck of cards is a pretty impressive creation – the diversity and complexity covered in all the games using a card deck is quite staggering. From games testing cunning and deception (poker), to games teaching teamwork and planning (bridge), to those based on statistical probability and counting skills (blackjack), to visual pattern-matching games (rummy), to the single-player solitaire/patience type games, and hundreds more in-between, the options are almost limitless. And all this based on a simple collection of 52 playing cards, involving four different ‘suits’ of 13 cards each.

Playing cards - a world of complexity lurking in a deck of 52 cards.(© All Rights Reserved)
Playing cards – a world of complexity lurking in a deck of 52 cards.
(© All Rights Reserved)

Playing cards have a long history – they were first found in China as early as the 9th century, and appeared in Europe around the 14th century. The first card decks containing the now-standard 52 cards consisted of suits with themes like polo sticks, coins, swords and cups. The famous suits of spades, hearts, diamonds and clubs, as we still use today, was first introduced in France around 1480. The Kings, Queens and Knaves (Jacks) in the different suits were based on English and French history, and referred to different historical characters such as King David, Julius Caesar, Alexander the Great, and others.

Beyond the historical connotations, a range of symbolic meanings are attached to the deck of cards as we know it. The 13 cards in each suit is said to refer to the 13 months of the lunar year; the 52 cards corresponds to the 52 weeks in a year; the Ace, which is both the lowest and highest card in each suit, is symbolic of the beginning and end, alpha and omega.

From a scientific point of view, playing cards represent an invaluable demonstration and teaching aid in fields such as mathematical logic, probability and statistics.

Whether you enjoy playing cards for the thrill and uncertainty of games of chance, or because of the complex mathematics they represent, or simply because of the social interaction inherent in many card games, today is the day to celebrate all facets of card playing. So while you’re enjoying that pleasant lull between Christmas and New Year, why not pull out a deck of cards –  play an old game, learn a new one, and lose yourself in the mathematical complexities hidden in your standard card deck.

Dice, gambling and statistics

Today is Dice Day. Actually it is National Dice Day, a US-only event, but what the heck, I wasn’t able to find a more interesting international celebration for today, so we’ll just drop the ‘national’ and internationalise it for now, if you don’t mind.

A die (or dice, in the plural) is essentially a small object with multiple sides/resting positions, and with different numbers or symbols inscribed on its various sides. It is designed such that, when it comes to rest after being thrown, each resting position has a main, opposite upper side. The number/symbol on this side represents the number of the throw. Dice are often used in gambling games and in many popular board games.

The more dice you throw, the more complex the maths can become.(© All Rights Reserved)
The more dice you throw, the more complex the maths can become.
(© All Rights Reserved)

The most common die is a standard, 6-sided cube, usually with slightly rounded corners, with a number of dots (ranging from 1 to 6) inscribed on each side. When thrown, each side of the die is equally likely to end up on top, the result thus being a random integer between 1 and 6. For specific games, multiple dice may be thrown at the same time, with the outcome given by the sum or combination of numbers on the dice.

Of course there’s no reason that a die has to be a cube. Various other equally valid shapes exist – tetrahedrons, octohedrons, dodecahedrons, and more. When you think about it, even a coin used in a coin-flip is essentially just a two-sided die.

Dice are not only useful for gambling and board games – they are also a very effective tool to teach and explain some rather complicated statistics, like conditional probability, that is, the probability that an event will occur given that another event has occurred.

Take the example where we throw two standard, 6-sided dice. The probability that the second die will land on a 1 is 1/6 (it can land on one of six values, all equally probable – simple enough). Now, what would be the probability that the second die lands on a 1, if we know that the first die landed on a 1? The answer is still 1/6, because the two events are independent.

Things change, however, if the events are no longer independent. For example, what is the chance that the total of the two dice will be less than 4, given that the first dice landed on a 1? Firstly, there are six possible values (1,2,3,4,5,6) for the second die given that the first die landed on a 1. Of these, however, only two options, 1+1 and 1+2, result in totals smaller than 4. So, the chance of the total of the two dice being less than 4, given that the first die landed on a 1, is 2//6, or 1/3.

Formally expressed, p(total < 4 | die1 = 1) = 1/3.

There you have it – just when you thought you were safe in the knowledge that we were going to stick to chatting about harmless things like gambling and board games, you got flashed by a lesson in statistics…

Sneaky little buggers, these dice! 🙂

Location, location, location (and time) – it’s GIS Day

November 14th is GIS Day, an annual event focusing attention on the field of Geographic Information Systems, its use and potential to impact on our lives.

GIS Day started in 1999 to create an opportunity for people to learn about geography and to discover and explore the benefits of GIS.

Spatially mapping your data enables you to identify trends and relationships that might not otherwise be apparent.
(© All Rights Reserved)

So what exactly is GIS? According to Esri, one of the leading international developers and vendors in the field of GIS, “A geographic information system (GIS) integrates hardware, software, and data for capturing, managing, analyzing, and displaying all forms of geographically referenced information. GIS allows us to view, understand, question, interpret, and visualize data in many ways that reveal relationships, patterns, and trends in the form of maps, globes, reports, and charts.”

Or, as Wikipedia puts it: “In the simplest terms, GIS is the merging of cartography, statistical analysis, and database technology.”

Spatially visualising information has many benefits. GIS enables us to map where things are and in what quantities and densities they are distributed. Modern GIS tools also allow us to map and visualise changes in these quantities over time. By seeing how various fields of data are dispersed geographically, and how they are changing, it is often possible to identify trends and relationships that might not otherwise be apparent.

This in turn leads to better decision making and improved communication.

GIS is a pervasive supporting technology throughout all aspects of modern society, with applications in business (banking, retail, etc), law enforcement, health, transportation, environmental systems, conservation, agriculture, forestry, mining, telecommunications, utilities management, research and education.

Capturing spatio-temporal location is key to GIS.
(© All Rights Reserved)

A GIS can typically employ and integrate data from a huge range of sources, as long as it has some key through which to relate it to the other data in the system. This key is spatio-temporal location – you need to know the location and time represented by the data. To map climate change, for example, you would include information on temperature and rainfall. But just having a list of temperatures and rainfall figures means nothing – to make it useful, you need some indicator of where and when each value was measured.

By promoting an understanding of this simple basic concept – that you massively increase the value and usefulness of any set of data by recording and including the spatio-temporal location of each data item – time and money spent on data collecting efforts can be leveraged so much more effectively.

Are you involved in data collection? Know someone who is? Even if space and time appear unimportant, record it anyway. Who knows – you may just discover something no-one’s thought of before…

Alfred Binet, the father of the IQ test

Today we have no big, UN-sanctioned observances. While there have been a few notable births on this day, none of it really caught my fancy. So instead, let’s commemorate the work of Alfred Binet, who died on this day in 1911.

Alfred who?  Well, Alfred Binet was the guy who, together with psychologists Victor Henri and Théodore Simon, developed the Binet-Simon Test – a test for verbal abilities in children. The test was later adapted by Lewis Terman at Stanford University, resulting in the Stanford-Binet Intelligence Scales, published in 1916, which was the most popular intelligence test for decades in the US, and formed the basis of IQ testing over the past century.

Yes, Alfred Binet is the father of the IQ test.

For the past century, IQ testing has been used as a measure of future educational achievement, special needs, job performance and income.
(© All Rights Reserved)

Mention the concept IQ testing, and you are very likely to get some really strong opinions. Some people (especially those treated favourably by the IQ scale, I guess) probably consider it a pretty accurate measure of their mental superiority. Many, however, seriously question its validity as an absolute measure of intelligence. Even Binet himself felt strongly that his tests had significant limitations, stressing that he saw extreme diversity in intelligence, and felt the result of the tests only had qualitative value, and should not be used as a quantitative measure.

It is exactly this diversity of intelligence that forms the basis of much of the criticism against IQ testing, with detractors insisting that it fails to accurately measure intelligence in its broader sense, pointing to the fact that it is not an adequate measure of creativity and emotional intelligence. It does not even come close to testing physical intelligence (hand-eye coordination, ‘ball sense’, etc).

Some critics go further, not only critisizing the scope of IQ testing, but disputing its validity entirely. Paleantologist Stephen Jay Gould, for example, in his book The Mismeasure of Man (1996), equated the tests to scientific racism, saying “…the abstraction of intelligence as a single entity, its location within the brain, its quantification as one number for each individual, and the use of these numbers to rank people in a single series of worthiness, invariably to find that oppressed and disadvantaged groups—races, classes, or sexes—are innately inferior and deserve their status.”

The above criticism rings quite true when you read about the early days of intelligence testing at the start of the 20th century. To quote wikipedia, “The eugenics movement in the USA seized on it as a means to give them credibility in diagnosing mental retardation, and thousands of American women, most of them poor African Americans, were forcibly sterilized based on their scores on IQ tests, often without their consent or knowledge.”

Given that, despite the criticisms of IQ testing, these tests have been performed extensively around the world for the past century, there’s obviously a lot of IQ data out there, which has been the source of some very interesting analyses and practices. Here’s some of my favourites:

  • Musical training in childhood has been found to correlate with higher than average IQ. As has listening to classical music (but you have to do your listening directly before the test – apparently it only serves as a 10-15 minute mental boost!). So I guess my years of listening to rock, folk and blues wouldn’t have helped much.
  • IQ has been used quite extensively in human resource evaluations, when hiring new employees etc. What’s interesting is that it’s not only a too low IQ that can count against you. Apparently some US police departments have set a maximum score for new recruits (example: New London, CT has set an upper limit of 125), the argument being that those with higher IQs will become bored to soon, resulting in a too high job turnover.
  • In terms of income, it seems that there is a general correlation between IQ and income up to a certain level of IQ, but there’s no correlation between very high IQ and very high income. Top incomes are dependent on so many factors that IQ doesn’t really feature at all.
  • As far as criminal tendencies are concerned, being very clever or very stupid (in IQ terms) generally speaking seem to keep you from pursuing a life of crime. Most criminals fall in the 70-90 IQ range (i.e slightly below average), with the peak being between 80 and 90. There is, of course, the argument that only the dumb ones get caught, so maybe criminals are much smarter than the statistics suggest.
  • Politically, studies in the UK and USA have found that young adults who classify themselves as very liberal have a higher than average IQ, while those who consider themselves very conservative tends to have slightly below average IQ scores.

There are many more interesting correlations, but let’s leave it there for now. What do you think – do you believe in IQ tests, or do you think its a load of hogwash?

I personally think that the best comment on intelligence comes from Albert Einstein (he really did leave us with the greatest quotes, didn’t he?), when he said, “Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.”

Flip a Coin day

Today we celebrate the randomness of coin flipping.  Do you have some tough decisions to make? Why not use this day as an excuse to leave it to chance, by simply flipping a coin?

The practice of coin flipping is said to date back to Julius Caesar, who used the technique for decisions where the right choice was unclear.  Roman coins had the head of Caesar on one side, so a “heads” result was considered a positive, or “yes” outcome.

Leave it to fate – today is the one day when making all those tough decisions can be as easy as flipping a coin.
(© All Rights Reserved)

The statistics of coin flipping is described through the Bernoulli process, and a single flip of a coin is called a Bernoulli trial.  The use of coin flipping examples is also a popular way of introducing some of the complexities of statistics.

Conditional probability is a specific field of statistics that is often quite difficult to understand intuitively, and is illustrated very well through a coin flipping game called “Penney’s Game”.

In this game, two opposing players each choose a sequence of (usually 3) coin flipping outcomes of heads (H) or tails (T), e.g. H-H-T, T-H-T, etc.  A coin is then flipped, and the player whose sequence appears first is the winner. If the second player knows the “trick” he is always more likely to be the victor.  The correct choice for player 2 will be to take the first two outcomes of player 1, and to precede this with the opposite of the second outcome, for example:

Player 1: H-H-T   Player 2: T-H-H
Player 1: T-H-H   Player 2: T-T-H
Player 1: H-T-H   Player 2: H-H-T
Player 1: T-H-T   Player 2: T-T-H

In all the above cases, player 2 is always at least twice as likely to win as player 1 – definitely not something that makes immediate intuitive sense!
(See the mathematical explanation here.)

Getting back to that difficult decision we mentioned earlier – if you secretly want to do one thing, but think you should do the other, use Penney’s Game and your new-found knowledge of conditional probability to stack the odds in your favour.

Come on, go flip a coin!