Tuesday, December 27, 2011

Rationality in Multitasking

There is typically a big gap between the tasks studied in psychological experiments and real-life cognition. This is because psychologists want to study the individual psychological functions in great detail, and therefore develop tasks that try to isolate these functions. Although this offers a wealth of information, the danger is that the interaction between those functions is not well understood. For example, in the example of the attentional blink that I discussed earlier, researchers still do not agree on whether the phenomenon is due to a attention, memory or control.

When Dario Salvucci and I were working on our book on multitasking, we also found an example of this gap. Observational studies had shown that the cost of interruption can be very large, in terms of tens of minutes. However, interruption costs in experiments are typically in the order of only a second. To study interruption behavior at a more realistic scale, Dario and his graduate student Peter Bogunovich designed an experiment that was in between real life and a basic experiment. Subjects had to answer emails for which they had to look up information using a web browser. Occasionally though, a chat window blinked in the background. If subjects clicked on this window, a question about movie preferences was asked which they then had to answer.
The important aspect of the task was that subjects had complete freedom in when they wanted to switch between email and chat, something that is typically not done in experiments. They could switch to the chat right when it would come up with a new question, or they could wait until they were done with the current email. The mail task was structured in such a way that information had to be remembered. For example, in the example in the picture, the price of the mp3-player Killor U-32 had to be looked up, which takes three clicks in the web browser. In the real experiment (as opposed to the picture) the windows were always on top of each other. So, when you have just clicked on "mp3-player" in the browser, it is not very clever to switch to the chat, because then you probably have forgotten "Killor U-32" when you get back.
It turned out that the subjects in Dario's experiment were indeed smart about this, and continued on the mail task until they reached a smart switch point, a point during which they did not need to remember any information.

Together with Jelmer Borst and a group of project students (Joost Timmermans, Anita Drenthen en Tom Janssen), we redesigned the experiment in order to try to tempt subjects into switching to the chat window at moments that information from the email had to be remembered. We did this by introducing delays in the web browser (after clicking a link it took a few seconds before the subsequent page appeared) and the email program (it took a second for an email to load). As a result, a substantial number of subjects now switched from mail to chat during a delay in the browser, at a moment when information had to be kept active. Moreover, the average time needed to answer an email with delays was longer than without delays, even if all the delays were subtracted from this time first. In the strongest condition the extra cost was more than 6 seconds (ok, still not minutes, but better than just a second). In other words, subjects would have been better off if they had just waited during delays instead of making a switch. Trying to use the waiting time for something else turned out to decrease efficiency instead of increasing it.

What these experiments show is that people are typically smart about their choices in switching from one task to another. But delays can surely thwart this rationality. Apparently, we'd rather act than wait, and that is probably why I prefer taking the bicycle over waiting for the bus, even though the latter option might get me there faster.

Saturday, December 3, 2011

Become a better multitasker through... sport!

Last week I gave a talk about multitasking in a symposium "The Science and Art of Brain Maintenance". Two of the other speakers in the symposium, Chris Visser and Erik Scherder, talked about the benefits of physical exercise. We all know that exercise is good for your physical health in a great number of ways, but Chris and Erik reported on research that shows that exercise can improve your mental prowess. Chris' research demonstrates that children that are good at sports on average also perform better in school on mathematics and language. Erik showed that exercise during your lifetime is correlated with a lower incidence of diseases like Alzheimer's and other forms of cognitive degeneration.

At first this may sound surprising, but if you think about it, sports is not just a matter of physical practice but also of mental practice. Sports requires motor coordination, discipline and skill learning. According to Chris and Erik, the main cognitive function of importance that connects physical and mental prowess is executive control. Executive control, located in the frontal areas of the brain, concerns our goals and the handling of our goals. It keeps us focussed on what we do, help us in determining our actions in the absence of outside information, and plays a role in juggling all the items that we needs to maintain in working memory. Sounds familiar? Those are exactly the functions that are needed for multitasking.


In my own talk, I showed a picture of brain pictures associated with sequential multitasking: an area in the frontal cortex and an area in the parietal cortex (made by Jelmer Borst). To my surprise, both Chris and Erik showed pictures of roughly the same areas.

If we connect the two together: physical exercise improves executive control, and executive control is the main factor of success in multitasking, then we can conclude that physical exercise can make us better multitaskers.
The pitfall of this line of reasoning is that most evidence is correlational. We cannot be sure whether physical exercise actually causes better executive function. Apart from that, it is an interesting thought that this is another reason why body and mind are maybe not as separate as we tend to think.

Monday, November 21, 2011

What makes a Genius?

After last post's New York experience, I will add a new one about one of the other hubs of multitasking frenzy, London. What do you do when an Apple laptop breaks down on a trip abroad? You go to one of their Genius Bars! Or, the story of how we spend half a day in an Apple Store.

Let me try to explain this without too much technical detail. The problem with the laptop was that it didn't boot, and this problem was caused by repeated attempts to power it up on an empty battery. The attempts made by our Tech gave some interesting insights in how these people are trained. What he basically did was attempt a series of solution procedures that he selected on the basis of association with the problem. Our two "symptoms": computer did start up, but got stuck in the process, and something wrong with power. In this case, the power was a red herring: the real problem was the computer not booting. But before acting on either symptom the first procedure the Tech did was to hook up the computer to a standard hardware test. This showed up no problems with power nor hard disk, but did show up a problem with some sensor: the second red herring. This is something that sometimes happens in the medical profession as well: you go to the doctor with a problem, they do a test, and you end up with two problems.

As a result, the Tech went backstage to try and push all the sensors back in place. An hour later he was back from doing this, with no result. No surprise in retrospect, because the sensor couldn't possibly have been the problem. He then started going through a whole bunch of procedures related to power, up to verifying that our battery was past its expected lifetime (which we already knew), and verifying that the power adapter was indeed working fine.
In the end he did not solve the problem, but I walked away with a pretty good idea what was wrong and fixed it when we returned home.

The point here is not to criticize the Tech. He was obviously not a genius in the normal use of the word, but he was trying his best, but probably on the basis of the wrong strategy prompted by the wrong training. His approach, to reiterate, was to throw solution procedures onto the problem based on associations with the symptoms. What he should have done is try to reason out what possible causes there are of a computer not booting. Defective hard disk? No. Will the computer boot from another source? Yes. Then there must be something wrong with the OS: it needs to be fixed or reinstalled.

So why is this story important for multitasking? In order to be a good multitasking, your skills have to be flexible. When I am cooking from a cookbook, I have to pay close attention to the order of the steps and the ingredients I need to gather. I cannot really multitask very well while cooking from a recipe. But when I am not cooking from a recipe, I can happily watch the news, chat with people and do other things. So why is that? When I am not working from a recipe, the representation of what I need to do is flexible, and I can immediately respond to what I see happening in the kitchen with the right action. But when I cook from a recipe, I have to repeatedly consult the recipe and check what step I am in and what to do next. In the recipe case, I have to maintain a mental representation of the state of the food, and can therefore not afford distraction, but if I cook without recipe the world is my mental state.

Training in terms of learning procedures is very common. I carried out some studies with airline pilots who need to program their onboard computer, and are also trained in doing this by memorizing procedures. The airplane computer is a masterpiece of user unfriendliness, so some pilots despair in using the thing, and training on it is sometimes compared to "drinking from a firehose". The main reason why pilots have so much trouble is that they do not understand what they are doing. They are just carrying out procedures (did someone say "Chinese Room"?).

Our lives are full of procedures: look at any electronics manual, medical practice, pilot training, probably Apple Genius training, etc. But opaque procedures turn us into automatons, and blunt our capacity for creative problem solving. And that while it is not so hard to change things. In our pilot example, just adding an explanation of what each step did increased performance on the task tremendously, and also allowed subjects in the experiment to come up with new procedures for problems they were not trained on.

So what makes a true Genius? If I only knew... But improving training and instruction may be a way to make us all a bit smarter. And better multitaskers too.

Monday, November 14, 2011

Reading a book is also multitasking

Reading a book is often considered as the epitome of monotasking. Indeed, it is the activity that is considered by some as the endangered species in the multitasking world. Well, bad news for you nostalgics: reading a book is also multitasking.
Why? This only became apparent to me in the hub of human multitasking, the New York City Subway. People do a lot of reading on the subway, but in New York the majority uses iPads and other eReaders. When Steffi and I were riding the subway, I noticed a men who exhibited some odd behavior while perusing his iPad. His thumb seemed to have a strange twitch, almost a tremor, and hovered over a corner of his iPad going back and forth. Upon closer inspection it became clear to me that the man was not suffering from a strange neurological disorder, but was caught up in a twist of multitasking conflict. He was reading a book on his iPad, and his thumb was above the corner that turns the page.

Now, think about how you read books (at least it is how I read books). If your gaze approaches the end of the page, say the last few lines before the end, you already lift up the corner of the page in order to be able to turn it right at the moment you read the last word. If you wait with picking up the page until you have read the last word you will lose time and might even disturb the flow of whatever you are reading. So, what you are in fact doing is engage in multitasking. Picking up the page before reading the end of the page is not part of the standard reading process. It is not as if you always pick up the page, say, exactly three lines before the end. Instead there is a parallel monitoring process that itches to pick up the page as you get closer to the end. You don't do it with a newspaper, and when you read a text online you do something different (scrolling).

But now enter the iBook reader on the iPad. It looks very much like a real book, including a stack of pages below the current page you are reading. If you touch the corner of a page, it will go to the next. It is almost like a book, but not exactly. If you act exactly as you would act with a real book, you turn the page prematurely (happens to me all the time).
Now, our poor multitasker was probably halfway learning to use the iBook reader, and the tremor in his hand was produced by the normal book-reading process willing him to touch the corner, and the new iBook reader override trying to convince him not to. Because he was so engaged in the reading but maybe also monitoring the subway ride, he probably didn't even notice it himself.

This small example demonstrates that many of our behaviors are composed of several smaller task that we execute in parallel. When we read a text we have a text-processing process, but also a process that makes sure our eyes are fed words at the right pace. And even though text-processing is usually the same, the word-feeding process may differ depending on the circumstances.

Wednesday, November 2, 2011

The benefits of multitasking

The discussion around multitasking usually revolves around the question whether multitasking is bad, with the most extreme position that people just cannot multitask. There are, however, several studies that show that multitasking can sometimes improve your performance. In other words, your performance on task A is better when you combine it with a task B than when you do task A alone!

Doodling for concentration
One example is a study that shows that doodling during meetings improves your recall of what has been said during the meeting. Apparently, it is easier to keep concentrated on boring stuff if you do something, well, boring on the side. For this reason I tell my students that making notes during class is a good idea, even if they never look at those notes again (and I pray that my lectures are not as boring as the doodle meetings). However, in such practical studies it is hard to investigate in detail what is going on.

Attentional Blink
To drill down to the details we did a more controlled laboratory experiment that shows the benefits of multitasking in some situations. The experimental paradigm is called Attentional Blink. In these experiments, subjects see a rapid stream of character, most of which are digits, but up to two characters are letters. Subjects are asked to identify and report the letters, and ignore the digits. You can try out the task yourself here.

In a normal attentional blink trial there are two letters in the stream. Due to the speed of presentation, accuracy is never at 100%, but often around 90%. However, the interesting result in an Attentional Blink experiment is that if the two letters are between 200 and 500 ms apart, the second letter often missed much more often. So while correctness on the first letter may be 90%, it is typically only 50% for the second. Interestingly enough, if either the letters are much closer (say, 100 ms), or much further apart, the second letter is reported correctly as often as the first letter (again, around 90%).

The Attentional Blink effect is interesting itself, and many researchers have been thinking about explanations and models. Additional experimentation showed that in some cases people may improve their performance on the task if you tell them not to try to hard, if they hear music in the background, if there is a starfield in the background, etc. So why is that?

To get a better grasp on this issue, some colleagues and I did a new experiment in which we gave people a second task next to just watching for letters. So we added another task to a task that was in itself already quite hard! The additional task was to track a gray dot that circled around the characters that were presented in the center of the screen. In some trials this gray dot momentarily turned red, and subjects were asked to report this in addition to the letters they saw.
And indeed, the results show that people show less "blink" if they are given the extra task.

Why is multitasking sometimes better?
The advantage of a laboratory task is that we can study what happens in detail. In order to increase our understanding, we built a computer model that mimics human behavior on the task. This model could explain why the Blink happens in the first place: once you have seen the first letter, you try to consolidate this in your memory, and you temporarily block additional letters from cluttering your memory process. In other words: you are too focused on the first letter and therefore miss the second.
If we now add a secondary task, we are slightly disrupting this focus. As a consequence, you will not try to remember the first letter too hard, and therefore also process and remember the second.

Back to the real world
Are you still with me reader? The bottom line is that sometimes we do better on tasks if we are a little distracted by a secondary task. Sometimes our automated processes do a better job than when we focus too much. This can be an issue in sport, for example. If an athlete doesn't trust his or her training, then too much thinking can ruin the effort. Another example: I have a hard time swallowing pills. My trick is to distract myself a bit while trying to swallow, for example by reading the package. I before I know it the pills are gone!

Wednesday, October 26, 2011

Powerful medicine against distraction

If you have followed my advise from the previous post, and have quit Farmville, you are probably still faced with distractions you just cannot erase from your life (you can quit smoking, but you can't quit eating).
David Neal recently gave a talk in Groningen in which he asked the audience to choose between two "self-help" books, one titled "Take control of your life!", and the other "Modify your environment to improve your life!" His prediction was that people typically prefer the former over the latter, because they overestimate the extent to which they can control their habits.

When we are faced with a potentially distracting multitasking situation, we have to ask ourselves the same question: are we going to be mentally strong, or are we going to modify our environment to avoid distraction?
Sometimes this choice is obvious. For example, the Dutch railways have special "Silence" compartments, so if I want get work done on the train I prefer them over compartments where everyone is chatting. Why make that choice? Because I know that talking by others will distract me in an unpleasant way, and there is no way I can overcome this by force of will.

But the situation is different when working on a computer, and temptations for distraction are attractive, like email, Facebook, Twitter, Farmville, et al.? Here typical advice focusses on mental discipline: only do your email once a day, focus on your task, etc. And even though this may help some, an alternative is to make sure we cannot be tempted at all. There are several solutions available for this. A mild medicine is a program called Focusbar. You type into focusbar what you are supposed to be doing, and it will periodically pop up to remind you of what it is you are supposed to do. So while you are reading the status of your friends, it will remind you that what you are really supposed to do is work on your thesis. As if you didn't know that. Now, some research suggests that people interact with their computers as if they are people (see Clifford Nass' The man who lied to his laptop), and might be intimidated to follow stern advise by a machine, but it doesn't work for me.

No, my favorite program is called Freedom. Freedom's brilliance is its simplicity. You start it up, and it will ask you for a number of minutes. It will then block your access to the internet for that amount of time. No cheating by killing the application, only rebooting your computer works (and I am not even sure that that really works). To go back to David Neal's self-help choices: while Focusbar appeals to you to take control yourself, Freedom modifies your environment by bluntly blocking you from whatever distraction. Of course you can still cheat if you have a Smartphone or iPad lying around, but they are still more than just a click away, and if you keep them out of sight you might be safe.

Sometimes you do need Internet for your productive work (like writing a blog), and for that purpose there are more sophisticated programs around, like Concentrate. Concentrate lets you specify which sites you are allowed to go to, and which not, and also blocks applications for you that are not part of your task. My former graduate student Jelmer Borst is an enthusiastic user, but I prefer the simplicity of Freedom.

What I find surprising is how efficient it is to change your environment instead of relying on the force of will. There are interesting analogs, apart from the train, for example in controlling bad food habits. Instead of having to be strong a hundred times to stay away from the cookie jar each time the temptation rises, you only have to be strong once in the supermarket if you decide to not buy them.

Friday, October 21, 2011

How to improve your productivity? Dump Farmville!

Is our brain addicted to internet distractions? Some people think human intelligence is going downhill because we can no longer focus on the things that are important too us. For example, Nicholas Carr claims that the internet is permanently rewiring our brains, gradually transforming us into internet junkies who can no longer get anything done.
Now, you may not fully agree with Carr's doomsday scenario, but it is nevertheless undeniable that many people feel their productivity is hurt by unproductive interruptions and distractions. Indeed, studies have shown that people are interrupted quite often (depending on how you count every 20 minutes or so), and that half of these interruptions is self-initiated. The devious thing is that after the interruption is over, people often don't go back to their main task, but switch to other minor interrupting tasks.

We probably all know the drill: you are working on a paper, but you see that your email inbox flag goes up. So, we are tempted away from our paper (we deserve a little break, right?), and check our email (email needs to be checked, right?). But instead of going back to writing, we post a tweet, check Facebook, check the news, until we feel guilty enough to go back to writing. At that point we have lost our mental context, and need to invest time to get back into it.
And before you know it the day is over and all you did was write half a page.

So what are all these devious distracting tasks? They are things that "need" periodic checking. Interruptions and distractions can be functional, and being flexible in attending these items can be productive. Answering emails is definitely productive. Checking the news? Maybe. Twitter and Facebook? Dubious. But there is one kind of task that need periodic checking that is utterly worthless. I'm referring to games that require periodic maintenance.
I have nothing against games, and do in fact like them a lot, but games are for leisure. They are not work. Periodic maintenance games don't respect the border between work and leisure.

So take Farmville, probably the most successful example. In Farmville you have to maintain crops, but it is not a game you can play continuously. You tend your crops, do some weeding and sowing, and then you have to wait, and return to the game later for some harvesting. If you wait too long, your crops will die and your score will diminish. In other words, this game is nestling itself among the other lingering goals in your mind, and will intrude each time your mind is looking for an excuse to interrupt itself from the important things you are doing. But Farmville and its ilk can't even claim the excuse of usefulness, as opposed to email and news. And it gets even worse! Because they are "social" games, you are supposed to bug your friends into participating in the game as well. So if you don't remind yourself to tend your crops, your friends will, so there is even social pressure to do your virtual chores.

To conclude, if there is one way to improve your productivity without any costs at all, quit Farmville! Burn down the virtual house and let the zero-calory crops die! It will clean your mind.

Monday, October 17, 2011

Driver distraction–one more attempt

The prototypical multitasking topic (apart from gender) is driver distraction. This morning the Dutch police and traffic safety organization launched a new campaign to warn drivers against multitasking while driving. Their motto is "Een beetje chauffeur laat zich niet afleiden" (which means something like: "If you are a cool driver you don't let yourself be distracted"). One of the goals is to try to convince people that even though they might think they are better at multitasking than others, they probably aren't.
Although I hope they have some success with the campaign, I am not very optimistic.

Problem 1: Multitasking is not just eyes
Even though using a cell phone while driving is illegal in the Netherlands, calling hands-free isn't. This is also what the campaign focusses on: your eyes have to be on the road. However, research has shown that driving with a hands-free set is as distracting as with a normal phone. This should not surprise us too much: making the call itself (i.e., dialing) is a much shorter task than having a phone conversation. But it shows that even though the driver may look at the road and have his hands on the wheel, he or she can still be distracted due to conflicting demands on working memory ("You want me to buy apple sauce? Ehrr.., but now I forgot what the maximum speed was...") The focus on the idea that is the eyes and hands that are responsible for multitasking trouble is particularly worrisome, because more and more technology tries to alleviate that aspect of multitasking–while ignoring the others. In particular, I have seen several mentions of the new iPhone 4S's Siri feature (you can talk to your phone and it talks back) as the perfect way to use it while driving.

Problem 2: People can multitask, so don't pretend they can't
What is the mantra of people that try to teach us to better our lives? People cannot multitask, they are built to only carry out one task at a time (David Peebles sent me this link in which some talking head is reiterating this as if it were truth). If that were true there wouldn't be a problem! But on the contrary, people can multitask quite well in certain situations, but not in others. My first post on "het nieuwe pinnen" shows an example of how a small modification in situation can change good multitasking into bad multitasking. And that's the trouble with traffic: there are many situations that allow for multitasking, but that can change rapidly into situations in which multitasking suddenly becomes very dangerous. The cognitive demands of driving vary widely in time, making reliable multitasking impossible. But because of the frequent low-demand periods, the temptation to multitask remains (along with the mistaken conviction that there is no harm in it).

I am not sure what the right way is to prevent people from multitasking in the car, but if we focus on the wrong issues we will definitely not crack the problem.

Sunday, October 16, 2011

Innovation thwarts multitasking

The way we pay for things has improved tremendously over the last couple of decades. As a student, I still had to go to the bank to retrieve money. To get that money I needed special forms that the bank sent me on a semi-regular basis, but that would sometimes run out. Now we just need a debit card that can be used everywhere and worldwide.
Unfortunately, the latest development has made paying slightly more complicated, because designers have not payed attention to the human capacity and need for multitasking.
Just as a cultural sidenote: in the Netherlands, you are yourself responsible for bagging the groceries in the supermarket. As a consequence, you have two more-or-less parallel tasks at the cashier: you have to take care of the payment while at the same time bagging your groceries. One of the central ideas around multitasking that I will present in this blog more often is that problems in multitasking occur when we need two cognitive resources at the same time.
In the grocery case the contested resources is not mental, but it is your hands: bagging groceries typically requires two hands, while paying typically does as well.
In the "old" system, there was a good solution on how to allocate the "hands" resource:
  1. Take the debit card out of your wallet swipe it through the reader, and but it back in your wallet. Now but your wallet back in your pocket.
  2. Enter your pin number
  3. Now start bagging your groceries. If you keep up with the cashier, you have bagged your last item just after the cashier announces the total.
  4. The total will appear on the reader, and all you have to do is press the "yes" key.
  5. And that's it!
But now enter the new system. The new system no longer uses the magnetic strip, but a chip on the card. This is all more secure, but it is also technology and not usability. So bring out the trumpets, it is time for "het nieuwe pinnen"!
  1. You again bring out your card, and put it in the reader. But, now the label warns you "Don't take out the card until the transaction is finished!". So you leave it, but are now stuck with your wallet in your hands.
  2. You would like to bag your groceries now, but you first have to put your wallet away in order to free your hands.
  3. Hopefully you can now keep up with the cashier in bagging your groceries.
  4. Once this is done, the cashier announces the total.
  5. You now have to enter your pin, check the amount and press ok. You now have to wait until your transaction is approved.
  6. Only after the approval you can take out your card. But hey, where's my wallet? Hopefully it is in your pocket, and not at the bottom of your grocery bag.
Now, the disadvantage of this additional hassle may be minor compared to payment 20 years back at a time when people still use checks to pay groceries (are you listening US?), but still I wouldn't be surprised if it takes 5-10 seconds extra per customer. And although this may not seem much, it adds up: an additional cashier in a larger supermarket, or longer lines leading to more pointless waiting.

This example is a nice illustration of how multitasking can work well, and how it can go wrong, even though the situation can only be different in a small detail. In the old system the cashier and the customer could carry out their parts in parallel, while in the new system they have to wait for each other. The brain works in a similar way: if multiple tasks are lined up well, it can carry them out efficiently, but if they are not, delays and mistakes are the result.