I see a dangerous trend here: thinking that data is the only thing we need and that having large data at our hands will solve some problems. But, data per se has no real value if it is not related to problems and people! So, I see many new interesting web sites (and tools!) popping up on the Internet but I don't see any guidance about what to do with them.]]> In realistic settings, where visualization and data analysis tools are really needed, people is not enthusiastic about data but about the impact data can have in their context once it is analyzed. An this context is one rich of background knowledge and business goals (in its broadest meaning). I don't see the same kind of richness around.

Tasks and people are the scarce resource

Unfortunately, while today data can be readily collected and transferred, problems and people cannot. If we don't realize this, we risk to run millions of useless studies, build thousands of useless tools, and waste enormous amounts of energy and resources.

If at least tasks would be joined to data, the situation would be largely improved. Instead of guessing what these data are for, we could try to solve some real problems connected to them. Take for instance the data.gov website, what if people could post interesting questions? Or what if another website would be available where people post data AND tasks?

Interesting good examples

Take the KDD Cup. It is organized every year at KDD, the premier international conference on knowledge discovery. Real world data is published to let researchers compete on a series of pre-defined TASKS. The last KDD 2009 is an excellent example. After few lines of description the web page has a large title: "Task Description". The competition is based on a real data set provided by Orange, the French telecommunications company, and the goal is to do better than a data mining tool developed internally. The tasks is to: "estimate the churn, appetency and up-selling probability of customers" ... "churn rate is a measure of the number of individuals or items moving into or out of a collection over a specific period of time" ... "the appetency is the propensity to buy a service or a product." ... "Up-selling can imply selling something additional, or selling something that is more profitable or otherwise preferable for the seller instead of the original sale. " Better than just data, isn't it?

The VAST Challenge is another excellent example. It is organized every year within the Symposium on Visual Analytics Science and Technology. Similarly to the KDD Cup, a new data set with tasks is published every year. The problems are selected in a way that visual analytics technology is needed to solve them, that is, plain automatic methods without iterative user sense making, are unlikely to be the solution. Another great thing about this challenge is that data is synthetically generated so that ground truth is available. In practice, this means that different solutions can be compared in terms of their ability to discover what there is to be discovered. So, the VAST Challenge provides complex data AND tasks. But even better they also provide people! Since the 2007 edition the contest includes also a session where contest winners have the opportunity to run a small contest live together with real analysts. This is exactly the direction to take in my humble opinion.

Conclusion

I think it is important to recognize this limits and opportunities and start behaving accordingly. It is for this reason that I am not overly enthusiastic about having a lot of data. And I think the the sooner we start differentiating between just data and data + problems the better will be for all of us.]]> tag:diuf.unifr.ch,2009:/people/bertinie/visuale//1.70 2009-06-22T10:56:00Z 2009-06-22T20:24:17Z

I am sorry guys, I feel a strong need to share my frustration with you today. I have discovered yet another infovis library to create the most beautiful visualizations in the world and instead of being excited I am depressed. That's great I really champion the effort of these good guys but a tough question keep hammering in my head: why so many libraries and so few tools? Libraries are great and really needed to speed up the development process but here I perceive a dangerous trend: there are a lot more libraries than real tools written with them!]]> There are a lot of people out there waiting for our useful tools to come and I think it is time we realize that developing real tools for real people is more important than writing toolkits. Personally, I am totally ready to accept a world with no toolkits and lots of tools.

I give a look around on the web and I cannot find a decent visualization tool freely available, only few expensive highly technical commercial tools. As Stephen Few pointed out in his talk at InfoVis a couple of years ago, there is a whole bunch of casual users out there whose job includes the need to analyze data. So, what are we waiting for? Who is expected to build these tools?

I am worried by this short sighted view and this auto-referential culture where infovis people build things for other infovis people, that's it. We develop libraries and then set up fancy examples to show to ourselves and our peers how good we are. Ok, this is useful and needed to some extent. It helps building a community, sharing knowledge and to consolidate good practices. But if we want to go to the next level and let infovis go beyond the toy tool stage, we have to go one step further and embrace the much riskier and tough question: who will use it?

I see so few examples around that I'm kind of embarrassed to talk about it. Can you list any serious and freely available tool that an average user could use in his or her daily activity? Do we maybe have something that minimally resembles a free Spotfire? We have a myriad or little toy vis scattered around on the web and nothing in our hands.

There are very rare exceptions. Robert Kosara has recently published its Parallel Sets in his EagerEyes and plans to keep the burden of maintaining it over the next follow up versions. This is a great thing. Parallel Set would not solve the analytic problems of the entire world but it is a step towards this direction. Therefore bravo Robert!

Another tool I've seen around recently is Verifiable. A very nice and well done tool to create charts directly on the web. Nothing really revolutionary, but what it does it does it well and with an extremely clear interface.

These have the shape of tools made for end users and this is what we need. C'mon folks, libraries are great but we need to show yet what we are able to do to the entire world. Let's develop tools, tools, tools!!!]]> tag:diuf.unifr.ch,2009:/people/bertinie/visuale//1.72 2009-06-16T07:19:42Z 2009-06-16T10:47:25Z

Shame on me, I didn't keep my word on reporting from EuroVis. Anyway here is a very small selection of the remarkable things I have seen during the conference. Pat Hanrahan's talk was really deep and thoughtful. A lot of...

Enrico Bertini Pat Hanrahan's talk was really deep and thoughtful. A lot of new basic material to think about visualization under a new lens. Then I forced myself to select only 3 paper out of the long program. They are engaging and new in some sense. Obviously this is totally personal. And there were many other good ones worth reading.]]>

Pat Hanrahan's Keynote

As I expected the talk was really great, but in a very special way that I couldn't imagine. The talk was titled "Systems of Thought" and was totally based on a deep retrospective of the basic scientific notions underlying visualization.

Hanrahan started his talk putting visual thinking in the context of various systems of thoughts, that is, the tools we humans have to reason and communicate, like: natural language, logic, mathematics, etc. It was kind of shocking to see how in this context visual thinking seems less developed and weaker than the other systems.

Then the talk went on by discussing when and how visualization is better than other forms of thinking, introducing basic studies from people like Herb Simon and Donald Norman, and demonstrating that a change in representation can greatly affect the way we perceive a problem and our ability to solve it.

The talk ended with a series of key questions suggested as a way to proceed when designing a new visualization:

1. What is the problem you are trying to solve?
2. How do you think about the problem? What are the semantic objects and their relationships?
3. What visual representations are already used? How does the visualization represent those objects and support reasoning about them?
4. How can the manipulation of the representation be embodied in the interaction?
5. How can visualization be coupled with other systems of thought?

They are meant to be asked in the given order and interestingly they offer a viable plan on how to proceed in visualization design. Asking about the problem first is really fundamental in my opinion because we still have too many solutions in search of problems around.

Thinking about the semantic objects in the domain of the intended users is also a key point, as well as the related need to see how currently people solve the problem.

The manipulation of the representation puts interaction at the center of the problem and give it its due role. Visualization is not only visual representation but also the great power of visual manipulation.

Last, coupling visualization with other systems of thought is really a great advice. When designing visualization there is always the risk to focus too much on the tools and overlook the whole system, tasks, and processes happening off of the screen. I'm not sure if this is what Pat intended but I think that reasoning on how visualization can be coupled with other system is in part a way to take a broader look and include the whole ecology of reasoning artifacts found around a problem.

If you want to know more I suggest to give a look the the slides (pdf). Just collecting the cited references and reading those paper would be an excellent exercise to deepen our thoughts around visualization.

Papers

This is a very short and personal selection of the papers I really liked. Visualisation of Sensor Data from Animal Movement (pdf link), by: Edward Grundy, Mark W. Jones, Robert S. Laramee, Rory P. Wilson and Emily L.C. Shepard - This also won the best paper award. These guys have developed a system to analyze the movement of animals around the world taking data from accelerometer sensors. The quality of the final images is surprising. Lots of interesting patterns can be seen and it is not hard to imagine how useful it is for researchers using these data.

Visualization of Vessel Movements (pdf link), by: Niels Willems, Huub van de Wetering, Jarke J. van Wijk - I have to admit it, I like this paper for the beauty of the images. GPS data from vessels is collected and used to create a sort of map of water highways. A clever rendering technique is used to represent both high level overview patterns and fine details of specific routes.

The Chinese Room: Visualization and Interaction to Understand and Correct Ambiguous Machine Translation (pdf link), by: Joshua Albrecht, Rebecca Hwa, G. Elisabeta Marai - I included this paper because it is a very good example of integration between machine intelligence and human intelligence. The system uses the output of an automatic Chinese to English translator to visualize the structure of the result. The user can better understand this structure visually and rearrange the items on the screen to create better results.

Final Thoughts

Overall EuroVis was really a great event, with a European touch that I like. The best thing about it is that it is small enough to have time to meet nice people, be engaged in interesting discussions and have some fun. The overall quality of the papers was quite good and I have the impression it is becoming better and better. So I'm looking forward to the next EuroVis in Bordeaux: French atmosphere and lots of wine!]]>

"Now You See It does for data analysis what Stephen Few's book Show Me the Numbers does for data presentation: it teaches simple, fundamental, practical techniques that anyone can use--only this time they're for making sense of information, not presenting it."

I was talking with Ilya, a new PhD student in our department, the other day and in front of a prototype he developed he said something like: "oh yes, and I should find the right color mapping here but ... how?" Oh well ... good question! Originally I wanted to write a whole new post on it but after some reasonings I came to the conclusion that not only it is a daunting task but also and more importantly I don't know enough to seriously teach about it.

But wait a minute, does it mean I cannot help him and the ever increasing pool of poor color choosers? No, there is one thing I can do at least: share my list of favorite sources of information on color. And maybe add some tips and rules of thumb I often use for myself.

So, no more excuses to use poor color schemes. Here is my annotated list of resources, plus some personal tips.

Color Use Guidelines for Data Representation. Brewer, C. A.,Proceedings of the Section on Statistical Graphics, American Statistical Association, Alexandria VA. pp. 55-60 (1999).
[ If you can read only one, read this ]
If you don't have time to read and you need one single source for practical advice stop here. This is the best and conciser explanation about how to use color in visualization you'll ever find. Cynthia Brewer is a cartographer and focused much of her work on color in geographical data but her suggestions apply broadly to any kind of data. You may see the result of her work in Color Brewer, an on line tool to learn how to select color scales. The tool alone is an eye-opener for those who don't know anything about the topic.

How NOT to lie with visualization. BE Rogowitz, LA Treinish, S Bryson, Computers in Physics (1996).
[ More into color for SciVis but still very useful and great examples ]
This is another classic, quite short and easy to read. I like it especially for its focus on how harmful color can be if not used properly. The use of color is discussed more in the context of scientific visualization where continuous shades of color are often the case, like in medical images and geographical mapping, but the results can be applied to any other visualization. It is especially interesting the notion that different color mapping strategies should/can be used according to the task at hand (e.g., segmentation, highlight, etc.).

Designing pixel-oriented visualization techniques: Theory and applications. DA Keim, IEEE Transactions on Visualization and Computer Graphics (2000).
[ Discussion (and code) of a "perceptually" optimal color scale ]
Though this is not only about color, the paper contains a very useful section on color and on how to build a perceptually optimal color scale. The color scale is called HSI (Hue, Saturation, Intensity) and is a variation over the most common RGB, HSB, etc. The very good point about it is that it is a very rare example of article where both color theory and practical implementations are discussed in the same place. The HSI color scale can be easily re-implemented by following the code they provide in a related paper: Issues in visualizing large databases. DA Keim, HP Kriegel - Proc. Conf. on Visual Database Systems, VDB'95 (1995).

Color Scales for Image Data. H. Levkowitz, G. T. Herman, IEEE Computer Graphics and Applications (12):1 pp.72 - 80 (1992).
[ Some relevant psychophysics theory and its relevance in color scale design ]
This is a purely theoretical paper. I included it because it contains some information that is difficult to find elsewhere. And also because I find it especially intriguing. Here we learn that (1) not all differences in color intensity are perceived by our eyes and (2) that a linear increase in color intensity is not necessarily perceived linearly. The concept of Just Noticeable Difference (JND) is introduced and applied to color scale design. One practical consequence is that it doesn't matter how well we map our data to color, some differences will always be lost.

Choosing Effective Colours for Data Visualization. Healey, C. G., Proceedings IEEE Visualization '96, pp. 263-270 (1996).
[ Not easy read, hard-core experimentation, but unique info on categorical colors ]
This is even more theoretical than the paper above. And be warned, it is not an easy read! Anyway, I put it in the list because it is the only "serious" reference I know where the selection of categorical colors, that is, colors that represents categories and not quantity, is discussed in fine details and an algorithm for their selection is discussed. Here we learn that color is not as powerful as we may think. The maximum number of distinguishable colors we can use to label data is around 12. Not so many indeed!

Book Chapters

Information visualization: perception for design (Chapter 4: Color) by Colin Ware.
Colin Ware's book is simply the best resource for whatever concerns perception theory applied to visualization. Admittedly, this is probably the best book on visualization ever. Chapter 4 is all about color theory and its content is obviously great. Theory and practice are well balanced and useful examples are illustrated throughout the chapter. I think it only missed practical advices and how to implement the suggestions in practice, but ok, maybe this would be out of the scope of the book.

Envisioning Information (Chapter 5: Color and Information) by Edward Tufte.
I don't think this book needs any introduction. It is part of the famous Tufte's trilogy and of course it contains some indications on color use. Even if here one can find many of the things discussed in other books and papers, but in a useful summarized version, it also contains some unique content in the usual original Tufte's style. A great piece of knowledge here is given right away as the chapter opens. Tufte summarizes color uses in information design as: to label, to measure, to represent or imitate reality and to enliven or decorate. These few tasks provide a useful framework around the work of a visualization designer.

Show Me the Numbers (Chapter 6: Visual perception and quantitative communication) by Stephen Few.
This chapter written by Stephen Few is the best summary I have ever seen on visual perception theory applied to visualization. Here you will find not only how to use color effectively but also how to boil down basic theory on how human vision works to few simple rules to apply in visual design. In a way it can be considered a sort of Colin Ware's book compressed in one pill. So again, if you don't have enough time to read, pick this one and study this chapter. You won't regret your choice.

Tips and rules of thumb

Finally I try to put something myself. This is just a random list of rules I learned the hard way by doing.

• Don't overestimate the power of color - Color is attractive and powerful and let's admit it, it is what makes most of our visualizations pretty and nice to see. But for any serious use it is important to realize how limited it is. The number of colors we can easily distinguish is incredibly low (this you can learn it from the refs above). For instance, it is estimated that the maximum number of categorical colors we can easily detect in a representation is around 12. Similar figures holds when presenting continuous data. Compared to other data features like position, length, size, it is visually perceived less efficiently. So just don't believe color mapping will do wonders, it is useful within its bounds.
  
  
• Always provide a color legend - I think this one goes in the list of the most common mistakes in visualization: some data feature is represented with color but then there's nothing in the interface that tells you what this color represents. A color legend is alway needed and not only for labeling. As an example, when it represents quantitative data it must also tell us to what numbers the brightest and darkest colors map to. So in short, please do your home work, provide a legend.
  
  
• Use color with extreme care and parsimony (above all do no harm!) - This is a sort of repetition of the first point but from a different angle. As color is added to an interface it soon becomes noise. Learn to use it with extreme care and parsimony. It is important for instance to realize that if color is used to represent a data feature it is extremely hard to use it for some other elements in the interface. In the end it is extremely important what Tufte says: "above all do no harm".
  
  
• Learn to love grays and gray scales (grids!) - The best use one can find of color is to understand how powerful colorless graphics are. In particular shades of grays are so useful in data representation that I am surprised there are so few, if any, specialists advocating for their use (Tufte mentions it by the way). Give a look around, pick the best known and best crafted tools and you'll see that most of the times their design is based on shades of gray. Gray is especially useful in segmenting the visualization space and organizing it in spaces. The most obvious example is the use of grids in charts and alternated rows in tables (Stephen Few shows excellent examples in Show Me the Numbers) but the same principle applies to thousands of other visualization components. So in short: learn to love gray and gray scales, they can do wonders and rarely do harm.
  
  
• Don't represent unordered data with ordered colors - This is self-explanatory but I see it so often that I think it's worth to add it. Also, I think not everybody would agree with me on that. Some people use different intensities of the same "hue" to represent categories. In my opinion this is poor use of color and opens the door to false interpretations. Ordered colors are automatically coded as "there's some ordered here" by our brain. Why do we want to fool our mind when there are better solutions? Use distinguishable hues and, if possible, make them of the same intensity. This will work best.
  
  
• Keep an eye to skewed distributions - Personally I always find this problem in my data visualizations and I am surprised it is not discussed more. When the dimension you map to color has a skewed distribution the result is incredibly poor: there are few items represented by the highest intensity and all the others flattened to the lower. In short, there's nothing really useful to see apart the fact that there are two or three items with very high values. In this case one option is to adopt a not linear mapping between data feature and color. Common solutions are logarithmic or square root functions that alleviate the problem and permit to reproduce a full progression of values.

Here was my list and .... oh before I forget there is one last major one!

• Don't use the (infamous) rainbow color scale - Maybe someone would laugh at this advice as something too obvious but then, thanks to Ilya I discovered that there is nothing to laugh about. If you are not convinced see this study on the uses of the rainbow color scale and discover how many professionals and researchers still believe it has some value:Rainbow Color Map (Still) Considered Harmful

Conclusion

A bit of self promotion here on Visuale today!

It's my pleasure to introduce to you one recent work of ours: The Extended Excentric Labeling. It is an extension to the original interactive labeling technique called Excentric Labeling, which was developed by Jean Daniel Fekete and Catherine Plaisant in 1998 at the University of Maryland. We extended it to solve some problems present in the original version and it will be presented and published at the next EuroVis 2009 conference in Berlin, next June.

Here is the EuroVis'09 Paper (pdf) and a Video (mp4) we produced to showcase the technique.

"The paper presents an extension to the Excentric Labeling, a labeling technique to dynamically show labels around a movable lens. Each labels refers to one object within the lens and is connected to it through a line. The original implementation has several known limitations and potential improvements that we address in this work, like: high density areas, uneven density distributions, and summary statistics. We describe the implemented extensions and present a think-aloud user study. The study shows that users can naturally understand and easily operate the majority of the implemented function but label scrolling, which requires additional research. From the study we also gained unanticipated requirements and interesting directions for further research."

The Motivation

The main motivation behind this technique and its relative study was a practical problem we encountered in the development of a new visualization. EL was just great and fit our need of understanding the content of screen regions. But the real problem was to have something flexible enough to deal with very sparse and very dense areas at the same time. In dense areas the original EL provided only a simple sampling mechanism that did not really help to interpret the data. Them while implementing a solution we discovered we could add some other useful and interesting features.

The Techniques

With the EEL we introduced the following features:

• Label scrolling: a mechanism to scroll through labels when there are to many of them.
  
  
• Focus area adjustment: an automatic mechanism to let the size of the focus area automatically adapt to the underlying data density.
  
  
• Summary statistics and filtering: a series of glyphs and interactive tools to summarize the content under the focus area and to disambiguate it through filtering.
  
  
• Inheritance of visual features: a visual mapping mechanism to let the labels inherit the visual/data features from their connected items.
  
  
• Layout and sorting: algorithms and techniques to permit effective positioning of labels and of their links to the items.
  

It's been almost by chance that I stumbled upon the old Parallel Coordinates InfoVis'97 paper "Multidimensional Detective" after my last post. And it's crazy how some information seems to reach you when you start following a new line of thought.

This post and my last one are very much in line with the idea that we do need to invest a lot more on the practice of visualization and not only on design.

Parallel Coordinates have been invented by Alfred Inselberg who is also the author of this paper. What is really impressive in retrospective, after more than ten years, is the need to communicate to people not only what Parallel Coordinates is, but more, and foremost probably, how they can be used.

In this paper Inselberg provides some great guidelines to use when performing visual data analysis. Here I provide some personal comments about each guideline.

Guideline 1 - Do not let the picture intimidate you. This seems to be the trademark of infovis. And indeed it perfectly pairs up with the first part of the famous InfoVis Mantra "overview first". If we accept the idea that infovis is effective when we present an overview first (IMHO this is somewhat questionable but well ... this is not the right place to discuss it), it's evident that we have to teach our analysts to not be intimidated by it. I've seen and designed countless of visualization which are intimidating at first. And often I receive comments like: "can you make it simpler"? Sure, but maybe less informative too? If only we can let the users have the chance to try first and see what's the result!

Guideline 2 - Understand the objectives and use them to obtain visual cues. Oh this is my favorite one! Too often I hear that infovis is for generic data exploration, like if one could approach data without no idea of the ultimate goal. There's nothing in reality like looking at the data for the sake of it. Inselberg acknowledges it and adds a worthy advice: make the statement clear about what you want to obtain form visualization and let the goal guide your analysis. Our perceptual system is designed to "tune up" our senses when we focus our attention on something (see the latest Colin Ware's book for details). So, let's exploit this feature and focus on what we care about.

Guideline 3 - Carefully scrutinize the picture. As a consequence to the previous step, it is necessary to look carefully to the picture and find visual cues that can help move at least little steps towards the prefixed goal. Visual patterns that tell us something are always there, it's only a matter of looking for them carefully. They usually trigger new questions and hypotheses and help us formulate new actions. We look into other segments of the data or manipulate the visualization in a way that helps us find clarifications. That is, a knowledge building loop.

Guideline 4 - Test the assumptions and the "I'm really sure of". Inselberg reminds us that what we get out of a visualization is not only what we see but also what we we already have in our mind, that is, our subjective world, which comprises: background knowledge, assumptions, beliefs, etc. An effective analyst must take this into account and understand how this can affect the analysis. Often the visualization present patterns that generate some skepticism but then it's exactly form these strange data segments that important discoveries stem. On a side note, it's surprising to see how much effort we have put into the analysis of objective perceptual processes in visualization and how few on higher level cognitive processes that involve subjective evaluation.

Guideline 5 - You can't be unlucky all the time.If you are not intimidated, you understand the objective, you scrutinize the picture and test the assumptions, well ... you can't be unlucky all the time! If I understand well, what Inselberg seems to tell us here is that even if a good proportion of the patterns extracted from the visualization can result in little advancement towards the goal or not useful discoveries, you can't be unlucky all the time. In the end there should be something that helps you progress towards your desired direction. This last one, is a very positive and encouraging advice, which I understand because visualization tools often lead to the discovery of trivial or useless information, but then by striving to find something, little gems often sort out of it.

In summary this quite old paper reminds me how hard is to be an analytical visual detective and also how far we are to help our customers focus on fruitful paths. It would be nice to see novel contributions in this direction, because this is in my opinion one of the greatest limits to infovis adoption. In the meantime this small advice, as simple as it seems, looks to me as the most solid I've ever seen.

The book is an easy read, written with a simple style that makes it accessible to everybody, and yet incredibly intriguing and informative for the knowledgeable reader.

Stephen interviewed tens of researchers and entrepreneurs around the US and put into focus one of the major trends of our days: not only an incredible amount of data has been and is collected everyday around the world but we are also finally starting to "use" these data to let us understand relevant aspect of the human being. Health, Finance, Marketing, Policy, are only few examples of areas where data is collected and deeply analyzed everyday.

Content

The book is organized around 7 chapters: Worker, Shopper, Voter, Blogger, Terrorist, Patient, Lover, in which people is modeled under the lens of a specific stereotype.

In Worker we are modeled according to our skills and the way we work. We meet people like Samer Takriti at IBM who is modeling about 300.000 IBM workers to understand the relationship between their skills and their performance and how to better allocate these skills in the company the same way we used to do with any other physical company asset.

In Shopper we are modeled according to the things we buy. Researchers are analyzing the millions of transactions we make everyday in stores to understand what "type" of buyers we are. Raiyd Ghani, for instance, analyzes with his group at Accenture Technology Labs grocery store transactions to provide personalized suggestions to shoppers through the use of carts equipped with personal assistants.

In Voter we are modeled according to ... to what? This is an impressive chapter because it demonstrates that we can be modeled in a given domain indirectly, using data that apparently has no connection with the subject matter. This is what Josh Gotbaum with his political firm Spotlight Analysis does. They provide detailed indications on swing voters based on data taken from large data companies like ChoicePoint and and Acxiom, who collect an incledible amount of data about us on almost every aspect of our life (scary?! :-)).

In Blogger we are modeled according to our opinion. Yes, our opinion. There are companies like Umbria Communications which analyzes the blogosphere to understand the opinion trends of millions of bloggers on whatever interests a given company. If I want to track how people react to a new product put on the market Umbria can tell.

In Terrorist we are modeled as potential terrorists or thieves. Here we meet people like Jeff Jonas, now at IBM, who helped casinos in Las Vegas sift through millions of internal records to single out suspect customers. And the same technology is used by In-Q-Tel, the venture capital arm of the CIA which invested in this technology, to cope with national security and counter terrorism.

In Patient we are modeled according to our body signals and medical records. This is the chapter I most loved, not only for its humanitarian applications, but also for the cleverness of some solutions. Eric Dishman launched the home health division at Intel where they design smart sensors like the "magic carpet" that monitors weight an movements to monitor the health of patients and where they try to predict the onset of diseases like Parkinson's and Alzheimer's by detecting suspect variations in the stream of data.

Finally, in Lover we are modeled according to our profile to find matches among us as potential lovers. We meet Helen Fisher, a Rutger's University anthropologist, who devised an innovative method to find matches between people which is the basis of the Chemistry.com dating website. Her method goes well beyond simple matching of demographic data, it is based on her theory that we can be split in four groups where a specific hormone is predominant and that the best matches comes from complementary hormones.

Reflections

The first issue the book raises is obviously privacy. I really liked the approach of Stephen Baker, equally distant from the excitement for the new opportunities brought by innovation and the potential for a super-controlled society where drawing a full profile of ourselves is becoming worryingly easy. Any other technological shift in history came however with the promise of new advancement in human being together with novel problems (think about cars and pollution). Stephen asks the right questions to some of the researchers he met. The most interesting in terms of privacy is the one with Jeff Jonas who is "vehemently opposed to the use of statistical data mining to predict the next terrorist attack" because of the high risk of intrusion and false alarms. And yet he believes that this technology can both protect our freedom and our privacy at the same time. I think this is one of the biggest challenges of our time, to find the right balance between the opportunities for increased freedom and security and the risks of intrusion, control, and faulty conclusions in the analysis of our own data.

From a more scientific and technological point of view what strikes me is the relevance prediction has in all the application areas described in the book. In traditional data analysis, especially for those with a visualization background, the focus is on "understanding" what is in the data to build a mental model out of it and in "discovering" some special gems out of chaos. Yet, however, real world applications are more concerned with elaborating actionable solutions to run and test, and I have the impression that "prediction" lends itself better to this goal. Think about it, through the book's examples, in workers the company wants to predict performance to put people in the right place, in shoppers the grocery store wants to predict what product can be sold to one specific customer to provide timely suggestions, in voter a political party wants to predict which population segment should be addresses with a targeted message to increase the chances they hit a group of swing voters, and so on. How do we, visual information designers and analysts, cope with this fact? Are we able to provide with our tools the same level of actionable knowledge or are we condemned to just describe things and hope that this information will be useful in some way?

Implications for Visualization

What is the role of visualization in the world of the Numerati. I think it is huge!!!

First of all all the technologies used by the Numerati are to some extent prone to errors and they are always the results of continued refinement of the underlying model. Visualization can play a significant role in helping the modelers understand and test their models and explore their implication as they are applied to new data. Without such a level of interaction the risk is to build monstrous black-boxes that spit oracles we all have to follow without really knowing why.

Another area where I see a large role of visualization is when mining is used in monitoring environments, where the timely detection and comprehension of the situation (more technically known as the situational awareness problem) is important. We have a long and respected tradition of research for knowing what works best in terms of visual representation when visual saliency, detection and contextual information are at stake. Well designed visualizations that permit to get the most out of a screen in a matter of seconds are of paramount importance here, from the need to analyze terrorist attacks to the doctor monitoring a patient.

A third potential I see for visualization is the need for personal data visualization. As these technologies develops, and the results of data analysis become more pervasive, I expect to see and increase in the need of managing personal data and the results of these analyzes by end-users. And how are we going to provide this information to the average person? Visualization can play a big role here and and again it would need to reinvent itself a bit. In this domain extremely simple and useful visualizations will be needed and some of them will be provided on non-standard devices like TVs, cell phones, public displays. We need flexible and simple solutions to provide to the large public.

So, in summary, the explosion of data analysis is good news for us! We have plenty of novel challenges to address. A somewhat silent mind shift is already going on underway ... I expect to see in the future an ever tighter integration of automatic mining technologies and visualization, as the recent Visual Analytics trend demonstrates after all.

DabbleDB

Magic/Replace

Few Reflections

• Data Management & InfoVis - It's really surprising to see how data management has been neglected in InfoVis, and yet it is so central in our daily activity, both as designers as well as users. I strongly believe the lack of simple data management tools in software products is one of the major barriers to the adoption of InfoVis. Tableau is another great example: data is loaded, managed, and manipulated in few intuitive drag and drop operations. It might be frustrating to accept it but a major quality factor of the systems we design does not reside in the fancy visual representations we make but rather in little details that make a difference like these.
• Data Management & Web 2.0 - These two tools are another example of the power of Web 2.0. I have already argued that we can probably already speak of Vis2.0, a series of visualization tools that work smoothly on the web. Web 2.0 data management tools are another piece added to the puzzle. And it's amazing because, as any other 2.0 tools, it is not only a matter of reproducing the same desktop tool on the web, but also to enable a whole new spectrum of possibilities. DabbleDB, for instance, lets people publish and update the DB on the web in a matter of second, thus enabling collaboration in a way that would be impossible to do with Excel or any other desktop product.
• Simple Intelligence in Software - All the simple tricks and smart guesses found in these two products let me think of the idea of embedded simple pieces of intelligence in software products. It's crazy if you think about it: we have been researching super complex AI algorithms for years, to do things of little use for anybody, and now there are some little intelligent and focused tricks that work really great for some people. It also reminds me of the concept of "Appropriate Intelligence" advocated by Alan Dix: "Designing interactions for appropriate intelligence is based on two principles: 1) it should do good things when it works, 2) it shouldn't do bad things when it doesn't". It is perfectly applied here.