Insightful, substantive and a must read for anyone working with data visualisation as consumer or designer. Julie Steele andNoah Iliisky’s new volume — Designing Data Visualisations — from O’Reilly succeeds in applying a long overdue and well crafted taxonomic process to the art of Data Vis. Building on their previous volume — Beautiful Visualisations — the authors take to the underpinnings of the pretty pictures and case studies presented in their edited volume. This shorter work would form a superb basis for an introduction to Data Vis course.

The structure starts with a smooth introduction to the terms that underlie visualisation breaking them down into a lovely typology bridging the discussion between the explanatory versus exploratory uses and techniques. The next section takes the data vis practitioner through the creative process (setting goals, choosing the right visualisation and then building the visualisation itself). This is not a hands on manual and the use of particular tools is left to other specific volumes, although a list o tools and resources is helpfully provided.

Although a concise volume, the wealth of useful information conveyed as both hints ‘use text sparingly — don’t feel obliged top label every tick mark’ to a five page color theory introduction provide hands-on and immediately usable nuggets. The nuggets are often basic — a demonstration of why mixed up lower case letters read easier than upper case letters — but as basic as they are they are great reminders that it’s all about conveying a message in the most effective way possible. Keeping things clear and concise and ensuring that the technology doesn’t get in the way of the message. The concluding checklist of best practices in data visualisation is a concise summary of the material covered in the book.

I would highly recommend this book to any academic researcher or digital humanities scholar and commend it as a superb overview of the practice of data visualisation.

a

The R Cookbook by Paul Teetor is a solid addition to the well respected series. Teetor provides a rich collection of useful examples written in the proven method and covering everything from installing, configuring and running R to carrying out sophisticated statistical analysis tasks that demonstrate the power of R. The book is targeted at a wide audience from R novice eager to just start playing in R to more experienced practitioners looking to hone and round out their R repertoire.

It can be used as an introductory training source for those who like to learn by doing and extrapolating knowledge from examples. It also has the useful ability of function as a reference source when plotting a particular R exercise.
The problem — solution — discussion pattern works well when the problem is clearly and concisely stated as Teetor does. As the book progresses it does move towards more advanced statistical manipulation and analysis, but then if you are using R in the first place then this is a fairly safe assumption. This is one of the more notable cookbook series for the thoroughness of the discussion. The inclusion of philosophical notes, parameter and options sections when necessary and finally the cross-indexing via the more information section set this book apart as a superb reference. In conjunction with the R in a Nutshell which was reviewed earlier, there are indispensable tools for the budding R enthusiast and in conjunction with the freely accessible R reference manuals from the Foundation form the optimal R library.
My only gripe is that there is less focus in this book on the visualisation end of R. That is not to say that there not vis exercises in the book. Simply that it is heavier on the analysis end on the language which is actually well and good as this is crucial to the latter and an area that I for one need the instruction.
This cookbook does not expect readers to arrive with extensive R knowledge and as I mentioned earlier is targeted for a broad audience of R practitioners.

a

The Data Source Handbook by Pete Warden provides a concise and handy guide to some of the main sources of public data accessible on the web today. It’s a very short book of 40 pages. This in itself does not stand against the book. These sources are rapidly changing and compiling and committing an exhaustive survey to a printed volume would damn it to almost instant obsolescence. It would also prevent any treatment of individual datasources in any useful detail.

As it is, Warden is able to pick a select few and identify strengths and available APIs in a useful fashion. He organises the type of sources into logical categories and identifies some key sources for each:

• Websites
• People
• Search terms
• Locations
• Companies
• IP Addresses
• Books, films, movies, music and products

He selects the key open providers of data in these areas and systematically shows how to access the information along with simple programmatic instructions. In a volume of such limited length you would not expect to find extensive instructions or discussion — and you won’t. What you have is a very concise survey identifying the key players and giving a nutshell indication of what you can use the datasources for.
This is a useful and quick reference for anyone routinely accessing, compiling, aggregating or augmenting their own datasets. Although very few of the sources identified would be new to most people in the data analysis space, this does provide a useful compilation and also handy concise reminder of how one might augment a limited dataset quickly in an automated fashion.
This is an easily accessible volume, well organized and with the only major failing that it will be become dated in a published form. However, as an eBook it is ideal and I would recommend it to anyone new to the area of adata visualisation looking for some great sample data to access, or to the more seasoned data traveller looking to keep their familiarity with the wide variety of available data current.

a

Data Analysis with Open Source Tools by Philipp K Janert is a simply superb, solid and exhaustive synthesis of instruction, workshops and hands-on exercises designed for those serious about conducting professional data analysis. This is not a lightweight undertaking. This is a serious get-down-to-it and do-it-right kind of manual. The author (as has been mentioned elsewhere) is passionate about his subject and it shows. He knows how to convey the most complex concepts in an approachable and effective way.

The title of the book suggests a very hands-on approach and suggests possibly that it is more bout applying known techniques utilising specific tools. The book delivers on this through a series of ‘workshops’ that are attached to each chapter. These sections lead you through specific application of specific data analysis tasks using particular packages (primarily Python and R) in a very useful manner. But don’t be deceived, the book is far more than these workshops and to its credit provides extensive grounding in the theory and principles of data analysis itself so as to ground you in the application.

Whether you are approach first attempts at data analysis and feel unsure about the practise, or you have been using simple tools such as excel to carry out your analysis or are looking to hone your techniques by exploring the power of R is moving to presentation of your analytical findings, you will find this volume a superb choice. Spending time working through the workshops will build a firm foundation of capability to extend the theoretical and additionally provides a superb grounding for approaching courses dealing more directly with the tools themselves, such as the volumes on R that I have explored previously.

Janert uses the progression of presenting the data (to find the patterns) –> modelling the data (to explore) –> mining the data computationally (to understand the data) –> and finally applying the data (to actually use it in real world instances) through the book. He uses examples liberally to maintain engagement and stylistically asks the reader questions and makes reminders constantly to keep you moving through what I remind is pretty heavy material. This is not a book to try in one setting, nor however, is it a reference manual. It is really a course that should be approached over a suitable length of time.

One of the questions the author poses early on is ‘what’s with the math?’ and he assures you that if you do find this intimidating its worth the time to familiarise and gain some comfort with them as they are necessary should you really want to carry out effective data analysis. He’s right and the way the book is structured you do need to take this advice on board. This will limit the book to people that are willing to commit if uncomfortable with the concepts. But as I mentioned above this is a serious book and the author seems to have made a commitment himself to deliver the material and asks for a bit of reciprocation. Now, I (possibly less than fondly) recall much of this from the distant haze of undergraduate statistics or mathematical economics, but there is a collection of great aids in the appendicies to the volume to help you out and these are well presented. I really could have used these 25 years ago when I was struggling through these courses.

All in all, this is a very good book. It actually does more than it promises and delivers a comprehensive and effective course in data analysis with superb hands-on exercises to drive home the learning.

a

Beautiful Data is a collection of essays on exploring the organisation, manipulation and display of data in ‘beautiful way’. The editors, Toby Segaran and Jeff Hammerbacher, have attempted to loosely organise the papers into logical process of: collection –> storage –> organisation –> retrieval –> visualisation –> analysis and in theory this works. The challenge as with any collection of papers from such a diverse set of authors (39 in this case) is finding that common thread that flows through the works. In this the editors achieve a passing grade, but frankly, this is sort of the book that offers the reader something they will find useful, but only due to the breadth of articles included. The downside is that there will certainly be articles that a reader will not. The authors seem to realise this and use the term ‘loose’ with some frequency. But I can’t criticise this and would want to. This is a strength of the book. It covers much ground and will appeal to many.

Conceptually, the demand for a book in this area is huge. Having delivered a number of workshops in this area and been asked to adjudicate on conference papers in the past two years, I am certainly aware of breadth, and the demand for skills and knowledge in this broad area.

The first article by Nathan Yau, builds from his popular blog posts on flowingdata.com and provides more depth on two case studies involving the collection, analysis and visualisation of data gathered from going about your own life. He is painting a picture of life to come as more of our life becomes monitored and we are raised to a new level of consciousness of how we live. His article explores how we might internalise the analysis of this data and how it could impact on life activities. This is a flavour of many of the articles in the book. They are on the cutting edge and offer speculative observation of how we are being impacted by emerging technologies and in this collection, you will find great food for thought. If there is any criticism to this it is in that much of this information comes from contributors that share their information via blogs and much seems familiar. If you sense a little trepidation in my review you can feel the hesitation in my fingers as I type. I like the concept and I really like many of the articles. Peter Norvig’s ‘Natural Language Corpus Data’ is particularly well crafted as is Dykes and Wood’s on ‘The Geographic Beauty of a Photographic Archive’. Both of these are targeted at beautiful data in the purest sense, the inner exploration of data as beautiful in itself when craftfully addressed.

This collection is a needed and valued contribution to a popular discussion. The editors have done an admirable job of locating a way to systematically tie the contributions together. The author’s of the specific contributions have also focussed on useful adaptations of theory to actual demonstrable practice. The breadth of the book is extensive and I guess my hesitatcy is just because this breadth is somewhat overwhelming. I would certainly recommend this book to anyone even remotely interested in any of the aspects that the book addressed in the broad field of data management, manipulation and presentation. You are sure to find a few articles of particular interest and possible pique new interest in area you may well not have previously explored. It is a very useful companion to Beautiful Visualisation edited by Steele and Iliinsky, both of whom contribute to this volume.

a

Great R:Level 1 by Michael Milton is a 2 hr video course which leads you from the basics of installing the R environment on your system to conducting basic analysis and presentation of your data. The course itself is delivered in a clear, concise manner and the author is very thorough in his approach. The pacing is very good and as a student you are kept engaged and don’t tend to fall behind, nor feel that you wish that things could hurry along. This video course is a superb complement to written tutorials and reference manuals on the R environment.

The video and voice quality are great and the small tutorial chunks make it very easy to stop and come back. Additionally, the summaries at that start of each chunk build into this and immediately let you pick up where you let off. During the course, small pop-up summaries appear at critical points summarising a concept or providing a clear description of the command being executed. This greatly enables the play along tutorial nature of the course.

The course is aimed at a student interested in discovering the power of R and demands no prior experience with R, statistics or a programming language. This does not mean that the assumption is that the user is absolutely naive and it is clear that some knowledge at different points is of advantage and plays well into the challenges set by the instructor.

I particularly like the teaching style of the author. He asks questions of you, the student, and sets up a series of periodic challenges to ensure that you are engaged and involved in the process…the best attempt to create an an active learning environment as if the tutor was there sitting with you.

The overall scope of the module is appropriate. You are introduced to the environment and carry out a series of tasks that build on your familiarity with R. Milton takes you through a number of real world exercise from start to finish. The file cycle allows for a thorough and grounded understanding.

One of the only small quibbles that I would note is the reliance on the external data source for the initial exercise. Although I was able to get the data from the UNData website, something was amiss with the CSV format and my screen then didn’t match that of the tutorial. In fact what had happened was the data provider added an additional column consequently R seemed to have a little trouble with empty values in that column. This was solved with a quick tinker, but points to a long term issue. I do like that the author decided to use real world data and to highlight the usefulness of R for working with public datasets, so am loathe to be over critical, but the challenge remains.

Much additional knowledge is conveyed both between discussion of types of arrays, and also the noting of smaller tips that the author has found that make working with the R environment more efficient.

I was very impressed with this course and would recommend it for anyone interested in delving into the world of R. In comparison to R in a Nutshell which I had previously reviewed, this course would make a very fine introduction and lead to picking up the further volume to expand on one’s knowledge of R. I would feel a lot more comfortable approaching the book after having gone through this video tutorial. All I can say now is bring on Great R: Level 2!

a

The Map Your Moves Challenge fascinates me. New York’s Public Radio station WNYC has devised a data visualisation challenge for their listeners. Curious about what makes people move from and to their community they polled stories from their listeners and collected them into a structured dataset and have released it into the wild. Now this is very cool…they want to take real stories and understand how these stories interact and how they can learn about their own community from them. Absolutely brilliant!

You can listen to the initial announcement of the project on their streaming player. OK..“Valerie, let’s talk a little about data visualisation…” I am very impressed. Telling a story in a different way is very exciting. I would admit that there is some conflation between data vis and infographics, but I am very, very impressed by the fact that this is coming from a real life curiosity to answer real questions. This is definitely the best advert for ‘doing’ data visualisation that I have found. Interestingly as well, they seem to have an evolving survey form that adapts to the information people.

They have realesed the dataset into the wild at via drop.io and have opened it up to anyone to play with the data and to find new ways to present the stories that it represents. They even have an ‘official’ Census Project Editor. Wow! What I find very cool is the wonderfully engaged way in which this public radio station is asking questions about how their community is evolving. I have been very impressed by the interest of the general population of Ireland whenever the National Archives releases another tranche of census info. The newspapers cover this as a major event and it becomes the fodder for discussion in the pub and over the dinner table. I was very surprised (but maybe I shouldn’t be) that there was this similar engagement globally. The census project site on the wnyc website touts that:

Every 10 years the country “counts heads” and uses those numbers to determine everything from election districts to funding levels. But the story of our neighborhoods, cities, and states is much deeper than what’s in the numbers. Join the Brian Lehrer Show as we make sure our listeners count, from in-depth coverage of the census process to interactive projects and all sorts of stories about who we are and how we live in 2010.

They are interrogating the statistics…counting the numbers…because they represent individuals and they are using technology to reach down and find the stories behind the numbers. I find this amazingly gratifying and have downloaded the great dataset they provided to try my own naive hand at finding some interesting ways to see the stories through the user contributed information.

There is an ironic timing to this discussion. Here I am being so very impressed at a community wanting to understand itself through the sharing of stories, when the Steven Harper government in Canada have decided that census should really be a voluntary sort of thing (kind of defeating the purpose eh?) and that the state has for too long being interested in the constituent members of the national community. They have, with the claimed agreement of Statistics Canada, decided to discontinue the use of the long form which was distributed to 20% of the Canadian population and sought to understand the demography and livelihood of the community. Although it was aggregated and kept anonymous (Stats Can has an unblemished record and zealously protects the privacy of Canadians), this seems to have had little bearing on the decision. The controversy has now resulted in the resignation of the Head of Statistics Canada, Munir Sheikh over the issue. National newspapers have suggested that the national animal be changed from the beaver to the ostrich and that “Opting to know less about ourselves is about as smart as flying without instruments” quips James Travers in the Toronto Star. Perhaps the decision is simply telling about the nature of decision-maming in the the Canadian government. Ontario has represented its deep disagreement with the decision and more importantly reminded the federal government that it does in fact base crtitical welfare, health and public service decisions it makes from knowing about the people it serves — and the crucial data provided by Statistics Canada. A rather telling comment on the Canadian political system. Let’s watch and see how this one plays out. In the meantime, I applaud communities wanting to know more about themselves as a means of improving the lives of those that call a place home.

a

Data visualisation has become very vogue in the digital humanities community. Although there have been a scattering of brave practitioners over the past few years, only very recently has this interdisciplinary area started to feature prominently at DH conferences as a mainstream practise worthy of consideration.

For the last few months I have been looking for an opportunity (i.e. a bit of time) to delve into R and Processing, specifically with an eye towards taking some existing visualisations I am working on to a new level.

The first book of interest is R in a Nutshell by James Adler recently published by O’Reilly.

R is a language and an environment to support data analytics and visualisation. Its approachable, extensible and open source. One of the advantages of R over other comers is the number of rather polished interpreters available for it and some of the great examples floating about that have been constructed in R. Hence my interest. I come to this interest from a digital humanities background and wondered whether the language could be of use for working with my own data coming from farm diaries exploring the cycle of seasonal farm activities.

I had an opportunity to peruse this volume and put it through it’s paces. I come to R as a neophyte having not attempted any custom data visualisation development aside from using commercial tools such as SAS, SPSS and Tableau or web services to create handcrafted visualisation for refinement in illustrator or Photoshop.

R in a Nutshell, as a book, is targeted at a user much like me. It offers a variety of approaches ranging from material for absolute beginners to experienced R users who would like to broaden their use of the environment. It allows for pick and choose areas of interest deigned to provide a bespoke learning course.

What do you need R for? Do you need to create customised statistical maniuplation of large datasets not typically accomplished using SPSS or SAS on your desktop? R may be the tool for you. This book is superb introduction to the tool, but it also serves the function of handy desktop companion for exploring further.

Starting from a detailed overview of the various ways in which R can be deployed on different operating systems and addressed as either a gui-based application to command line operation. Adler makes copious use of great examples of code to play with yourself. The structure of the book offers diverse patterns of approach to cater to people of all levels of experience with R. It is well organised and has the useful ability to locate information when it is of immediate applicability. If there is a downside, it is the amount of time it takes to play through examples to really determine whether R is the tool for you. I certainly can’t fault the book for this especially as it offers such a well structured approach to learning R. However, I can see where a digital humanities scholar who hadn’t ever considered doing any bit of code and waded into R right off the bat may simply be discouraged by a systematic process of introduction rather than a cookbook approach.

This is where we make a clever shift to our other book of interest: Getting Started with Processing by Casey Reas and Ben Fry. As you might suspect this book takes a very different and possibly more useful to a neophyte approach — moreover, I really enjo

yed this book. In a word it is superb. It approaches learning Processing from simple beginnings and truly embraces the idea of learning by doing. It is full of exercises and short snippets of discussion that simply ‘get you Processing’. As an added benefit and a superb way to do things, the latest downloadable builds of Processing include all the examples in this book accessible through the Examples menu. So as you work along you can edit to your hearts content and if you want to start with code and just simply tweak, you have it at your finger tips. If your style of learning is otherwise and you want to manually work through the examples, do that and if at the end things don’t quite work out you have the working code readily available. This works very well.

The book is organised into a few simple and general chapters. As you dig down into the recipes within the chapters you are presented with the breadth of the language. This approach wouldn’t work if things were formal and too structured. However, this book is very informal in tone and approach and you playfully follow through, not really conscious that you are following a rigid path.

Even the typography of the book adds to the fun. There are oversized headers and very nicely visual word snippets of the ‘experience’ of playing with Processing. It adds a wonderfully lyrical aspect to the book and really engages so as to keep you reading and playing along with the authors. The book mixes visuals and textuals very well. The decision to use ‘sketchy’ visuals as though they were hand-drawn keeps with this informal and engaged approach. Works for me.

Processing is very powerful and I feel that after working through this book I can see some very exciting projects that I can consider for my own work. I feel armed with just enough information to be dangerous and look forward to playing further. For this I will fall back more on Processing (published a few years back) the more definitive guide to the wealth of options available in the language. This is where R in a Nutshell offer the bonus of being a fine tutorial, but also a very useful reference tool. These are books about rather different programming tools that offer very different perspectives on accomplishing similar things. They provided me with a great intro to both tools. many books promise to be logical in their structure and to build to the more complex. I found that R in a Nutshell adopted a more reasoned approach to this. By that I mean the author sought to provided a variety of structured paths through the material depending on level of expertise or desired outcome/usage of the language. These were well considered. I apprroached this from the perspective of the complete neophyte (not a difficult role to fill — nor that far-fetched). Getting Started with Processing was more fun.

I should note that both of these books were provided in electronic form for review and I am grateful to Mary Rotman at O’Reilly for making this possible.

a

Eric Fischer has posted a new series of visualisations ‘Locals and Tourists’ depicting the location of photos taken in urban areas around the world. In this series he attempts to distinguish between those taken by tourists (people who seem to be a local of a different city and who took pictures in this city for less than a month) and those by locals (people who have taken pictures in this city dated over a range of a month or more). Intriguing.

What immediately struck me was his ingenious re-use of the existing data to create new information. By exploring individuals posted pictures over time he was able to hypothesise as to whether they were visiting or residing in a particular area. This allowed for a means to compare the gaze of the two groups.

I immediately started to explore his map of Dublin to see if any patterns emerged and then to try and suggest explanations for them. There is a healthy and regular mix of photos by both groups in the central core, but immediately to the east is a large blue box of photos taken by locals. It appears to surround the new Aviva Lansdowne Stadium in Ballsbridge. Additionally on the northside the National Botanical Gardens have a heavy concentration of photographs by locals.

The most practical application of Locals versus tourists is to consider how a visitor might use these visualisations to find the hidden city known only to its inhabitants — to find those secret spots worthy of capture by locals, but seemingly missed in the tourist guides.

This set builds on his earlier work ‘The Geotaggers’ World Atlas’ looking at from where the pictures were taken, whether from car, bicyle or when walking.

a

I came across this one in a book on the Rush Library. Not that earth shattering, but something about the textual overlay caught my eye. Could be the use of text rather than colour and legend, or rather than icons to represent the use of the space. Well done.

a