Visualization and art

by Artvisualizer Press Media Friday, May 01, 2009

The artists see the unseen and communicate it. That is exactly the same goals of scientific visualization.

There are obvious relations between science and art and between scientific visualization and art. Here I collected material, thinklets and references to explore this issue and to decide if this area is worth pursuing.

This research can be trivial or interesting depending on how I would consider the relationship between science and art: "…we serve any enquiry into art and science badly if our criterion is superficially the influence of science on art, or the influence of art on science", reminds us the paper From science in art to the art of science by Martin Kemp (Nature 434, pag. 308-309).

Directions

So which directions should I investigate?

Common traits between scientists and artists from the human point of view.
Core differences between science and art.
Visualization is science and art, but goes beyond nice images alone.
Metaphors from art used by visualization techniques.
What could each form of art provide to visualization?
Creativity: how is it considered by scientists? Which kind of support does creativity receive from visualization techniques?
Advantages and risks of mixing art and science.

Possible questions

After collecting references and ideas, what goal I want to reach? What questions should I pose? For example:

Role of beauty in selecting methods of representation especially in chemistry visualization.
Observe how an artist works. Compare the result with the original scene: what has been ignored?
Could I learn anything from the process of creating a watercolor paint?
How the exploratory visualization and artistic processes compare?

Scientists and artists

Leonardo the scientist and Leonardo the artist were one man, not two. Many of the qualities of a successful scientist and a talented artist are the same:

creativity
intuition
willingness to throw out the rules
close observation of nature
ability to see patterns where others see chaos

We can add also those common interests and themes:

Knowledge of human perception rules
Need to communicate something abstract (emotions, numbers)
Use of aesthetics to capture user attention

What specific characteristics artists have?

An eye trained very well in pattern recognition.
An unbiased eye that sees what usually scientists do not see because they are blocked by been able to see only what they are expected to find.
An artist also knows what (details) are not important and should be ignored.

Visual literacy: skill developed in interpreting, judging, responding to and using visual representations of reality. If you want to appreciate an artistic artifact, you should learn to see.

Researchers in many fields are becoming aware that in order to do really creative work, they may need to go back to visual approaches once again.

Very high level and creative achievement in the sciences has often come from the neurological resources linked to success in the arts.

Which is the process of visual understanding?

Artists work to achieve variations in visual expression, and may at times strive for the viewer to experience the emotional turmoil embedded in the art form. The artists' tools of line, shape, form, contrast, color, scale, composition, and movement are manipulated in order to affect the emotions and, at its zenith, the passions in others. But like the scientist, the artists' creations, to be successful, must maintain the intellectual components which are based upon the building blocks of structure, and the elements and principles of design.

This "visual communication" is computer graphic's link between scientists and artists, often those with differing approaches to the same information.

Steve Jobs: Real Artists Ship Products.

Art often convey a meaning to which we can relate.

Creative insight does not happen in a vacuum. Artists need models, examples, sketches. Artists benefit from rapid test of new ideas and from multiple approaches to the same problem (See Picasso's Guernica preparatory sketches).

"Can the Arts offer alternatives in setting research agendas, interpreting results and communicating findings?" from Stephen Wilson. Surely it can! An artist is an originator of ideas.

"What I knew for myself in my music system, and Peter so delicately reminded me in the domain of paint programs, was that the ten years that this has taken are nothing special - at least when contrasted to the years that the artists themselves have invested in developing their unique skills. While the essence of the artist is reflected in their work, it is rooted in skill – skill which is hard earned, and therefore worthy of respect by the instrument builder, or "luthier." But it is precisely these same skills which are so poorly captured by most computer-based tools. (From: Buxton, W. (1997). Artists and the Art of the Luthier. Computer Graphics: The SIGGRAPH Quarterly, 31(1), 10-11.)

"There is no science without fancy, nor art without facts"— Vladimir Nabokov

Science and art core differences

Prof. Kemp warns us of one of the fundamental differences between art and science. And again this warning helps us to go beyond trivial comparisons between these two fields.

"Many authors, particularly those aiming to communicate to an audience outside their immediate professional orbit, use artistry to stimulate engagement, impact and excitement. For many scientists, these dimensions are barely implicit, but for others the aesthetic motive is consciously present throughout."

"A work of art always remains open for interpretation, drawing the spectator into the shape of the artist's visualization, but without being able to exert fixed control over the feelings it induces. There is always room for the beholder's share."

"Scientists may wish to engage the reader or spectator in a wonderful journey of imaginative visualization, but in the final analysis they wish to communicate an interpretation that embodies testable content in an unambiguous way."

There is a continuum between an artist and a scientist. But there are also notable differences between those two kinds of person:

Artists take it as a given that there are no absolute points of reference in music, sculpture, or painting, and artists, their peers, and the buying public must make value judgments.
Scientists find the prospect of relative truth utterly distasteful and believe their ultimate arbiter is Nature.

To establish a dialog a common frame of reference should be set.

(From: Colwell, B.: Frames of Reference, Computer – Volume 38, Issue 6, May 2005, Pages: 9-11)

An artist has his personal agenda and a very particular point of view: communicate part of himself he wants the world to perceive.

Specialist often has a myopic perspective that precludes the comprehension of larger important patterns. Also unconnected knowledge quickly slips away. Unfortunately the environment for research in most universities is one which rewards increased specialization rather than encourages interdisciplinary cooperation.

Arts integrate, specialized science disintegrate knowledge into a myriad of different fields that lost communication between them.

The role of mental images in science

Many of our scientific models originated as psychical images whose forms could not be communicated to others except perhaps notionally. Because objectivity and reproducibility are the ultimate goals of scientific models, scientists and engineers who create and use these models rarely allude to this psychical process and hence effectively deny the motivation and inspiration behind this creative visual experience. J. W. Gibbs, James Clerk Maxwell, Albert Einstein, and Richard Feynman were exceptions in that they would openly talk about their "visual mental models".

The discovery process reported by scientists and engineers using visual tools closely parallels comments made by researchers who used a visual cognitive process to create scientific models. In both cases the "minds eye" is used to gain insight into complex abstract processes.

Visualization is science and art

This old IBM advertising expresses really well the role of visualization and computing…

…using an image normally related to artistic orientation.

Visualization facts

Visualization enable the researchers seen the unseen, the information hidden inside the data.
Visualization leverages human perception to make structure, regularities and patterns apparent from the numerical data.
Visualization anchors our mental models to external cognitive artifacts.
The cognitive artifacts use spatial and visual metaphors to make visible the data.
Visualization has a twofold role: as an exploratory tool and as a communication tool.
Visualization is an interface technology inside the scientific discovery loop.

Visualization is that initial step taken to establish a clearer pictorial representation of the problem and continue the development of the picture before the formal mathematics is done.

"Computing, and in particular supercomputing, without visualization, is like assembling a jigsaw puzzle in the dark". Richard Weinberg

Visualization and art

Which are common themes between an artist and who does scientific visualization? The same commonalities found between an artist and a scientist.

Images can resemble art, but the primary purpose is to communicate information.

The challenges facing information visualization researchers often involve finding innovative graphic and interactive techniques to represent the complexity of information structures.

Then from Einstein: "If we trace out what we behold and experience through the language of logic we are doing science; if we show it in forms whose interrelationships are not accessible to our conscious thought but are intuitively recognized as meaningful, we are doing art."

An important risk is about fidelity: visualization is about seen the unseen, or about seen the inexistent? Is what I see contained in the data or is it added by the visualization algorithms?

There is also the reverse problem: there are people that are 3D-blind. So why try to get an accurate picture if they don't see it right? The consensus is that to "teach people to see" we must learn from those in the fine arts. If we listen to the artists then we can go into a new phase with our pictures.

Artists can tell us what works and what does not works in a visual representation.

As for art, the interpretation and evaluation of patterns is entirely subjective.

Is visualization science?

Here are some ideas from the panel: "Is Visualization Struggling under the Myth of Objectivity?" from IEEE Visualization '95.

The problems that visualization has to fit in the mold of science derive between other things from: the divorce of human error and perception from the process of discovery, denial of emotional interpretation, poor aesthetic judgment and other subtle misdirections. Of particular interest will be those choices and presumptions born out of an automatic observance of the commandment of scientific objectivity.

Visualization works out of the grey area of human perception and cognition. So far it has not been found to be self-consistent; there are no rules which precisely govern what approach will work under a given set of conditions. Visualization is not strictly repeatable; one user may perceive a relationship (which may not be verifiable using visualization techniques) while another sees nothing at all or something entirely different. Visualization relies on subjective interpretation; contextual cues are usually derived from "common" experience which is neither guaranteed to be common or commonly integrated into personal experience. Visualization is resistant to the systematic evaluation and assessment procedures common to science; it still remains difficult to ascertain if a particular instantiation has been "successful". And there are persons who see the compelling nature of the imagery as an open temptation for abuse and trivial indulgence.

So how does it contribute to science at all? Visualization is part of a process of discovery. It appears to aid the intuition in identifying relationships, some of which can be later described by formal analysis. In cases where there is no such analysis, the results of visualization cannot be properly integrated back into the scientific domain.

What art teaches to visualization

Metaphors
Tell a story. A story with a beginning, an end and a point.
Engage the audience
Visual organization

Metaphors and mappings

The concept of mapping is closely related to visualization but it makes sense to keep it separate. By representing all data using the same numerical code, computers make it easy to map one representation into another: grayscale image into 3D surface, a sound wave into an image (think of visualizers in music players such as iTunes), and so on. Visualization then can be thought of as a particular subset of mapping in which a data set is mapped into an image.

In such situations designers and their clients have to choose which dimensions to use and which to omit, and how to map the selected dimensions. This apparently rational decision process could be made explicit, instead of presenting only the end result to the user. Art can instead make the method out of irrationality and data visualization can try this also.

Most mappings in both science and art go from non-visual media to visual media. Is it possible to create mappings that will go into the opposite direction?

And if modernism reduced the particular to its Platonic schemas (think of Mondrian, for instance, systematically abstracting the image of a tree in a series of paintings), data visualization is engaged in a similar reduction as it allows us to see patterns and structures behind the vast and seemingly random data sets.

Thus it is possible to think of data visualization as a new abstraction. But if modernist abstraction was in some sense anti-visual – reducing the diversity of familiar everyday visual experience to highly minimal and repetitive structures (again, Mondrian's art provides a good example) – data visualization often employs the opposite strategy: the same data set drives endless variations of images (think of various visualization plug-ins available for music players such as iTunes.) Thus, data visualization moves from the concrete to the abstract, and then again to the concrete. The quantitative data is reduced to its patterns and structures that are then exploded into many rich and concrete visual images.

I often find myself moved emotionally by visualization projects outcomes. Why? Is it because they carry the promise of rendering the phenomena that are beyond the scale of human senses into something that is within our reach, something visible and tangible?

This promise makes data mapping into the exact opposite of the Romantic art concerned with the sublime. In contrast, data visualization art is concerned with the anti-sublime. If Romantic artists thought of certain phenomena and effects as un-representable, as something which goes beyond the limits of human senses and reason, data visualization artists aim at precisely the opposite: to map such phenomena into a representation whose scale is comparable to the scales of human perception and cognition.

(From: Lev Manovich: The Anti-Sublime Ideal in Data Art, Berlin August 2002)

An already explored strategy has been to use metaphors from other areas such cartography or the natural environment.

The natural environment we inhabit has always been a source of inspiration to artists. For example the organisms in a mature ecosystem live following some rules. And those rules can be mapped to visualization design in the context of the optimization of the cognitive potential of visualization users.

Do not forget that the users live in the real world, not in a metaphorical one. Some unneeded metaphors are nice at first, but are immediately tiring.

Analogy and metaphor are important contributors to creative thinking. An analogy is more creative the smaller the distance within local subspaces and the larger the distance between comparative subspaces.

The concept of metaphor derives straight-forwardly from analogy, in that they both force a different perspective or interpretation of a situation or construct.

Artistic forms comparison

Visual arts: Obvious comparison.
Photograph: Show what the photographers want to show.
Narrative: Tell a story, engage audience, create visual writing, ignite imagination.
Music: Structures, symmetries. Communicate information in the absence of visual contact.
Plastic arts: Direct manipulation, 3D, touching and force feedback. Beauty as balance: any addition is superfluous, every subtraction leaves a hole.
Cinema: Tell a story, focus attention.
Graphic design: Composition, balance, usability. They developed guidelines (related to modes of perception) for the use of type, form and color to optimize visual communication.
Illustration: Highlights interesting parts, omits unimportant details. Specific features enhancement and highlight.
Optical illusions: Pre-wired interpretations and assumptions.

Creativity

Do not forget creativity. There are interesting parallels with visualization (and obviously arts).

Graphical methods support: Mindmaps, Concept Maps
Process: Sensemaking cycle and creativity process. Knowledge as a net of relationships
Go out of the box: See the data in unusual ways, force unexpected relationships
Ambiguity: Ambiguity to facilitate flexible thinking

A key activity in the process is the exploration of ideas, knowledge, and options. Some examples of aspects of exploration that were identified from empirical studies are summarized here:

Breaking with convention. Breaking away from conventional expectations, whether visual, structural, or conceptual, is a key characteristic of creative thought. Events that hinder such breaking with convention are avoided, whereas positive influences are embraced.
Immersion. The complexity of the creative process is served well by total immersion in the activity. Distractions are to be avoided.
Holistic view. The full scope of a design problem is only fully embraced by taking a holistic, or systemic, view. The designer needs to be able to take an overview position at any point and, in particular, to find multiple viewpoints of the data or emerging important design.
Parallel channels. Keeping a number of different approaches and viewpoints active at the same time is a necessary part of generating new ideas.

From: E. Edmonds, L. Candy: Creativity, art practice, and knowledge, Communications of the ACM, Volume 45, Number 10 (October 2002), Pages 91-95

Ben Shneiderman for his work on human computer interaction and in the area of computer support to creativity.

Creativity and the exploratory visualization process: Which tools are needed? Which visualization tools do I use that help my exploratory visualization tasks?

Creativity: process toward an outcome recognized as innovative
Exploratory visualization: process toward an outcome from data analysis that is non trivial and unexpected

Margareth Boden makes the point that changing a constraint might be at the core of creative thinking.

Other important characteristics of creative designers:

Holistic thinkers. Look at the broad scope before going into details.
Solution-lead. Designer often proposes several candidate solutions early on in order to better examine the problem. Designers impose constraints that help reduce possible solutions and help new ways of thinking.

A key activity in the process is the exploration of ideas, knowledge and options.

Modeling the creative process support systems shows that knowledge and visualization are essential ingredients of creativity work.

Parts of the creativity process:

Exploration. This is an open-ended process, possibly without observable direction. However, the thoroughness and selectivity of the process is critical to the generative stage.
Generation. Creativity is demonstrated by the generation of many potential solutions instead of gravitating quickly toward a single and (usually) familiar solution that is not necessary the optimal one.
Evaluation. The use of expert knowledge in evaluation has been identified as an important aspect of successful solution finding.

"The disappointing fact is that the majority of design is not in groundbreaking innovation, but in variations on something successful."

Analogy and metaphor are important contributors to creative thinking. This is accomplished by generating pre-inventive forms or structures and then tweaking (that's jargon for twisting, forcing, inquiring, etc.) the form to relate it to a novel avenue of thought or procedure. Normally it doesn't appear to fit immediately. That's precisely the point; one has to generate novel relationships and perspectives to be creative, and this mechanism forces one to do just that. An analogy is more creative the smaller the distance within local subspaces and the larger the distance between comparative subspaces.

The concept of metaphor derives straight-forwardly from analogy, in that they both force a different perspective or interpretation of a situation or construct.

Mixing disciplines

There is a potential of misinterpretation due to the artistic inherent subjectivity of the process.

Artist: how much objectivity before starts losing intuition?
Scientist: how much intuition before they lose scientific foundations?

Art: pleases the eye. Science: communicates.

Generate new way of thinking. Not only provides new tools for the artist. This is not the point. But a computer could add new artistic dimensions (time) or break established conventions.

An interesting aspect of the collaboration between artists and scientists is the way in which it provides participant with more than one viewpoint about the nature of the creativity process.

The artist (or the visualizer) needs the correct visual vocabulary to make realistic proposals to the scientist.

Techniques from the arts

Depth cues: atmospheric perspective, perspective, size gradients and occlusions
Remove unnecessary details. And add details when needed to clarify (as seen in Tufte)
How to make a surface visible (surface textures)
Oil paint brush strokes
Esthetic to capture user attention and to communicate better
Guiding the eye where it should look
Light usage
Preattentive processing
Color. From a perceptual perspective, as a medium to capture attention, as a medium to transmit information.

On the minus side:

Dequantification: no references to the quantitative dimension of the data
Normally very limited form of interaction with the artistic artifact

General resources

First of all two names:
- Donna Cox
- Victoria Interrante
They have done a lot of work on visualization and art.
Then Computer Graphics and Applications from IEEE has interesting papers on artistic concepts used by scientific visualization like Yeary, R.: The Art of Science, March-April 1997 Volume: 17, Issue: 2 pag. 4-5.
The visualization guru Edward Tufte is preparing the book Beautiful Explanations. Book excerpts are on his site.
Don't forget a Google search on "Artist and Science" or "Artist and Visualization".

Interesting pages

The Contribution of the Artist to Scientific Visualization
I think this is the most interesting essay on the topic.
Dejan Vinkovic: Sci-Art
Page with interesting links and material.
The Art and Science of Depiction
It is focused more on computer graphic issues, but has very interesting references on visual perception.
Resources for integrating graphics with science and technology education.
Links and resources on how to start a visualization career.
Visualization, Cultural Mediation and Dual Creativity
Dense essay, but I do not have the courage to read it…
Where Science Meets Art
Some press releases on the topic.
Image and Meaning
Events related mostly to visual communication.
Scientific Visualization in Art@Science by Johanna Hyrkäs
This paper tries to define the concept of scientific visualization both from the viewpoint of art and that of science. Also introduces three case-studies on the use of scientific principles in painting.
Scientific Visualization: Collaborating to Predict the Future
One of Donna Cox essays. In the same site there are other interesting papers, for example The Art and Science of Collaborative Visualization.
Another explorer of the relations between science and art is prof. Martin Kemp, professor of the history of art at the University of Oxford, and co-director of Wallace Kemp Artakt. Some of his books seems interesting:
- Martin Kemp, Seen and Unseen (to be published)
- Martin Kemp, Visualisations, The Nature Book of Art and Science, OUP, 2003
- Martin Kemp, Immagine e Verità, Il Saggiatore. 1999
Prof. Kemp does not provides much information on-line, except in the description of his course Renaissance Art and Visual Knowledge and the already cited paper on Nature 434, 308-309 (17 March 2005) titled From science in art to the art of science
Greg Judelman: Aestetic and Inspiration for Visualization Design: Bridging the gap between Art and Science, Eighth International Conference on Information Visualization (IV 04)
In The Visualization Handbook, Editors Chuck Hansen and Chris Johnson, Academic Press 2004 (editor page) there are two interesting essays:
- Victoria Interrante: Art and Science in Visualization pages 781-805.
  Nice examples from research in illustration methods applied to visualization
- Robert Kirby, Daniel Keefe, David Laidlaw: Painting and Visualization pages 873-891.
  Not only techniques that visualization can borrow from painting, but also artistic critique methods to test visualization effectiveness.
Use of Visual Information in Art: A brief review of some of the ways artists can create particular experiences. A really nice interactive tutorial.
Buxton, W.: The Role of the Artist in the Laboratory, in C. Schopf, (Ed.). Meisterwerke der Computer Kunst. Bremen: TMS Verlag.(1988) pp. 29-32.
Lists three benefits from scientist-artist collaboration and references the work of various artists.