This article comprises 1) an introduction to aspects of the field of digital art history; 2) a presentation of a digital experiment with photographs of objects from our shared cultural heritage; 3) sections introducing technical concepts and image-related issues in the digital field; and finally 4) a section on the wider perspectives. The technical sections are placed in the last part of the article. While we considered presenting them at an earlier point, we decided \u2013 given that the text is aimed at art historians without special knowledge of the field \u2013 to use general and widely accessible terms which are then later expanded with explanations of a more technical nature. At the same time, the intention is to have the analysis of the digital experiment and its visualisation act as a catalyst for bringing out more general explanations and relevant issues.<\/p>\n
The end notes contain references to a wide range of literature, discussions and various types of projects in the field of digital art history, reflecting the article\u2019s intention to contribute examples and reflections that can hopefully inspire more people to get involved in developing the discipline of art history in an ongoing interaction with digital technology.<\/p>\n
Digital art history<\/h2>\n
The term \u2018digital art history\u2019 has been used to frame the field since around 2010. Courses, summer schools and working groups have been established, and in 2015 an international journal was dedicated to the field, while other journals such as the American Panorama<\/em> have expanded their editorial staff with a digital art history department in 2021.International Journal for Digital Art History<\/em>, no. 1, 2015, pp. 50–64. Diana M. Zorich was in charge of the report Transitioning to a Digital World: Art History, its Research Centers, and Digital Scholarship<\/em> for The Kress Foundation, 2012: https:\/\/www.kressfoundation.org\/Resources\/Sponsored-Research\/Research-Items\/Transitioning-to-a-Digital-World<\/a> For comments on this report, see K. Bender’s blog entry of 1 June 2012: http:\/\/kbender.blogspot.com\/2012\/06\/digital-art-history-brave-report-and.html?view=magazine<\/a> In 2012, the German association of art historians set up their Arbeitskreis Digitale Kunstgeschichte<\/em>. International Journal for Digital Art History (2015 -), website: https:\/\/dahj.org\/<\/a> In their call for applicants, the editors of Panorama. Journal of the Association of Historians of American Art<\/em> states that ‘our discipline has been slow to keep up’ as regards the integration of digital technology: https:\/\/editions.lib.umn.edu\/panorama\/article\/digital-reviews-initiative\/<\/a> All links in the notes accessed 1 April 2021.\n\">2<\/a><\/sup>\u00a0\u2018Digital art history\u2019 is used as an umbrella term covering and connecting a number of activities within art history education, curation and research, where digital tools and methods are used to a greater or lesser degree. Thus, this broad definition of digital art history includes everything from establishing image databases and using PowerPoint in an art history lecture to applying artificial intelligence for visual analyses of digital images. But, as the oft-cited digital humanist and artist Johanna Drucker objects, the field can be divided into two main areas, making a distinction between \u2018digitised art history\u2019 and \u2018digital art history\u2019. Digitised art history includes the building of digital infrastructure such as digitising archives and images, and the term describes art history\u2019s general shift from analogue to digital tools and platforms. Digital art history, on the other hand, is defined more narrowly by Drucker as studies of art and culture where computational processing is used in the analysis of the digitised or digital art historical material.Visual Resources<\/em>, vol. 29, nos. 1–2, 2013, p. 7. Also consider the notion of computational thinking<\/em>, which is about formulating problems in such a way that they are amenable to being solved by computers. ‘Computational Thinking is the thought processes involved in formulating problems and their solutions so that the solutions are represented in a form that can be effectively carried out by an information-processing agent’, see Jeanette M. Wing: ‘Computational Thinking: What and Why?’, The LINK. The magazine of Carnegie Mellon University's School of Computer Science<\/em>, 2010.\n\">3<\/a><\/sup>\u00a0In practice, this means that digital art history is about using computational power such as artificial intelligence to analyse digital written or visual material<\/em>, a process which involves reflections on both theory and method. This distinction corresponds to the differentiation of digital humanities that Jan Luhmann and Manuel Burghardt refer to as digitised humanities<\/em> and numerical humanities<\/em> (the use of computational models), to which we may add the term humanities of the digital<\/em> (the study of digital phenomena, such as online communication) and the field of public humanities<\/em> in a digital perspective.Journal of the Association for Information Science and Technology<\/em>, 2021 p. 2.\n\">4<\/a><\/sup><\/p>\n Work on the development and use of digitisation to benefit the discipline of art history has been done for some time now. However, it is only in recent years that academics have begun earnestly to consider how the computer can be involved in actual analysis and to reflect on the significance this may have for the future development of the discipline and its repertoire of methods. As Drucker describes the purpose of digital art history: \u2018We have to take into account the ways digital humanities more broadly have taken up computational techniques and then consider the specificity of visual art objects and their particular requirements and points of resistance\u2019.5<\/a><\/sup><\/p>\n One of the early sources of funding for digital art history, The Getty Foundation, openly and heuristically defines the field in these terms: \u2018The term \u201cdigital art history\u201d has become a shorthand reference for the potentially transformative effect that digital technologies hold for the discipline of art history\u2019.www.getty.edu\/foundation\/initiatives\/current\/dah\/<\/a>\n\">6<\/a><\/sup>\u00a0But how should this ‘transformative effect\u2019 be understood? Critics of digital art history argue that even though some tools have gone digital, the core activities and methods of art history are still fundamentally unaffected by computational methods that still leave much to be desired.International Journal for Digital Art History<\/em>, no. 3, 2018, pp. 133–139. Claire Bishop warns that the use of computational methods may simplify complex methods used by the humanities, thereby curtailing the humanities’ academic expertise. She does so in her article ‘Against Digital Art History’, Humanities Futures<\/em>, Franklin Humanities institute, 2017 Against Digital Art History – Franklin Humanities Institute (humanitiesfutures.org)<\/a>. For example, she states that ‘theoretical problems are steamrollered flat by the weight of data’.\n\">7<\/a><\/sup>\u00a0We would tend to agree with this. However, we would add the caveat that even simply taking a stance on these technological possibilities and their possible integration into art historical research can in itself cause a shift in the understanding of the discipline and its methods more generally.<\/p>\n Our starting point is that art history and digital art history are not opposites, but that the work with art and visual culture involves a great variety of research questions that can be solved across digital as well as analogue working methods. As will be evident from the examples provided in this article, digital and analogue work processes intertwine. In order for art history professionals to navigate the opportunities available, make constructive choices in relation to the individual tasks and promote further developments, we must each be able to justify our decision to opt in or out. But this also means acquiring knowledge and experience that can hone our eye for the latent possibilities \u2013 in the digital and analogue realms alike.<\/p>\n Visual documentation of cultural heritage, which is the focus of the case presented in this article, is increasingly being released by institutions around the world and is used for many different purposes by individuals, professionals and artists. Contemporary art very much already occupies a digital reality alongside documentation of older art and visual culture more generally, where many disciplines other than art history also work with such material. As a discipline with a long-standing tradition of reflection on the use, perception and theory of images, it is only natural that art history should help to develop criticism and increase quality when the field and objects addressed by art history \u2013 in the broadest sense of the term \u2013 are treated digitally.<\/p>\n A steadily growing group of art historians, primarily outside of Denmark, are able to personally code and use computational methods to analyse art historical data sets, whether focusing on visualizations or statistics. However, less extensive skills and less experience with interdisciplinary collaborations in the digital field may still be sufficient for those wishing to initiate criticism and reflection on what this kind of research means. Given this background, the present article introduces selected aspects of the field of digital art history, providing examples of how the core skill of the discipline \u2013 visual analysis \u2013 can unfold in the digital field. We have selected issues that are useful to bear in mind when using computers to analyse images of artistic and cultural objects documented by museums and others. We also touch upon how image analysis is (or can be) conducted using a computer. The open question is whether greater awareness of these issues and processes in themselves may contribute to the further development of methods in art history, with or without a digital element. Critical reflection on the involvement of the computer in art historical research is an important topic, but this article aims more at contributing to the basis for understanding the field than at unpacking the critical dimension.<\/p>\n In the USA, large trusts have funded digitally oriented projects in the realm of humanities studies, prompting the development of centres for digital humanities that include art history. Dutch research efforts have long included statistical studies, geomapping and socio-economic analyses of art and culture, for example from the early modern period, and these studies have taken on a digital dimension.Cornelia (projectcornelia.be)<\/a> about Flemish seventeenth-century artist scenes and The Goupil Stock Books Project | Getty Iris<\/a>, where art dealers Goupil & Cie’s collection of art sales forms the data basis. An oft-quoted project is Pamela Fletcher and Anne Helmreich’s studies of the London art market in the nineteenth century, see Pamela Fletcher and Anne Helmreich: ‘Digital methods and the study of the art market’, in: Kathryn Brown (ed.), The Routledge Companion to Digital Humanities and Art History<\/em>, 2020, chapter 12 (see also other projects in this anthology in ‘Part II Archives, Networks, and Maps’. Other projects present network studies via online data visualisations, see Connect Vermeer<\/a>, and the database https:\/\/rkd.nl\/nl\/ also includes maps charting the artists’ travel patterns. The number of this type of project is strongly on the rise these years. In data visualisations, Lauren Tilton and Jordana Cox link the travel patterns of twentieth-century American photographers with the photographs they took along the way. In this way, they link up geodata and visual information, creating a basis for formulating and answering research questions about issues such as the social status of photographers and the cultural imprint of their photographs. In another project, Lauren Tilton and Taylor Arnold used distant viewing techniques to analyse each individual frame of two American TV series to examine how the female lead was represented visually. The computer analysis could then be compared with the structures of the narrative and contribute to an analysis of the representation of gender. Cox, Jordana and Lauren Tilton: ‘The digital public humanities: giving new arguments and new ways to argue’, Review of Communication<\/em>, Taylor & Francis Online, vol. 19, no. 2, 2019, pp. 127–146. https:\/\/doi.org\/10.1080\/15358593.2019.1598569<\/a> and Arnold, Taylor and Lauren Tilton: ‘Distant viewing: analyzing large visual corpora’, Digital Scholarship in the Humanities<\/em>, Oxford University Press, vol. 0, no. 0, 2019, pp. 1-14.\n\">8<\/a><\/sup>\u00a0Other countries \u2013 such as the German-speaking territories \u2013 have traditions for empirical studies and archival studies that give them an excellent basis for converting analogue information gathering into digital formats that can form the basis for computer-based analyses.Institut f\u00fcr Realienkunde des Mittelalters und der fr\u00fchen Neuzeit, Object Links. Dinge in Beziehung<\/em>, vol. 1, 2019, pp. 95–126. REALonline: https:\/\/realonline.imareal.sbg.ac.at\/ueber-uns\/<\/a> For a fuller description of the project, see Matschinegg, Ingrid et al., Dataen neu verknoten. Die Verwendung einer Graphendatenbank f\u00fcr die Bilddatenbank REALonline<\/em>, DARIAH-DE Working Papers 31, G\u00f6ttingen, 2019. See the project VIKUS Viewer on coins: https:\/\/uclab.fh-potsdam.de\/fw4\/<\/a>.\n\">9<\/a><\/sup><\/p>\n So there is a large art history research community in the Netherlands, the USA and Germany, where the presence of research groups and infrastructures offers more opportunities. By comparison, the two art history departments found at Danish universities are less prone to work with tasks that require major organisation of and access to the cultural heritage, such as extensive studies of objects and sources or digitisation processes. Generalising greatly, Danish universities tend instead to look towards research questions based on aesthetic, philosophical and theoretical approaches, and working methods. Unless you work with topics such as computer art or media theory, it can be difficult to identify clear-cut ways of operationalising philosophical questions and identifying information that can be translated and processed by a computer in relevant ways. Figuring out exactly how the computer can help process words or images requires not only technical developments, but careful thought too. For example, one might inquire into the concept of atmosphere and its applicability to museum architecture, or how aesthetic principles are embodied visually or textually in certain environments. While such projects do exist, the material and the research questions would require a development of ideas, both in terms of technology and method. Furthermore, the field offers fewer established lessons on which to draw.Pamphlets of the Stanford Literary Lab<\/em>, no. 16, 2017. Bell, Peter and Bj\u00f6rn Ommer: ‘Computer Vision und Kunstgeschichte – Dialog zweier Bildwissenschaften’, in: Piotr Kuroczy\u0144ski, Peter Bell and Lisa Deickmann (eds.): Computing Art Reader – Einf\u00fchrung in die digitale Kunstgeschichte<\/em>, Heidelberg: arthistoricum.net, 2018, pp. 61–75. Impett, Leonardo: ‘Analyzing Gesture in Digital Art History’, in: Kathryn Brown (ed.), The Routledge Companion to Digital Humanities and Art History<\/em>, 2020, chapter 28. Bell, Peter and Impett, Leonardo: ‘Ikonographie und Interaktion. Computergest\u00fctzte Analyse von Posen in Bildern der Heilsgeschichte’, Das Mittelalter<\/em>, no. 1, vol. 24, 2019, pp. 31–53.\n\">10<\/a><\/sup><\/p>\n This is to say that effort is required to identify where (and whether) the computer can make a useful and adequate contribution \u2013 and whether an applicable model already exists for the task, such as object detection, semantic analysis of words or network analyses of written or visual material. Next, the question is whether a dataset is in place \u2013 one that is delimited, annotated and digitised. For example, work has been done on image data sets such as nineteenth-century auction catalogues, medieval religious motifs, photographic portraits and fashion magazine covers.<\/p>\n Within cultural institutions and research communities in Denmark, it is certainly possible to find datasets, potential datasets and research questions that could benefit from digital methods in different ways. But before any computer specialists can be involved, the art historian must, at a conceptual level, have some sense of the working conditions and potentials offered by the computer. As was indicated in the above, the present article wishes to help build a clearer sense of this.<\/p>\n The project behind this article examines how selected computer methods can be used in art historical research, while at the same time drawing on the extensive theoretical tradition in art history and using this to stimulate reflection on the use and perception of images in the digital humanities.Digital Art History: Rediscovering the Painting<\/em> is conducted as a collaborative effort involving Aarhus University and SMK – Statens Museum for Kunst, with Lisbet Tarp as PI, Anne Haack Christensen as postdoc fellow at SMK and the Center for Humanities Computing Aarhus and SMK Open as partners. The project is funded by the New Carlsberg Foundation (2019–20) and The Independent Research Fund Denmark (2020–22). \n\">11<\/a><\/sup><\/p>\n One of the fundamental tenets behind our efforts concerns the development of the theoretical aspects of our work in an ongoing process of exchange with digital experiments, ensuring that the project is rooted in actual practice. The material analysed in our project is digital photographs of artworks that have been produced and processed in different ways, prompting us to develop the concepts of the expanded<\/em> and the reduced<\/em> artefact. To help inform definitions of concepts like these, visualisations are carried out on the basis of digital experiments with photographs of cultural heritage objects.<\/p>\n The mediation processes regarding the materiality of the artefacts play a significant role in this regard, and visual analysis as a method is a main focal point for the studies. Later in this article, the idea of \u2018the reduced artefact\u2019 is illustrated with a single example where artificial intelligence has contributed to a visual analysis of thousands of photographs of artworks. The project is thus based on definition of digital art history put forward by Drucker, in which the computer is involved as a kind of analytical partner. Thus, this article and the project in general is not about the actual digitisation process of art and cultural objects, the establishment of digital infrastructures in the museum institution, the FAIR principles or the efforts at ensuring open access.Findable, Accessible, Interoperable, Reusable<\/em>. Luther, Anne: ‘Digital provenance, open access, and data-driven art history’, in: Kathryn Brown (ed.), The Routledge Companion to Digital Humanities and Art History<\/em>, 2020, chapter 32.\n\">12<\/a><\/sup>\u00a0Nor are they about the study of digitally based art or computer art, which are also considered part of digital art history.<\/p>\n The relationships between a cultural object, its documentation and data visualisation constitute an important but also difficult topic, the discussion of which has gained increased relevance with recent technological developments. We can now increasingly collect, record and digitally store and process various types of documentation about the objects, including digital images. At the same time, we face the enduring question of what kind of information should be prioritised and registered in each individual case, and how this information should be represented and interpreted, especially for academic purposes. In many cultural heritage projects, digital photo documentation is of crucial importance for the project design and for enabling interpretation of the datasets collected.Time Machine Europe<\/a> and, regarding the technology of photogrammetry: Cultural Heritage Imaging | Photogrammetry<\/a>\n\">13<\/a><\/sup>\u00a0At a more basic level, the fact that photography is itself an interpretation is also an issue to be considered.<\/p>\n While art history studies have made a particular virtue of first-hand knowledge of artworks, reproductions have often been used to develop familiarity with these artefacts both now and in the past. Art historian Christopher Lakey offers an example of how photographic reproductions have led to possible misinterpretations in art history research. In his study of photography of reliefs from the late Middle Ages, he shows that the angles used to take the pictures have had a decisive impact on how art history scholars have interpreted matters of style and expression. The reliefs have been photographed from the front, making the depictions appear static and lifeless, which affected how the style was described. However, if one look observes the reliefs from the original intended viewing position \u2013 from below and at an angle \u2013 they take on a very different, more dynamic expression.14<\/a><\/sup>\u00a0In terms of digital images, the case of \u2018the yellow milkmaid\u2019 has drawn attention to how the many digital photos of art found online can result in distorted perceptions of what a given artefact actually looks like.The Problem of the Yellow Milkmaid | Europeana Pro<\/a>\n\">15<\/a><\/sup>\u00a0In 2011, Europeana called attention to what has since been called \u2018The Yellow Milkmaid Syndrome\u2019, acknowledging the fact that a great many poor-quality photographs of art and culture are circulating on the Internet, including pictures of Johannes Vermeer\u2019s (1632\u20131675) painting of a milkmaid. Awareness of this fact has spurred on the efforts by museums to make professional and authorised photographs available to the public.<\/p>\n On another level, it must be noted that no matter how high-resolution, well-lit and finely honed a photograph is, it will always only reproduce a limited range of the physical properties of the original work. This limits what the images can be used for in digital data processing if the direct relationship to the depicted work is relevant in the analysis. The singular photograph represents the appearance of a work in a specific, choreographed moment, and the photograph will always already be a reduced version of the remediated artefact \u2013 one among many possible representations. This basic art historical insight is not a natural part of the reflections when photographs of artworks are used in computer analysis. Overall, the critical tradition in art history of image analysis and theory often plays a very small role in how other disciplines use such photographs in the digital field.<\/p>\n By describing the digital photographs of artworks as reduced<\/em> artefacts, we wish to raise awareness of how photography is a form of mediation where only parts of the materiality and expressive devices of the original object are reproduced in each individual image, whether digital or analogue.16<\/a><\/sup>\u00a0To this we may add that digital data processing further reduces the photographic record of the work in the translation from light input to numbers.<\/p>\n Reproductions can take on a life of their own and help increase interest in the artefacts they depict. However, in the context of this project, the assumption is that the digital experiment must to some extent contribute to the understanding of the SMK \u2013 National Gallery of Denmark collection, even if this is done by means of reproductions. This is to say that the extent to which the individual artefact is reduced in the mediation matters, and the relationship between artefact and reproduction is still relevant.<\/p>\n The remediated versions of a given object divide and rearrange specific features, but image analysis performed by a computer can potentially uncover aspects that have hitherto been unrecognised or even invisible to the human eye, whether these are relevant to art history research or not. At the same time, reductions and expansions of the artefact in the digital dimension can promote awareness of what is lost and gained in the process, both visually and materially.<\/p>\n By this stage, the sheer quantity of available digitised information on art and cultural objects means that it can make sense to have the computer identify and point out structures in visual and textual material alike. In the case study we present here, the content of our images are compared and analysed quantitatively by the computer. In short, the visual analysis carried out by the algorithms is applied to a large quantity of images and then presented in a visualisation that can be accessed via a web browser. This computational approach to image analysis has been called \u2018distant viewing\u2019 as a visual counterpart to literary theorist Franco Moretti\u2019s concept of \u2018distant reading\u2019, in which the occurrence and frequency of words and structures in very large text corpora are analysed by means of algorithms.International Journal for Digital Art History<\/em>, no. 1, 2015. Regarding distant viewing, see also Harald Klinke: ‘The Digital Transformation of Art History’, in: Kathryn Brown (ed.), The Routledge Companion to Digital Humanities and Art History<\/em>, 2020, chapter 3. \n\">17<\/a><\/sup>\u00a0Computer specialist and cultural analyst Lev Manovich is frequently highlighted as one of the first to perform concrete \u2018distant viewing\u2019 analyses and produce the distinctive data visualisations in which thousands of images are displayed as small icons gathered in a cloud formation \u2013 just as we do in the digital experiment presented in this article.International Journal for Digital Art History<\/em>, no. 1, 2015, pp. 12–35.\n\">18<\/a><\/sup> But whereas Manovich works with photographs from social media and the cultural spheres in which these images operate, the case described in this article is based on photographs of artworks associated with other social practices and codes defined (in part) by the museum institution and its traditions regarding documentation.<\/p>\n SMK\u2019s image dataset was less than optimally suited to the kind of computational analysis where one aims to identify visual relationships between the depicted artworks. This holds true for many institutions: their data material is shaped and informed by the documentation practice of the institution and its various changes over the course of decades. In SMK\u2019s digitised collection of artworks, one can trace a development extending from black and white photos to colour photography and digital images taken under standardised lighting conditions, accompanied by varying quantities and types of text-based metadata. As in all other endeavours involving empirical data, the less-than-perfect nature of the visual documentation establishes the perimeters for how the material can be used in a research process.<\/p>\n If one wishes to get to know a museum collection and its individual objects, one would typically visit that museum, read exhibition catalogues and research-based literature, search the database or study the website. Only in special cases would it be possible to view things kept in storage. The literature and exhibition rooms often present only a fraction of the collection, and the presentation, interpretation and learning materials only offer information on a small selection of artefacts, most often the best known. While work is currently being done to create alternative points of access to the SMK\u2019s online collection, visitors to the museum website will often come across reproductions via the database, meaning that they will only be able to view smaller portions at a time.The Close-up Cloud<\/em> is working to develop interfaces where details, totalities and contexts can be accessed flexibly. Juninger et al.: ‘The Close-up Cloud: Visualizing Details of Image Collection in Dynamic Overviews’ in: International Journal for Digital Art History<\/em> 5, 2020. \n\">19<\/a><\/sup>\u00a0As a visualization format, an image cloud offers a different type of navigation than the database website, thus providing more direct access to a wider section of the collection. It can supply a different kind of overview of the digitised part of the collection, its composition and particular centre of gravity. Less well-known art appears and visually the cloud reveals the overall composition of the collection. At SMK, for example, the scope of the collection of works on paper is very noticeable, even though only a small part of the total amounts of works on paper have been registered with photographs as yet.https:\/\/www.smk.dk\/en\/article\/studiesalen\/<\/a>\n\">20<\/a><\/sup>\u00a0While it can be argued that image clouds should be made up of relatively homogeneous material \u2013 only photographs of paintings, for example \u2013 in order to avoid giving the computer an ill-defined task, it also means that a visualisation of SMK\u2019s complete image data set can render visible and convey the broad and composite nature of the collection. Among other things, this trait can point back to the many different collectors who have contributed to the collection through the ages, while at the same time offering perspectives on the institution\u2019s status as a museum today. Of course, certain limitations apply: artefacts that have not yet been registered with associated words and images cannot be included in the computer\u2019s analyses, meaning that there will be gaps in the overall impression conveyed by a picture cloud or a semantic analysis.<\/p>\n However, the way in which the image cloud provides access to the visual material also imposes demands on the user. The fact that the images are in principle equally accessible does not make access to them any more democratic. The image cloud can appear unmanageable, opaque and labyrinthine, and one might well struggle to find something interesting \u2013 a challenge that applies to general audiences and art history professionals alike. The possibilities for getting something useful out of visualisations such as the image cloud are still governed and limited by the individual user\u2019s previous experience and current conditions.21<\/a><\/sup>\u00a0Perhaps in order to meet the need for navigating vast quantities of images, Google has created a kind of guided tour for some of their visualisations of photographs of artworks, while others use more context-based interfaces such as VIKUS viewer.t-SNE Map by Cyril Diagne, Nicolas Barradeau & Simon Doury | Experiments with Google<\/a> The Google Arts & Culture project has carried out many other attempts at creating narratives in the selection of artefacts. M\u00fcnzkabinett Berlin visualisation: https:\/\/uclab.fh-potsdam.de\/coins\/<\/a> and https:\/\/olh.openlibhums.org\/articles\/10.16995\/olh.280\/<\/a>. Both were inspired by VIKUS Viewer, see note 9. \n\">22<\/a><\/sup><\/p>\n Google\u2019s visualisations are typically conglomerates of photographs of artworks and objects from many different collections, while the digital experiment we conducted constitutes an abstract image of a single museum\u2019s digitised portion of its collection. It can make sense to look at the computational analysis of a single collection, but there are also perspectives in visualising structures across collections.23<\/a><\/sup><\/p>\n The following presents a digital experiment featuring SMK\u2019s approximately 40,000 photographs of artworks from the collection.SMK Open<\/a> offers access to information on approximately one-third of the SMK collection of more than 260,000 artefacts.\n\">24<\/a><\/sup>\u00a0The experiment with the SMK photographs was based on the PixPlot software developed by Yale University\u2019s centre for digital studies, the Yale Digital Humanities Lab.https:\/\/github.com\/YaleDHLab\/pix-plot\/pull\/64\/files<\/a> produced by the Yale University Library Digital Humanities Lab, YaleDHLab.\n\">25<\/a><\/sup>\u00a0The actual data visualisation, PixPlot \u2013 SMK<\/em>, was carried out by the Center for Humanities Computing Aarhus, and this article addresses the first version of the visualisation.Visualizing Image Fields (smk.dk)<\/a> and Center for Humanities Computing Aarhus (chcaa.io)<\/a>\n\">26<\/a><\/sup><\/p>\n Developed in 2017, PixPlot was inspired by the seminal paper \u2018Visual Patterns Discovery in Large Databases of Paintings\u2019 presented at Digital Humanities Krak\u00f3w<\/em> i 2016.Digital Humanities 2016: Conference Abstracts<\/em>, Krak\u00f3w 2016, pp. 169­–172.\n\">27<\/a><\/sup>\u00a0Back then, artificial intelligence in the form of a so-called convolutional neural network (CNN) had been successfully tasked with visualising patterns in database images of paintings. In terms of performance, this so-called deep learning approach is still recognised as one of the most effective methods for studying image datasets. The Yale Digital Humanities Lab worked on further developing this way of analysing large quantities of photographs, but instead used the more sophisticated CNN network Inception V3, which had been developed as open-source software, mainly by professionals from Google.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition<\/em> (CVPR), 2016, pp. 2818–2826.\n \n\">28<\/a><\/sup>\u00a0With PixPlot, they were successful in further compressing the results from this network, so that they could be presented in a two-dimensional interactive visualisation. The objective was to produce a tool for exploring large collections of digitised visual data. The first PixPlot visualisation was based on a data set comprising approximately 27,000, primarily older digitised photographs from the Beinecke Rare Book & Manuscript Library. Since then several other collections at Yale have been successfully visualised, including the Yale Centre for British Art and The Medical History Library. In this way, PixPlot has helped to promote this type of visualisation for a wide field of images.<\/p>\n Others have also used PixPlot to explore image sets of cultural heritage objects. Two examples are shared on the PixPlot page on the Github platform. In the first example, PixPlot was used to visualise 24,026 photographs from the Bain News Service covering the years from approximately 1910 to 1912, while the second example involves 31,097 photographs from various Norwegian cultural institutions disseminated on the website oslobilder.no<\/a>. The German research project \u2018Training the Archive\u2019 also used PixPlot, focusing particularly on the curatorial possibilities offered by visualisations.https:\/\/github.com\/DominikBoenisch\/Training-the-Archive<\/a>\n\">29<\/a><\/sup><\/p>\n While other models exist, such as VGG16, ResNet50 and Xception, PixPlot was an obvious choice for our experiment, partly because Inception V3, on which PixPlot is based, is recognised as an excellent model for classifying images. What is more, PixPlot has been thoroughly tested, and implementing the model is relatively efficient. The resulting visualisation has an interactive and visually appealing interface which is simple to use. While other models could have provided other visual constellations of the data set, studying such nuances was not the intention here. If one wants new variations of the visualisation, it would make more sense to instead adjust and rebuild PixPlot \u2013 SMK. Art historians need to actively use these tools and get to know them in order to identify needs that the computer might be able to meet. In that process, adaptations or even the choice of other models may become relevant, especially as the technological possibilities continue to evolve.<\/p>\n The objective of analysing a large amount of photographs of artworks by means of the computer was to investigate how<\/em> it would group the photographs and thereby define their visual similarities and relationships. The PixPlot \u2013 SMK project did not predefine which images were visually related: the computer was left to form its own categories of images based on the similarities it found. Metadata about the artefacts, such as artist names and dates, were not part of the computational analysis at this stage.<\/p>\n The first step was to find answers to some general questions, for example by analysing the visualisation visually and subsequently increasing the complexity of the questions until the image cloud would need revision in order to answer them, for example by adding additional layers of information to the computer analysis.<\/p>\n Our first question was whether it was possible to outline some of the visual features that determined how the computer classified the artworks. This is to say that our efforts focused on looking at the visualisation as a result of the computer\u2019s actions and examining what governed the formation of the individual clusters.<\/p>\n Can these traits be described and defined, and might one possibly be able to find examples that were difficult for the computer to place? Does the computer sort images differently from how a human being would typically do it? Can the answer from a computer to what constitutes visual similarities in works of art inspire us in our study while also telling us more about the processes of algorithms? In the field of computing, the end result of the relatively closed computer analyses is often examined to gain insight into how one might further work with the algorithms. However, closer visual analyses of macro-visualisations of art and cultural artefacts are rarely found in the available research literature.<\/p>\n One of the fundamental skills of the art historian concerns visual analysis and the ability to interpret and contextualise visual phenomena. As an object of analysis, the PixPlot visualisation of photographs of artworks poses a challenge \u2013 what exactly can one say here? Resembling a cloud formed by images, the visualisation consists of thousands of photographs of artworks, each with their own visual idiom [Fig. 1]<\/strong>. The cloud consists of individual photographs placed near other photographs with which the computer has identified a degree of visual affinity. Whereas the individual photographs point back to a range of artistic intentions, contexts and messages in the original object, the image cloud instead is machine-produced, presented without explanation. It appears to be without intentions, and information on the choices behind the composition of the cloud is limited. What is more, technical errors in the coding may also affect the results, governing parts of the cloud\u2019s figuration \u2013 and in such cases a computer specialist is required to see through the errors, both in the code and in the interface.<\/p>\n In PixPlot \u2013 SMK, the 40,000 photographs of artworks from the museum are reproduced as thumbnails in a cloud formation of shallow depth. The depth allows users to see the stacks in which images are arranged when the computer judges them to be very similar. The cloud condenses around a centre while a series of arms and islands extends from this centre of gravity. As was previously mentioned, the images are distributed in so-called clusters according to the algorithmic assessment of their visual relationships. These clusters are listed with a row of images on the left side of the interface, showing each cluster with an image selected by the computer as an ideal example. The individual clusters are numbered, but they do not automatically have any content description or a so-called label. This must be added manually.<\/p>\n The following sections on analysis present selected analytical angles that can be generally applied, and offer some suggestions on what should, at a minimum, be examined in this kind of macro-visualisation.30<\/a><\/sup><\/p>\n In our analysis of the image cloud, we tended to look for patterns and similarities between images, yet at the same time the things that stood out and seemed unusual caught our attention. As a consequence, we would take turns spotting what we have called transitions, outsiders and centres of gravity. It can be useful to be aware of which of these aspects one would like to investigate. It can also make sense to review and analyse the aforementioned series of clusters defined by the computer. By studying the characteristics of the reproductions of artworks and the relationships between them, signs of the computer\u2019s actions \u2013 and the reasons for them \u2013 are also analysed.<\/p>\n During the analysis process, we began to form a sense of the composition of the image cloud, meaning that we gradually grew better at predicting in which cluster a particular photograph would be found. We verified this by selecting artefacts in the database and then trying to find them in the cloud. This was possible to achieve as long as the images were not part of the large, vaguely defined beige clusters. Here, studies of details would be required to find the similarities, but overall the computer often has more difficulty in analysing line drawings than, for example, painting. The overlap between photographs also makes it difficult to closely study the content of the individual clusters.http:\/\/vy.nasjonalmuseet.no\/?lang=en<\/a> For an introduction to the project, see Prosjekt: «Prinsipalkomponenter» – Nasjonalmuseet<\/a> Also see Google Arts & Culture Experiment Freefall where four different viewing options (Big Bang, Sphere, Waves and Timeline) arranges the many thousands of images. The number of photographs is impressive, but the graphic layout and dynamism, especially the Waves option, soon becomes uninteresting.\n\">31<\/a><\/sup><\/p>\n SMK\u2019s photo documentation comprises both black-and-white and colour photographs, and a large proportion of them depict works on paper in predominantly light beige shades. In the PixPlot experiment, we were interested in seeing whether the computer would make the simple move of doing an initial filtering by colour scale and black-and-white reproductions. That particular approach is evident in Yale\u2019s PixPlot visualisation of a collection of primarily black-and-white photographs from the nineteenth century, where the cool grey and yellowish white tones of the photo paper proved decisive for how the computer sorted the material into two major clusters.https:\/\/dhlab.yale.edu\/projects\/pixplot\/<\/a>\n\">32<\/a><\/sup>\u00a0This means that other possible relationships in the subgroups could theoretically be weighted as less significant, causing these traits to be split up on each side of the overall sorting. This may be unhelpful depending on the purpose of the visualisation. The basic tone of the paper may be a relevant filtering factor when studying issues of distribution or provenance. But the question is, then, whether PixPlot is sufficiently accurate as a tool. It may therefore be relevant to examine what weight seems to be given to the individual visual features and what consequences this may entail. In the PixPlot visualisation of the SMK images, the light-coloured works on paper dominate, forming large, coherent clusters, but closer inspection reveals that the complexity and compositions of the diverse subjects depicted were just as crucial for the filtering process as the basic tonality of each photo. For example, a large cluster of paintings have been collated in the visualisation, even though they were depicted in both black-and-white and colour photographs. This suggests that the nature of the compositions rather than colour was a decisive factor in the computer\u2019s analysis [Fig. 2]<\/strong>. There are several examples of brightly coloured artefacts standing out prominently from the surrounding light-coloured works on paper [Figs. 3a,\u00a03b, 3c]<\/strong>. In those cases, perhaps the simple composition is responsible for the inclusion of the coloured image in the relevant cluster. In another example, it appears that a square shape in the image determines its placement rather than its colour and contrasts [Fig. 4]<\/strong>. This is to say that the Renaissance dispute between disegno <\/em>and colore <\/em>is repeated in the digital realm.<\/p>\n Looking at other examples of macro-visualisations of reproductions of artefacts in which colour plays a significant role, the Google Arts & Culture Experiment has conducted some of the most comprehensive projects. Using advanced machine learning, they have carried out two visualisation experiments with photographs of artworks, Curator Table and t-SNE Map, both involving a clear sorting by colour.https:\/\/artsexperiments.withgoogle.com\/curatortable<\/a> and https:\/\/artsexperiments.withgoogle.com\/tsnemap\/<\/a>. See also https:\/\/artsexperiments.withgoogle.com\/freefall\/random<\/a> in which the individual thumbnails appear to act more as fillers in various more or less dynamic formations.\n\">33<\/a><\/sup><\/p>\n In Curator Table, the images form a square containing an extensive hilly landscape, while the t-SNE visualisation has an open edge and a more uneven distribution. Both display the artworks in patches of similar hues. Up close, there are clear subgroups, such as jewellery photographed against a black background or blue-painted porcelain, clearly showing the importance of colour as a filtering parameter. On the other hand, it is difficult to see the reasoning behind some of the transitions between the various patches of colour. For example, it is often not clear whether there are visual connections in terms of subject matter or form between a cluster of dark images and an adjacent group of light images. Has the computer identified the two clusters separately, and is it just a coincidence that the two groups end up next to each other in the visualisation? Other transitions have obvious formal similarities; for example, one may can find circular shapes appearing across changes in colour.<\/p>\n Colour sorting can be helpful if you are looking for certain artworks \u2013 particularly if they feature bold, distinctive colours. In other cases, it would be more useful to be able to see structures across the colours, thereby dialling down the dominance of colour and give other visual features greater weight.CHI ’11, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems<\/em>, 2011, pp. 1215–1224.\n\">34<\/a><\/sup>\u00a0In any case, it is often relevant to consider the relative weighting between form and colour when training your algorithms based on a specific purpose.<\/p>\n The perimeters between the artefact and its surroundings are typically marked by the frame and framing features such as passe-partouts, the shape of the artefact and border decorations, especially when it comes to artworks such as painting, drawing and prints. In art history, these features have often been perceived as being of secondary significance and value, while the weighty, meaning-bearing elements are to be found in the subject matter, which typically holds a central position. By contrast, a computer conducting a visual analysis is often more preoccupied with framing features than with what we regard as the main subject matter. For example, an oval format or a white passe-partout will often be the defining feature of a given image in the computer analysis, regardless of subject matter.<\/p>\n Thus, one of the tasks facing us today is not only to get the algorithms to recognise structures and patterns and filter the images by them, but also to make them ignore certain visual formal features that we might not be interested in filtering subjects by.<\/p>\n In this context, it should be added that in PixPlot \u2013 SMK, photographs of three-dimensional objects such as books and sculptures tend to appear in clusters with frame-like features, perhaps because the surroundings included in the photograph are interpreted as passe-partouts or frames [Fig. 5]<\/strong>. In other sections of the image cloud, the subject is given greater weight than clear framing features. For example, a portrait is placed in the portrait cluster even though its gold frame might just as well have delegated it to the group of images where the frame is the common feature regardless of subject. Elsewhere, a cluster of photographs of drawn portraits are grouped according to the common trait of being upside down. This prevents them from being grouped with related portraits that are turned the right way around. The example is a case of the computer not having been made aware of conventions which assist humans in their processing of images. If, on the other hand, you ask the computer to correct specific errors that have been found in the dataset, for example to remove gold frames that were not intended to be included, or to turn portraits around, it can \u2013 once the code is written \u2013 typically perform these task much faster and on a larger body of material than a human being can.<\/p>\n As we study the cloud of images, we come across a black-and-white photograph of a baptismal font. It reminds us that a photograph can itself be a work of art from the collection \u2013 or a means to register a given artwork. For the computer, there is no difference, and in several respects photographs of three-dimensional objects require greater care when used as data sets, especially if you wish to include them in purely visual computer analyses like this where metadata has no role. Artworks such as paintings and prints are also three-dimensional and material objects with many features that are omitted in photographs, such as their backs, edges and the shape of the surface, but the issue of omission is even more pronounced in artefacts such as sculptures or installations. Usually, such objects cannot be fully and adequately captured from just one angle. The setting of the work will often be included in the photograph, and the choice of angle, cropping and surroundings can radically change the formal features and composition of the image. It thus becomes unclear what the computer has in fact been asked to analyse as regard the image and the depicted work of art. If, on the other hand, one wanted to examine a particular style of photographing artworks and how it has changed over time, for example focusing on photographs of Bertel Thorvaldsen\u2019s (1770\u20131844) sculptures, the task is so clearly defined that the computational visual analysis might well make sense.35<\/a><\/sup>\u00a0In such a case, the photograph itself and its formal logic becomes the focus of the analysis rather than the depicted work of art. This example illustrates how it is necessary to try out, but also to be inspired by the technological tools in order to adapt the tools and research questions in a process of mutual exchange.<\/p>\n In PixPlot \u2013 SMK, photographs of sculptures and installations are relatively frequently found among other artefacts with which they share certain similarities. One example is a photograph of the installation Eight-Part-Piece<\/em> by Robert Smithson (1938\u201373), its light beige colours and geometric shapes resembling the works on paper that surround the photograph in the PixPlot visualisation [Fig. 6]<\/a><\/strong>. By contrast, a series of photographs of the installation Ritual II<\/em> by Tonny H\u00f8rning (b. 1941), an installation reminiscent of a church interior, is judged to be so distinctive and particular that the computer ends up allocating them to an isolated cluster [Fig. 7]<\/strong><\/a>. This surprised us, as the photographs might – with their simple geometric shapes and brown-beige contrasts \u2013 just as well have been placed among the prints. One possible reason may be that the network was trained using images from ImageNet, which include churches and church interiors. As a result, the network may recognise similarities between the installation and this theme, prompting it to establish a separate cluster despite the visual similarities between the photo reproduction and the fine-art prints. If you needed to get the computer to distinguish between church interiors and gallery spaces, you could in principle train the network to do so with a training set where the visual differences between the two types of space are manually defined (supervised learning, see later). In carrying out such a task, one main issue would be whether the gallery space can be defined with sufficient visual clarity and distinctiveness to ensure that the definition would bring together installations and other three-dimensional objects, including the photographs of H\u00f8rning\u2019s work. Whether it is possible establish a set of visual markers clear enough for the computer is one of the issues to be considered if you want to fine-tune a network to filter out images based on specific visual criteria. But this is also where the computer can potentially surprise us, for example by highlighting or linking artefacts in thought-provoking ways.<\/p>\n As the two examples provided above illustrate, outsiders can be found inside clusters or be isolated images placed far away from the others. Some outsiders can be truly puzzling. For example, the predominantly green painting Cosmic Landscape<\/em> by Egill Jacobsen (1910\u201398) is placed entirely on its own next to a cluster of landscapes which it does not, according to the computer, sufficiently visually resemble to become a part of \u2013 although this does not mean that it ends up in one of the non-figurative clusters or is pulled close to other photographs of predominantly green artworks [Fig. 8]<\/strong>. Perhaps Jacobsen\u2019s image is also similar to the arrangement of objects that constitute the closest cluster in the opposite direction. Might this indicate that in the computer\u2019s analysis, this image is poised between two visual centres of gravity that pull in different directions, leaving it suspended on its own? Or is the image really so special that it should not have any close neighbours?<\/p>\n The outsiders, whether integrated or solitary, can sometimes be used by computer specialists to identify errors in the code. In other cases, outsiders can be used to train the network, nudging it in a specific direction and thereby acting as keys to adjusting the analysis process. By giving the system feedback on its choices, you can increase a specific kind of accuracy that is simultaneously defined by such feedback. You can always carry out the analysis again, change the composition of the cloud and promote nuances in the computational analysis. For example, a computer can be trained to recognise the style of a given artist, which could in principle enable it to group together their artworks in the cloud despite their differences in subject matter. In another type of task, algorithms have been trained to recognise structures in paintings by Frans Hals (c. 1582\u20131666), upon which the trained network has been used to detect areas in a selected painting in which hands other than Hals\u2019s have contributed brushwork.NICAS project week<\/em> by Robert Erdmann, 1 December 2021: https:\/\/youtu.be\/Cw5Lan4mhbA<\/a> See also the reconstruction of the missing parts of Rembrandt’s The Night Watch<\/em> by means of artificial intelligence and a contemporary copy of the complete painting: https:\/\/www.rijksmuseum.nl\/en\/stories\/operation-night-watch<\/a>\n\">36<\/a><\/sup>\u00a0Seen from this perspective, no categorisations are erroneous until one defines what is and is not an error. Once again, it is relevant to consider the similarities and differences between how humans and machines analyse the images. Similarly important is whether you want to guide the computer in a certain direction, such as to recognise a given style or a particular choice of subject matter \u2013 that is to say that it should learn to mimic human image recognition and traditional approaches to sorting and filtering images. Within the field of self-driving cars, a key effort is to have their navigational system mimic how humans recognise objects by feeding it images. Building on the connoisseurship tradition, various milestones have been reached by the so-called technical art history and conservation by supplementing the human ability to study similarities and differences in the visual and physical structures of the artefacts.21ST-CENTURY CONNOISSEURSHIP – NICAS (nicas-research.nl)<\/a>. See also the interview with Erdmann by Lisbet Tarp in the forthcoming issue Det r\u00e6sonnerede v\u00e6rkkatalog – dets genkomst og betydning i den digitale tidsalder<\/em> at Passepartout – skrifter for kunsthistorie<\/em>.\n\">37<\/a><\/sup>\u00a0If the computer could become an optimised analysis partner for art historians, this could have an impact on the methods and processes of art history as a discipline.<\/p>\n The question is, then, whether the more significant impact would be if the computer could challenge us and offer us alternative analyses of the material that might become relevant to us as human beings. For example, we might be able to obtain a new, relativized understanding of what visual affinities between images can be. Or we might find that hitherto unrecognised connections in our image culture become visible to us. Perhaps the reinforcing effects of the computer on embedded value judgments introduced with the training set might even highlight aspects of our culture that should be taken up for reassessment, for example subconscious visual expressions of misogyny or racism. Seen from another angle, computational work with images is also about how its various analyses feed off of and play into our society \u2013 often without the image becoming visible at all. To this end, artists have long worked with the social aspects of human\/machine interaction and the opportunities in developing artefacts with aesthetic and critical agendas.AI & SOCIETY<\/em>, 14 September, 2020. https:\/\/doi.org\/10.1007\/s00146-020-01059-y<\/a>\n\">38<\/a><\/sup>\u00a0In the context of this article, it could be that computational and algorithmic ways of working with cultural heritage documentation may in time affect and shift some of the fundamental hierarchies in art history: which artworks are most important to our culture? On what values \u200b\u200bare the judgments of art history based? What concepts of art or culture are applied by individual cultural institutions? Exploring this would also make it possible to render visible the constructed boundaries between visual\/material culture and traditional art forms such as painting, drawing, sculpture and architecture.<\/p>\n A very simple factor here, already clearly evident in PixPlot \u2013 SMK, concerns how the computer can visualise and highlight artworks and groups of artefacts that otherwise receive little or no attention. Making greater use of such an approach to the collections and obtaining greater knowledge of the less frequently treated and recognised empirical data could potentially create a shift in the fields of interest and hierarchies found in art history. As we will discuss later, this is also about implicit and perhaps subconscious value judgments, an issue which is a major topic of discussion when considering the application of artificial intelligence in society in general.<\/p>\n <\/p>\n Another analytical perspective on the image cloud concerns identifying and examining centres of gravity and transitions between the clusters. Gravity centres comprising relatively similar images are to some extent defined by the clusters illustrated in the bar to the left of the interface (clusters 1 \u2013 20), but many subgroups do not appear there. Busts and artefacts with a circular composition are very clearly gathered in separate clusters, but here the way in which the objects are photographed is also relatively uniform [Figs. 9a, 9b]<\/strong>. Some photographs of three-dimensional objects are also grouped together despite the very different formal qualities of the actual artefacts [Fig. 10]<\/strong>.<\/p>\n The gradual transitions in the cloud may be relevant to study if one is working with large groups of very uniform material. It can provide useful impressions of variants of similar visual features and subject categories. In the large clusters of works on paper, for example, we found two transitions: one transitioned from a loose\/restless drawing style to a more controlled, thinner line, while the other transition went from framed artefacts to artefacts with a white passe-partout. We also discovered other types of transitions governed by subject matter. One example concerns a transition from naked to clothed human bodies; another transitions from human bodies to images of carcasses and meat; and yet another concerns a shift in shades of colour between light-coloured, uniform photographs. In one place, a succession of portraits ended up in an image with no human figure but with a lace-like pattern [Fig. 11]<\/strong>. This may point back to the fact that the sitters in the portraits are dressed in various types of ruffles and lace. Here, the computer analysis may have been distilled out a single feature.<\/p>\n These examples involve easily recognisable motifs and formal features, and when one finds a centre of gravity \u2013 for example a cluster of images of mammals \u2013 one can also spot examples strewn around a centre of gravity, intertwined with other forms of subject matter. Here, however, it also becomes clear that you run the risk of seeing connections that may have arisen quite randomly rather than as a direct result of the computational analysis. All the clusters mentioned include exceptions and deviations from what one would typically see as the defining feature.https:\/\/graph.chcaa.au.dk\/output\/#<\/a>\n\">39<\/a><\/sup><\/p>\n In those cases where the clusters formed in PixPlot \u2013 SMK were too mixed for us to put our finger on any common features, a sense of doubt arose: are we reading too much into what we see as connections between artefacts that have been grouped together? Did what we see as connections arise randomly as outliers on the peripheries of adjacent centres of gravity? Do we overlook the visual similarities with which the computer actually works? At the same time, such nagging doubts can also spread to the observation of more obvious groupings where one was initially quite certain. For example, does the computer recognise that a given picture shows a horse, or do completely different features govern the choice to group the horse pictures in clusters? Such doubts arise out of the fact that our theories on which visual features the computer has used to defined individual clusters cannot be conclusively read from the visualisation \u2013 it contains no documentation for the individual choices. For example, it seems obvious that the computer has used the round shape as the defining feature when grouping photographs of coins, but it would require a major technical manoeuvre to confirm this. The algorithmic process is not yet fully transparent\u2013 some call it a black box, and in this respect one often resorts to educated guesses. Furthermore, the visualisation is also an interpretation on a technical level: the computer compresses its multidimensional comparisons of photographs of artworks to create a visualisation that can be displayed on a screen in a simulated three-dimensional format.40<\/a><\/sup>\u00a0As we describe later, the artefact is indeed reduced when converted into numerical values, but the individual pixels and their relationships are also analysed in completely different ways by computers than by the human eye.<\/p>\n While working with PixPlot \u2013 SMK, we saw opportunities for 1) forming an overview of a very large quantity of image material; 2) contributing to critical thinking and reflection on image analysis and theory when computers are involved in the processes; 3) prompting fundamental discussions about methods, empirical evidence and theories found in the field of art history; and 4) considering consequences of non-human views of visual phenomena. Not all aspects can be covered here, but we will briefly say a few words about how the computer strengthens our awareness of processes, as we believe that efforts relating to this particular competence could strengthen art history as a discipline.<\/p>\n PixPlot \u2013 SMK was not developed to solve a specific task, and as yet we have not formulated any specific possible applications at SMK. Overall, the use of image clouds has not really taken off in the field of art history, but we still sense that visualisations such as PixPlot \u2013 SMK have potential in connection with learning and development processes in research, curation, teaching and dissemination.Digital Humanities Quarterly<\/em>, 2017. Regarding working with timelines, see Olivia Vane: Timeline design for visualising cultural heritage data<\/em>, PhD dissertation, Innovation Design Engineering, Royal College of Art, 2019. The Met and British Museum have had timelines made: Heilbrunn Timeline of Art History | The Metropolitan Museum of Art (metmuseum.org)<\/a> and Museum of the World (britishmuseum.withgoogle.com)<\/a> See also the Training the Archive<\/em> project mentioned in note 31.\n\">41<\/a><\/sup><\/p>\n In purely practical terms, academics may need alternative ways of building charts and aides memoire<\/em> of their empirical data and of illustrating the structures they see \u2013 or they may simply need the provocation offered by the computer analysis. Data visualisation can also be used as the starting point for developing new ideas \u2013 perhaps even in collaboration with others, where such visualisations can support communication and offer ways of illustrating one\u2019s thoughts. In teaching, for example, the image cloud can be used to help students find and build narratives based on photographs of artworks. But the act of working with a data set and its visualisation also has its own cognitive effects and learning potential. This has been documented in a concrete teaching project where computer science students and art history students were brought together. They were to work on processing and visualising a data set on Renaissance humanists, and Elizabeth Honig et al. have summed up the experience as follows: \u2018The shuttle back and forth between quantifiable data and humanistic inquiry through data and its visualisation was a hugely fruitful exercise\u2019.The Journal of Interactive Technology & Pedagogy<\/em>, no. 18, 2020, the section ‘Interpreting the Visualized Data’. Honig et al. includes this quote on the basis of Johanna Drucker, ‘Humanities Approaches to Graphical Display’, Digital humanities quarterly<\/em>, vol. 11, no. 1, 2011. See also Drucker’s other publications on digital visualisations of data in the humanities, e.g. Drucker, Johanna: Visualization and Interpretation. Humanistic Approaches to Display<\/em>, The MIT Press, 2020.\n\">42<\/a><\/sup>\u00a0The iterative processes yielded a special understanding of the data sets as well as of academic and technical requirements, and we derived some of the same lessons from working with PixPlot \u2013 SMK. From an art historian\u2019s point of view, this provides a basis for entering into conversations with computer specialists on how computer analysis, data sets and visualisation might be further developed \u2013 or it can throw up ideas on how new data visualisations and research questions could be designed.<\/p>\n Our approach to establishing the image cloud was, thus, exploratory and aimed at reflection-in-action.N\u00e5r nogen ler, er der noget p\u00e5 spil. En unders\u00f8gelse af designprocessen som social konstruktion herunder af humorens betydning for kreativiteten i designprocessen<\/em>, PhD thesis, Arkitektskolen Aarhus, 2011. See p. 74ff. on design processes and reflection-in-action. One limitation of the digital methods may be that in many cases, no hypotheses have been developed and that the hypotheses lack clear demarcations, meaning that the expectations for the possible outcome have not been clarified. Lisbet Tarp is grateful to Mette Volf for her collaboration on teaching and for rewarding conversations about the relationships between design processes and research development. For art historical development work in relation to digital visualisations, see Elizabeth Alice Honig et al.: ‘Thinking Through Data in the Humanities and in Engineering’, The Journal of Interactive Technology & Pedagogy<\/em>, no. 18, 2020 and Langmead et al. 2020. \n\">43<\/a><\/sup>\u00a0Viewed in this light, the image cloud and other digital experiments need not constitute results in themselves: they can offer scope for reflection or constitute stepping stones for reaching new places.<\/p>\n Our next step is to add additional layers of information to PixPlot \u2013 SMK, such as metadata or broader contexts associated with the photographs of the artworks, ensuring that social and cultural dimensions are accessible in the analysis efforts.44<\/a><\/sup>\u00a0PixPlot \u2013 SMK will also be linked to data sets from other collections in order to follow the changes in the configurations of the visualisation. Doing so will offer a different way of studying and comparing the development of collections (for example collections in the Nordic countries), their constituents and the collection patterns. At the same time, we need to take a closer look at those clusters which have already yielded materials and connections that can be used in entirely different studies, for example on early modern depictions of natural objects.<\/p>\n The next challenge is that working with the computational results requires a degree of flexibility when interacting with any visualisations, and one may need to regularly adjust the computer analysis or data input. In this regard, art historians will often need to rely on professionals from other disciplines and on the completion of often time-consuming manual tasks, for example on rewriting code or digitising material. In the long term, we hope to see the arrival of user-friendly software that enables users to upload their image material and have macro-visualisations of various kinds generated \u2013 without the need for any coding skills.<\/p>\n Digital art history is poised between several professional disciplines, and if one wants to get involved in the field, it is necessary to \u2013 at least partly \u2013 learn the specific professional terminology associated with computer science and the digital humanities. Digital art history projects often involve several disciplines, and the various participants need to communicate about goals, process and results.International Journal for Digital Art History<\/em>, no. 3, 2018, pp. 152-180. In Baca, Helmreich and Gill 2019, emphasis is also placed on the importance of developing collaborative skills when working with digital art history.\n\">45<\/a><\/sup>\u00a0Accordingly, the following sections address some technical concepts relevant to anyone wishing to understand the computer\u2019s visual analysis. Along the way, points from the work with PixPlot \u2013 SMK and other relevant projects with art historical aspects are raised.<\/p>\n Artificial intelligence (AI) is about making the computer mimic (and surpass) human cognitive processes and reasoning, for example when playing chess or engaging in dialogues. Basically, artificial intelligence consists of algorithms, meaning sequences of code that describe a series of computational operations capable of processing a dataset or performing some task. These algorithms can be built in so-called artificial neural networks \u2013 a term inspired by descriptions of the biological processes in the brain. The analysis process takes place in layers of neurons, each of which assesses the input based on different properties. These deeper networks (known as convolutional or deep neural networks) are increasingly able to deal with complex issues and find subtle patterns not noticed in human analysis.Neural networks and deep learning<\/a>\n\">46<\/a><\/sup>\u00a0For reasons of clarity we only employ the terms computer<\/em> and algorithms <\/em>here, as artificial intelligence is a major field with many application areas and sub-domains, including machine learning and deep learning.Deep learning<\/em>, The MIT Press, 2019. \n\">47<\/a><\/sup><\/p>\nDanish digital art history?<\/h2>\n
The project behind this article<\/h2>\n
The digital image \u2013 the reduced artefact<\/h2>\n
Quantitative image analysis<\/h2>\n
Know your collection<\/h2>\n
Digital experiment featuring approx. 40,000 photographs of artworks<\/h2>\n
![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.1_0.jpg\")
The data visualisation SMK \u2013 PixPlot as an object of visual analysis<\/h2>\n
Navigation<\/h2>\n
![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.2.jpg\")
Form, colour and black-and-white<\/h2>\n
\n![](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.3a_0.png\")
<\/p>\n![](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.3b_0.jpg\")
<\/p>\n![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.3c_0.jpg\")
![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.4_0.jpg\")
Frames<\/h2>\n
![](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/03\/fig._5_skaermbillede_2021-11-12.png\")
Three-dimensional artefacts<\/h2>\n
Outsiders<\/h2>\n
![](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/03\/fig.8.png\")
![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.9a.jpg\")
![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.9b.jpg\")
Fig. 9b.\u00a0<\/strong>Screenshot of https:\/\/pixplot.smk.dk<\/a> (20-12-2020).<\/p>\nTransitions and centres of gravity<\/h2>\n
![](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/03\/fig.10.png\")
Doubts and black box<\/h2>\n
![\"\"](\"https:\/\/www.perspectivejournal.dk\/wp-content\/uploads\/2023\/02\/fig.11_0.jpg\")
Data visualisation as an iterative learning process<\/h2>\n
The computer as beginner: Learning process<\/h2>\n