LAST SUPPER INTERACTIVE [LSI]
by Franz Fischnaller

© 2023 Franz Fischnaller – All Rights Reserved

SUMMARY

The Last Supper Interactive [LSI] is an 8K-3D stereo Immersive virtual (VR) journey into Leonardo da Vinci’s Last Supper masterpiece. It is an interactive, immersive user experience showcased in multiple locations, utilizing the Deep Space system.
Venues include the Ars Electronica Center in Linz, Austria, the West Bund Museum in Shanghai, and a new Deep Space system inaugurated in a cultural complex in Hangzhou.

Visitors are virtually transferred and can interact in real time from the “front/in/inside” and behind the painting, exploring from different viewpoints, collocation, height, distance, and perspectives over the full 360° range in high-resolution zoom-in capabilities. LSI uses the highest resolution photograph in the world, which is made of the Last Supper painting by Haltadefinizione. Likewise, visitors are teleported into the historical and architectural scenario where Leonardo created his masterpiece: The Convent of Santa Maria delle Grazie in Milan, Italy. An architectural complex, including the Convent, the Church, a cloister, and its garden, is a UNESCO World Heritage Site today. The immersive nature of the LSI application enables users to gain deeper insights into the narrative, symbolism, perspective, composition, and techniques behind the Last Supper, enhancing their knowledge and appreciation of the masterpiece.

The Premiere has taken place during the Ars Electronica Festival 2023 in the Deep Space 8K at the Ars Electronica Center, Linz, Austria, Sept.5 -10th 2023.

LSI has debuted in Deep Space 8K. A spectacular 3D-8K stereo multi-user immersive projection space of 16×9 MT wall and floor projection with laser tracking, for interactive, stereoscopic, and high-definition content, can host up to 160 persons simultaneously. LSI will run until Sept. 2024 in the Deep Space 8K at the Ars Electronica Center at Linz, Austria.

Last Supper Interactive (LSI) opening: Deep Space 8K at the Ars Electronica Festival 2023
Photos: Any Tingay, Alberto Sanna, Daria Tsoupikova, Michael Tingay, Franz Fischnaller

LSI Press Release | Franz Fischnaller | LSI in Deep-Space 8K | About the Deep-Space 8K | ARS Articles

Ars Electronica Festival – 2023 “WHO OWNS THE TRUTH?”

MEDIA CONTACT: media@franz-fischnaller.com

NOTES: Gigapixel image of the Last Supper© Haltadefinizione® Image Bank
By concession of Ministero della Cultura – Direzione Regionale Musei Lombardia (IT)

INTRODUCTION

The LAST SUPPER INTERACTIVE (LSI) is an 8K-3D stereo Immersive interactive VR journey into Leonardo’ Last Supper and to the historical and architectonic scenario where the masterpiece has been created: the north wall of the refectory (Dining Hall) of the Convent of Santa Maria delle Grazie located at the Piazza Santa Maria delle Grazie (CH-SMGZ) in Milan, Italy. An architectural complex including the Convent, the Church, the cloister, and the garden. The painting was commissioned in 1495 by Leonardo’s patron Ludovico Sforza, Duke of Milan, as part of a plan of renovations to the church and its convent buildings and completed in 1497.

LSI will debut in the Deep Space (DS-8K-AEC ). A spectacular 3D-8K stereo multi-user immersive projection space of 16×9 mt wall and floor projection with laser tracking, for interactive, stereoscopic and high-definition content, that can host 90 persons simultaneously. The opening has taken place at the Ars Electronica Festival. Sept.5 -10th, 2023 – Who owns The Truth?

The Last Supper (Italian: L’Ultima Cena), Artist: Leonardo da Vinci, Year: 1495-1498 – Type: tempera on plaster, pitch and mastic – Dimensions: 460 cm × 880 cm (181 x346 in) – Location: Refectory of the Convent of Santa Maria della Grazie, Milan, Italy – Photographic reproduction from the original painting: Copyright 2006-2007 HAL9000 S.r. This Photographic reproduction was shot after the Last Super of Leonardo was restored 1978-1999 guided by Pinin Brambilla Barcilon

HIGHLIGHTS OF THE “MAKING OF”

Digital Media, scientific tools, Gigapixel imaging, 3D digital data acquisition, photogrammetry, laser scanning, CG 3D modeling

The LSI project uses the highest-resolution photograph in the world of the Last Supper painting by Haltadefinizione: a 21-billion-gigapixel image stitched together with 1042 panoramic images of the 460×880 cm mural painting.

8K stiched details of the Gigapixel image of the Last Supper© Haltadefinizione®

The Department of Mechanics of the Polytechnic of Milan carried out ad hoc a high-resolution 3D digital data acquisition by laser scanning (interior/exterior) of the architectural complex of the Dominican monastery of Santa Maria delle Grazie (SMG) (i.e., the monastery, the church, the cloister, the garden, and the Piazza)—never done before.

High-resolution 3D digital data acquisition by laser scanning (in-/exterior) – Pointcloud+3D model of the architectural complex of of the Dominican Convent of Santa Maria delle Grazie in Milan, Bramante Dome and Piazza Santa Maria delle Grazie at Milan, Italy

In addition, the Last Supper painting was digitally replicated with CG 3D models produced ad hoc, including the three-dimensional environment of the painting and surrounding architecture, the perspective structure and composition, and the room and ceiling in the back of the apostles.

CG 3D models digitally replicating the Last Supper painting and the 3D environment, the surrounding architecture, the three-dimensional compositions and multi-perspective pictorial space of the painting and the room and ceiling Leonardo painted in the back of the apostles

… where Christ is at the center aside of the surrounding groups of four, the 12 apostles with their body and hand expression in the exact location around the table, as depicted in the original giving like this the total immersion into the perspective Leonardo has depicted on the 2D surface of the wall-painting.

Different viewpoints of the CG-3D models digitally replicating Christ and the apostles figures represented in the original painting. © F. Fischnaller

3D graphics and immersive digital effects were implemented to visualize how Leonardo used the mathematical principles of linear perspective – parallel lines, the horizon line, and a vanishing point – to create the illusion of depth (3D space) on a flat surface, such as the Last Supper painting.

LINEAR PERSPECTIVE IN THE LAST SUPPER

Leonardo was a pioneer of the single-point (linear) perspective, a mathematical/geometrical system for creating an illusion of depth on a flat surface that employs parallel lines (orthogonal) that converge at a single vanishing point on the composition’s horizon line in a painting or drawing.

The Vanishing point is on the image plane of a perspective rendering where the two-dimensional perspective projections of mutually parallel lines in three-dimensional space appear to converge.

He is considered one of the masters of one-point perspective in world Art History. Leonardo used the linear perspective to create the Last Supper, a masterpiece known for the perspective the artist successfully employed. Every element of the painting directs attention directly to Christ’s head, right in the middle of the composition.

Studies, Drawing and 3D models simulating the refectory y (Dining Hall) of the Convent of Santa Maria delle Grazie, Milan, Italy, were Leonardo has created the Last Supper, a late 15th century mural of 4,6 m x 8,8 m painted between 1494 and 1498 on the north wall of the refectory by commission of Ludovico Sforza, Duke of Milan

Leonardo accomplished this by driving a nail into the wall at Jesus’ temple, the exact focal point of the painting. From the nail, he radiated string in various directions to help him see the room’s perspective and paint the other elements in the piece in a way that led the eye back to Christ.

ALBERTI’S THEOREM VIRTUAL TOOL (ATVT)

ATVT is an augmented virtual immersive interactive learning device inspired by Leon Battista Alberti’s rules of linear perspective (Costruzione Legittima). It was designed and implemented ad hoc for the LSI project and serves as an interactive tool for the user to experience in real time how Leonardo applied the linear perspective in the Last Supper.

ALBERTI’S THEOREM VIRTUAL TOOL (ATVT): augmented virtual immersive interactive learning device, inspired by Leon Battista Alberti’s rules of linear perspective, designed for the LSI project, serving as an interactive tool for the user to experience in real-time how Leonardo applied the linear perspective in his masterpiece

Viewpoint through Alberti’s Theorem Virtual Tool (ATVT)

The visitor can look through the alignment of the perspective lines of the tilt grid and experience the perfect matching of the linear perspective used for the construction of the painting in real-time.

Viewpoint through Alberti’s Theorem Virtual Tool (ATVT)Tracking the Linear Perspective: experiencing several alternative virtualized views and interactive navigation allowing visitors to experience the artwork from multiple viewpoints, co-location, distance, levels of exploration. VR Experience in the Deep Space 8K, in ARS Electronica Festival. © F. Fischnaller rendering

THEMATIC, DRAMATURGY, EMOTIONAL REALISM, GOLDEN RATIO

The Last Supper painting shows most bodily proportions using the Golden Ratio, a mathematical way to decipher artistic proportion. There are allusions that the number three (3) has mathematical symbolism, likely representing the Holy Trinity.

The disciples are seated in groups of three. There are three windows, and Jesus is depicted in a triangular shape with his outstretched arms, representing the famous biblical scene.

LSI is a dramatic, electric, human, and emotional scene. Each figure, through his body and facial expressions, expresses an emotion, creating a dynamic synergy of emotions, including shock, fear, surprise, denial, and anger. All the apostles are shocked by Christ’s proclamation.

… ” Bartholomew, James the Less and Andrew has his hands up to stop the words. Judas, Peter and John form the next group of three. Peter is angry and John seems faint, but Judas has his face in the shadow, but lost from Christ’s light – literally – as he is covered by a shadow. After Christ is Thomas, James the Greater and Philip, each agitated, stunned or confused, respectively. Matthew, Thaddeus and Simon form the last three, with the first two looking to the latter for an explanation”.

Christ, on the other hand, represents calm in the face of chaos. As the apostles run the gamut of emotions, Christ is in the center of the storm, peaceful and expansive, providing a perspective and emotional anchor for the scene. The objects around Christ have fallen over, representing the chaos when he is not around, while those in front of him are orderly. With his arms spread out, possibly representing the Holy Trinity, he points to the bread and wine before him, each a symbol of the moment Jesus will sacrifice himself.

INTERACTIVE STORYTELLING – USER EXPERIENCE

DIGITAL VISUAL NARRATIVE, SCENES, IMMERSIVE NAVIGATION, AND SOUND ART

The story departs from an elevated view of Corso Magenta, Milan, Italy at the level of the Piazza of Santa Maria Delle Grazie (P-SMGZ). Visitors can admire the exterior architecture …

Afterward, head in the direction of the Church of Santa Maria delle Grazie (CH-SMGZ) and enter through the portal. Once there, visitors can navigate, explore, and elevate themselves in the central church’s nave, aisles, and colonnades …

at the height of the transverse nave, visitors can turn left and exit to the cloister and the garden.

Visitors can explore around the Cloister and Garden build by Bramante

…  when getting close to the Refectorio’ door visitors are unexpectedly dragged into a vortex of point cloud in motion aligned in perspective with multilayer 3D graphic animations representing the construction of linear perspective concept – the arrangement of the parallel lines, the horizon line, and the vanishing point immersing the visitors within the illusion of depth in a 3D space.

A light beam radiates from the center projection and the horizon of the painting’s perspective, teleporting the visitors inside the refectory, home of Last Supper Painting (LS).

Inside the refectory, visitors can experience and explore the linear perspective concept from different viewpoints in real time and from different viewpoints of the paintings.

VIEWPOINTS

Viewpoint 1: Floor level at 6 Mt distance to the painting, 4.6 Mt below the Horizon Line, at the centre of the projection of the painting. The exact view visitors experience at the real site: the Refectory in Milano.

Viewpoint 2: Visitors are elevated on a virtual ramp 4.6 Mt above the floor level at 6 Mt distance to the painting, aligned straight to the Horizon Line at the center of the projection depicted in the painting.

Viewpoint 3: At 4.6 Mt above floor level and 3 to 0 Mt distance to the painting, visitors can explore virtually the painting in all its details in very high resolution zoom-in capabilities, up to a square millimeter of the 21-billion-Gigapixel.

Interactive experience

Viewpoint 4: At the height of 4.6 Mt above floor level aligned at the center point of the painting and its central perspective projection, where Leonardo has placed a nail to construct his perspective alignment. Visitors are swallowed up into the threshold of the painting and transported through a powerful, immersive dynamic tunnel, bringing the visitors inside the painting.

Visitors can keep navigating and get close to where Leonardo hammered the nail into the wall at Jesus’s Temple, the exact focal point of the painting.

Viewpoint 5: … to after plunge “inside””” the painting, a 3D virtual simulation, an environment replicating the Last Supper and the composition representing the instant when Christ announces the betrayal by one of his apostles before he was arrested and crucified.

Once there, visitors can navigate and interact in real time in 360, within the pictorial multi-perspective space of the painting’s geometrical composition… explore closely the body and gestural emotional expression of Christ and his 12 apostles in 3D … gathered around the table, when reacting in different ways, movements, expressions after Christ says “I tell you the truth, one of you will betray me.” One magnificent freeze-frame actually feels as if were a series of sequences in motion…

Viewpoint 6: Visitors can leave the dinner scene and enter the room behind Christ and the Apostles. (Which was digitally reconstructed in 3D), explore the view from the back of the painting and the “Center Point” and see the scene from inside out, toward the Refectory gaining the feeling of how the Refectory looks from “inside-out”. Thus, experiencing first hand and in an immersive manner the relationship between the authentic architecture of the Refectory and the painted architecture by Leonardo.

… Likewise, the visitors can see in the far back the virtual representation of the Crucifixion (1495) by Donato Montorfano, painted on the wall facing the Last Supper, to which Leonardo added the details of the portraits of the dukes of Milan …

… Visitors can after exit from that point to the one of the departure of Corso Magenta, Milan, Italy.

CONTENT VISUALIZATION, IMMERSIVE STORYTELLING, AND SOUND ART

LSI combines emerging forms of 8K immersive narratives with perspective in motion, VR dramatization, aesthetics, interactivity, and location awareness.

LSI blends multilayered digital visualization, immersive VR, simulation of three-dimensional environments, 2D/3D information, 3D point cloud visualization, 3D animation, stereoscopic optical illusions, multiple viewpoints, visual effects, music, and sound effects to augment visitors’ emotional immersion.

Focus was given to implementing novel approaches to immersive narrative to create a new generation of digital cultural heritage and ART experiences.

INTERDISCIPLINARY METHODOLOGIES

LSI intersects and uses methodologies from the traditional humanity disciplines and leading edge technologies and innovative tools.

LSI is an interdisciplinary project that combines and bridges traditional humanities disciplines, cultural heritage, humanities, and social science with art, design, science, leading technologies and advanced media tools, immersive technology, interactive techniques, and computer graphics.

Emphasis is provided on empowering the use and combination of experimental approaches to design and create novel forms of virtual storytelling and immersive narrative in the fields of the arts and digital heritage.

The LSI is the result of the author’s transdisciplinary research, knowledge creation, and strategic collaboration with professionals in the fields of Art, design, humanity, cultural heritage, science, and technology.

Visitors can then exit from that point and proceed to the departure point of Corso Magenta, Milan, Italy.

NOTE: References will be added

CREDITS

Author: Franz Fischnaller (IT); Development support & coordination: DIAC (IT); UNITY VR software Programming: Mathieu Godineau, Univ Rennes (FR); Prof. Gabriele Guidi, PhD. Indiana University. Department of Informatics – Virtual World Heritage Lab (USA)- Prof. Laura Micoli, Umair Shafqat Malik, PhD. Politecnico di Milano, and Department of Mechanics – Computer Vision and Reverse Engineering Lab (IT); Music & Sound: Steve Bryson (USA), Alfredo Miti (IT), Development by DIAC Design of Ideas – Architects of Culture (IT); © Haltadefinizione® Image Bank | By concession of Ministero della Cultura – Direzione Regionale Musei Lombardia (IT). Haltadefinizione is a Benefit Corporation directed and managed by Franco Cosimo Panini Editore S.p.A. (IT); Production support: Ars Electronica Center (A).

I want to thank Prof. Daniele Marini, who inspired me to initiate the project, and all the other institutions, companies, and people who have collaborated on it.

Thanks to Yecenia Maharaj Singh for making this project a reality.

Acknowledgment

The LSI Project has been possible to be developed due to the support and collaboration of government agencies, museums, industries, labs, foundations, universities, interdisciplinary teams and researchers, art collectors. We will like to thank all of them, including:

Università degli Studi di Milano, Italy;

Robotics srl, Italy; L’EMMECI srl, Italy;

Ars Electronica Future Lab AEC, Austria;

The National Museum of Science and Technology, ‘Leonardo da Vinci’ Italy;

CIVES Foundation and Agency Ville Vesuviane, Italy; the Virtual Archaeological Museum, Italy;

INSA Rennes, Institut National des sciences appliquées de Rennes, France, Université de Rennes 1, 35000 Rennes , France, INRIA, institut national de recherche en sciences du numérique, France, IRISA, Institut de Recherche en Informatique et Systèmes Aléatoire;

CINECA – InterUniversity Consortium, 40033 Bologna, Italy;

INP G-SCOP Laboratory, Industrial Engineering School, Institute Poly technique de Grenoble, France;

IMéRA, Institut d’Études Avancées, laboratoire MAP Marseille;

MuCEM, Musée des Civilisations de L’Europe et de la Méditerranée, Marseille, France;

F.A.B.R.I.CATORS, Milan, Italy;

MEDIARTECH Tuscany Region, Province of Florence, Italy;

Electronic Visualization Laboratory [EVL], School of Art & Design University of Illinois at Chicago, USA;

Tuscany Hi Tech Network, municipality of the region of Tuscany, Italy;

Medienmuseum ZKM|Zentrum für Kunst and Design, Karlsruhe, Germany;

High Performance Networks Laboratory, Bristol, Network Interaction Laboratory, University of Essex, UK, University of Bristol, Faculty of Engineering, UK;

EU-FP7 VISIONAIR -VISION Advanced Infrastructure for Research;

Marco and Luisita Angiolini, among others.

COMPLEMENTARY INFORMATION

FRANZ FISCHNALLER SHORT BIO (IT)

Interdisciplinary scholar, polymath, and practitioner (author, artist, designer, project manager) whose work spans arts, design, technology, science, ecology, digital humanities, and cultural heritage. FF’s artistic hybridism gravitates toward unconventional combinations of multiple perspectives and multi-layer creativity. His approach embraces a permeable interdisciplinary applied research methodology across intuitive interconnection and cross-fertilization through different art forms, disciplines, and shared practices with minded knowledge creation.

DEEP SPACE 8K – ARS ELECTRONICA CENTER (DS-8K-AEC)
DS-8K-AEC large-format projection space for interactive, stereoscopic, and high-definition content. A VR multi-user interactive storytelling of 16×9-meter wall and floor projections with laser tracking and 3D 8K resolution images that can host 90 persons simultaneously

In DS-8K-AEC, groups of up to 90 people can simultaneously experience a broad spectrum of visuals: artistic productions based on various imaging techniques, interactive narratives featuring new approaches to interaction, and art historical, architectural, and astronomical visualizations.
DS-8K-AEC serves as both an exhibition space for members of the public touring the AEC as well as an R&D platform for AEC Future lab staff. The program is articulated by several sections upon field and thematic, including to Art, culture, heritage, society, ecology, science, technology, design, architecture and digital heritage through explore classical statues, structures or entire streets from new angles thanks to 3D laser scanning technology

ARS ELECTRONICA FESTIVAL Ars Electronica is one of the world’s largest media art venues, a digital music festival, a showcase for creativity and innovation and a playground for the next generation – Ars Electronica is a world-class festival for art, Science, technology and society.

ARS ELECTRONICA CENTER (AEC) IN LINZ, AUSTRIA, ALSO KNOWN AS MUSEUM OF THE FUTURE, The Ars Electronica Center (AEC) is a centre for electronic arts run by Ars Electronica situated in Linz, Austria, at the northern side of the Danube opposite the city hall of Linz. It has been built on the right side of the Nibelungenbrücke. The building is also known as the “Museum of the Future”. Ars Electronica’s activities focus on the inter-linkages between art, technology and society. It runs an annual festival and manages a multidisciplinary media arts R&D facility known as the Future Lab. It also confers the Prix Ars Electronica awards.

LSI Website