LAST SUPPER INTERACTIVE [LSI]
by Franz Fischnaller

LSI short version Video Demo (7:13 min)

© 2023 Franz Fischnaller – All Rights Reserved

SUMMARY

LSI is an 8K-3D stereo Immersive interactive VR journey into the full picture, Leonardo da Vinci’s Last Supper. Visitors are virtually transferred, explore and interact in real time from the “front”, “in”, “inside” and behind the painting. As well as from different viewpoints, collocation, height, distance and perspective over the full 360° range in high resolution zoom-in capabilities. LSI uses the highest resolution photograph in the world made of the Last Supper painting by HALTADEFINIZIONE. Likewise, visitors are teleported into the historical and architectural scenario where Leonardo created his masterpiece: The Convent of Santa Maria delle Grazie, located in Milan, Italy. An architectural complex including the Convent the Church, a cloister and its garden, today a UNESCO World Heritage Site. The immersive nature of the LSI application enables users to gain deeper insights of the narrative, symbolism, perspective, composition and techniques behind the Last Supper enhancing their knowledge and appreciation of the masterpiece.

PREMIERE: Ars Electronica Festival 2023 – Deep Space 8K – Ars Electronica Center, Linz, Austria, Sept.5 -10th 2023

LSI has debuted in Deep Space 8K. A spectacular 3D-8K stereo multi-user immersive projection space of 16×9 MT wall and floor projection with laser tracking, for interactive, stereoscopic and high-definition content, that can host up to 160 persons simultaneously. LSI will run until Sept. 2024 in the Deep Space 8K at the Ars Electronica Center at Linz, Austria.

Last Supper Interactive (LSI) opening: Deep Space 8K at the Ars Electronica Festival 2023.
Photos: Any Tingay, Alberto Sanna, Daria Tsoupikova, Michael Tingay, Franz Fischnaller

LSI Press Release | Franz Fischnaller | LSI in Deep-Space 8K | About the Deep-Space 8K | ARS Articles

Ars Electronica Festival – 2023 “WHO OWNS THE TRUTH?”

MEDIA CONTACT: media@franz-fischnaller.com

NOTES: Gigapixel image of the Last Supper© Haltadefinizione® Image Bank
By concession of Ministero della Cultura – Direzione Regionale Musei Lombardia (IT)

INTRODUCTION

The LAST SUPPER INTERACTIVE (LSI) is an 8K-3D stereo Immersive interactive VR journey into Leonardo’ Last Supper and to the historical and architectonic scenario where the masterpiece has been created:the north wall of the refectory (Dining Hall) of the Convent of Santa Maria delle Grazie located at the Piazza Santa Maria delle Grazie (CH-SMGZ) in Milan, Italy.  An architectural complex including the Convent the Church, the cloister and the garden. The painting was commissioned in 1495 by Leonardo’s patron Ludovico Sforza, Duke of Milan, as part of a plan of renovations to the church and its convent buildings and completed in 1497.

LSI will debut in the Deep Space (DS-8K-AEC ). An  spectacular 3D-8K stereo multi-user immersive projection space of 16×9 mt wall and floor projection with laser tracking, for interactive, stereoscopic and high-definition content, that can host 90 persons simultaneously. The opening has taken place at the Ars Electronica Festival. Sept.5 -10th 2023 – Who owns The Truth?

HIGHLIGHTS OF THE “MAKING OF”

Digital Media, scientific tools, Gigapixel imaging, 3D digital data acquisition, photogrammetry, laser scanning, CG 3D modeling

The LSI project uses the highest resolution photograph in the world made of the Last Supper painting by Haltadefinizione. A 21-billion- Gigapixel image, stitched together with 1042 panoramic images of the 460×880 cm mural painting.

8K stitched details of the Gigapixel image of the Last Supper© Haltadefinizione®

A high-resolution 3D digital data acquisition by laser scanning (in-/exterior) of the architectural complex of the Dominican monastery of Santa Maria delle Grazie (SMG) (i.e. the monastery, the church, the cloister, the garden and the Piazza – never done before – was carried out ad hoc by the Department of Mechanics of the Polytechnic of Milan.

High-resolution 3D digital data acquisition by laser scanning (in-/exterior) – Pointcloud+3D model of the architectural complex of of the Dominican Convent of Santa Maria delle Grazie in Milan, Bramante Dome and Piazza Santa Maria delle Grazie at Milan, Italy

In addition, the Last Supper painting was digitally replicated with CG 3D models produced ad hoc including the three dimensional environment of the painting and surrounding architecture, the perspective structure and composition, the room and ceiling in the back of the apostles …

CG 3D models digitally replicating the Last Supper painting and the 3D environment, the surrounding architecture, the three dimensional compositions and multi perspective pictorial space of the painting and the room and ceiling Leonardo painted in the back of the apostles

… where Christ is at the center aside of the surrounding groups of four, the 12 apostles with their body and hand expression in the exact location around the table, as depicted in the original giving like this the total immersion into the perspective Leonardo has depicted on the 2D surface of the wall-painting.

Different viewpoints of the CG-3D models digitally replicating Christ and the apostles figures represented in the original painting. © F. Fischnaller

3D graphic and immersive digital effects were implemented to visualize how Leonardo used the mathematical principles of linear perspective – parallel lines, the horizon line, and a vanishing point – to create the illusion of depth (3D space) on a flat surface, such as the Last Supper painting.

LINEAR PERSPECTIVE IN THE LAST SUPPER

Leonardo was a pioneer of the single point (linear) perspective. A mathematic/geometrical system of creating an illusion of depth on a flat surface that employs parallel lines (orthogonal) that converge at a single vanishing point, on the composition’s horizon-line in a painting or drawing.

The Vanishing point is a point on the image plane of a perspective rendering where the two-dimensional perspective projections of mutually parallel lines in three-dimensional space appear to converge.

He is considered one of the masters of one point perspective in world Art History. Leonardo used the Linear Perspective to create the Last Supper. A masterpiece known for the perspective the artist successfully employed. Every single element of the painting directs attention directly to Christ’s head, right in the middle of the composition.

Studies, Drawing and 3D models simulating the refectory y (Dining Hall) of the Convent of Santa Maria delle Grazie, Milan, Italy, were Leonardo has created the Last Supper, a late 15th century mural of 4,6 m x 8,8 m painted between 1494 and 1498 on the north wall of the refectory by commission of Ludovico Sforza, Duke of Milan

Leonardo accomplished this by driving a nail into the wall at Jesus’ temple, the exact focal point of the painting. From the nail, he radiated string in various directions to help him see the room’s perspective and paint the other elements in the piece in a way that led the eye back to Christ.

ALBERTI’S THEOREM VIRTUAL TOOL (ATVT)

ATVT is an augmented virtual immersive interactive learning device, inspired by Leon Battista Alberti’s rules of linear perspective (Costruzione Legittima), that was designed and implemented ad hoc for the LSI project, serving as an interactive tool for the user to experience in real-time how Leonardo applied the linear perspective in the Last Supper.

ALBERTI’S THEOREM VIRTUAL TOOL (ATVT): augmented virtual immersive interactive learning device, inspired by Leon Battista Alberti’s rules of linear perspective, designed for the LSI project, serving as an interactive tool for the user to experience in real-time how Leonardo applied the linear perspective in his masterpiece

Viewpoint through Alberti’s Theorem Virtual Tool (ATVT)

The visitor can look through the alignment of the perspective lines of the tilt grid and experience in real-time the perfect matching of the linear perspective used for the construction of the painting.

Viewpoint through Alberti’s Theorem Virtual Tool (ATVT)Tracking the Linear Perspective: experiencing a number of alternative virtualized views and interactive navigation allowing visitors to experience the art work from multiple viewpoints, co-location, distance, levels of exploration. VR Experience in the Deep Space 8K, in ARS Electronica Festival. © F. Fischnaller rendering

THEMATIC, DRAMATURGY, EMOTIONAL REALISM, GOLDEN RATIO

The Last Supper painting shows that most bodily proportions with the use of the Golden Ratio, a mathematical way to decipher artistic proportion. There are allusions that there is mathematical symbolism to the number of three (3), likely representing the Holy Trinity.

The disciples are seated in groups of three, there are three windows and Jesus is depicted in a triangle shape with his outstretched arms representing the famous biblical scene.

LSI is a dramatic, electric, human and emotional scene. Each figure expresses an emotion, his body and face expression, and creates a dynamic synergy of emotion among others: shock, fear, surprise, denial, anger. All the apostles are shocked by Christ’s proclamation.

… ” Bartholomew, James the Less and Andrew has his hands up to stop the words. Judas, Peter and John form the next group of three. Peter is angry and John seems faint, but Judas has his face in the shadow, but lost from Christ’s light – literally – as he is covered by a shadow. After Christ is Thomas, James the Greater and Philip, each agitated, stunned or confused, respectively. Matthew, Thaddeus and Simon form the last three, with the first two looking to the latter for an explanation”.

Christ, on the other hand, represents calm in the face of chaos. As the apostles run the gamut of emotions, Christ is in the center of the storm, calm and expansive, providing both a perspective and emotional anchor for the scene. The objects around Christ have fallen over, representing the chaos when he is not around, while those in front of him are orderly. With his arms spread out, possibly representing the Holy Trinity, he is pointing to the bread and wine before him, each a symbol of the moment Jesus will sacrifice himself.

INTERACTIVE STORYTELLING – USER EXPERIENCE

DIGITAL NARRATIVE, SCENES, NAVIGATION PATH, IMMERSIVE VISUALIZATION AND SOUND ART

The story departs from an elevated view of Corso Magenta, Milan, Italy at the level of the Piazza of Santa Maria Delle Grazie (P-SMGZ), Visitors can admire the exterior architecture …

… to  afterward  head in the direction of the  Church of Santa Maria delle Grazie (CH-SMGZ) and enter through the portal. Ones there the visitors can navigate, explore, elevate … explore  the  central church’ nave,  aisles and colonnades …

at the height of the transverse nave, visitors can turn left and exit to the cloister and the garden.

Visitors can explore around the Cloister and Garden build by Bramante

…  when getting close to the Refectory’s door visitors are unexpectedly dragged into a vortex of  point cloud in motion aligned in perspective with multilayer 3D graphic animations representing the construction of  linear perspective concept – the arrangement of the parallel lines, the horizon line, and the vanishing point immersing the visitors within the illusion of depth in a 3D space.

A light beam radiates from the center projection and the horizon of the painting’s perspective, teleporting the visitors inside the refectory, home of Last Supper Painting (LS).

Ones there inside the refectory visitors can can experience and explore the Linear Perspective concept from different viewpoints in real time and different viewpoints of the paintings.

VIEWPOINTS

Viewpoint 1: Floor level at 6 Mt distance to the painting, 4.6 Mt below the Horizon Line, at centre of the projection of the painting. The exact and same view visitors experience at the real site: the Refectory in Milano.

Viewpoint 2: Visitors are elevated on a virtual ramp 4.6 Mt above the floor level at 6 Mt distance to the painting, aligned straight to the Horizon Line at the center of the projection depicted in the painting.

Viewpoint 3: At the height of 4.6 Mt above floor level in 3 to 0 Mt distance to the painting visitors can explore virtually the painting in all its details in very high resolution zoom-in capabilities, up to a square millimeter of the 21-billion-Gigapixel.

Interactive experience

Viewpoint 4: At the height of 4.6 Mt above floor level aligned at the center point of the painting and its central perspective projection, where Leonardo has placed a nail to construct his perspective alignment. Visitors are swallowed up into the threshold of the painting and transported through a powerful, immersive dynamic tunnel bringing the visitors, inside the painting.

… visitors can keep navigating … get close were Leonardo hammered the nail into the wall at Jesus’temple the exact focal point of the painting.

Viewpoint 5: … to after plunge “inside” the painting, a 3D virtual simulation, an environment replicating the Last Supper  and the composition representing the instant when Christ announces the betrayal by one of his apostles before he was arrested and crucified.

Once there, visitors can navigate and interact in real time in 360, within the pictorial multi-perspective space of the painting’s geometrical composition… explore closely the body and gestural emotional expression of Christ and his 12 apostles in 3D … gathered around the table, when reacting in different ways, movements, expressions after Christ says “I tell you the truth, one of you will betray me.” One magnificently freeze-frame that actually feels as if were a series of sequences in motion…

Viewpoint 6: Visitors can go out of the dinner scene and enter into the room behind Christ and the Apostles. (Which was digitally reconstructed in 3D), explore the view from the back of the painting and the “Center Point” and see the scene from inside out, toward the Refectory gaining the feeling of how the Refectory looks from “inside-out”. Thus, experiencing first hand, and in an immersive manner the relationship between the real architecture of the Refectory and the painted architecture by Leonardo.

… Likewise, the visitors can see in the far back the virtual representation of the Crucifixion (1495) by Donato Montorfano, painted on the wall facing the Last Supper, to which Leonardo added the details of the portraits of the dukes of Milan …

… Visitors can after exit from that point to the one of the departure of Corso Magenta, Milan, Italy.

CONTENT VISUALIZATION, IMMERSIVE STORYTELLING AND SOUND ART

LSI combines emerging forms of 8K immersive narratives with perspective in motion, VR dramatization, aesthetics, interactivity and location awareness.

LSI blends multilayered digital visualization, immersive VR, simulation of three dimensional environments, 2D/3D information, 3D point cloud visualization, 3D animation, stereoscopic optical illusions, multiple viewpoints, visual effects, music and sound effects for augmenting visitors emotional immersion.

Focus was given to the implementation of novel approaches to immersive narrative for the fruition of new generation digital cultural heritage and ART experience.

INTERDISCIPLINARY METHODOLOGIES

LSI intersects and uses methodologies from the traditional humanity disciplines, together with leading edge technologies and innovative tools.

LSI is an interdisciplinary project combining and bridging traditional humanities disciplines, cultural heritage, humanities, social science, together with art, design, science, leading technologies and advanced media tools, immersive technology y interactive techniques and computer graphics.

Emphasis is provide for empowering the use and combination of experimental approaches for the design and creation of novel forms of virtual storytelling immersive narrative in the field of the arts and digital heritage

The LSI is the result of the author’s trans-disciplinary research, knowledge creation and strategic collaboration with professionals across the field of Art, design, humanity, cultural heritage, science and technology.

Visitors can after exit from that point to the one of the departure of Corso Magenta, Milan, Italy.

NOTE: References will be added

CREDITS

Author: Franz Fischnaller (IT); Development support & coordination: DIAC (IT); UNITY VR software Programming: Mathieu Godineau, Univ Rennes (FR); Prof. Gabriele Guidi, PhD. Indiana University. Department of Informatics – Virtual World Heritage Lab (USA)- Prof. Laura Micoli, Umair Shafqat Malik, PhD. Politecnico di Milano, and Department of Mechanics – Computer Vision and Reverse Engineering Lab (IT); Music & Sound: Steve Bryson (USA), Alfredo Miti (IT), Development by DIAC Design of Ideas – Architects of Culture (IT); © Haltadefinizione® Image Bank | By concession of Ministero della Cultura – Direzione Regionale Musei Lombardia (IT). Haltadefinizione is a Benefit Corporation directed and managed by Franco Cosimo Panini Editore S.p.A. (IT); Production support: Ars Electronica Center (A).

Thanks to Prof. Daniele Marini, who has inspired me to initiate the project. Thanks to all the others institutions, companies and people that have collaborated in this project.

Thanks to Yecenia Maharaj Singh for making this project become reality.

Acknowledgment

The LSI Project has been possible to be developed due to the support and collaboration of government agencies, museums, industries, labs, foundations, universities, interdisciplinary teams and researchers, art collectors. We will like to thank all of them, including:

Università degli Studi di Milano, Italy;

Robotics srl, Italy; L’EMMECI srl, Italy;

Ars Electronica Future Lab AEC, Austria;

The National Museum of Science and Technology, ‘Leonardo da Vinci’ Italy;

CIVES Foundation and Agency Ville Vesuviane, Italy; the Virtual Archaeological Museum, Italy;

INSA Rennes, Institut National des sciences appliquées de Rennes, France, Université de Rennes 1, 35000 Rennes , France, INRIA, institut national de recherche en sciences du numérique, France, IRISA, Institut de Recherche en Informatique et Systèmes Aléatoire;

CINECA – InterUniversity Consortium, 40033 Bologna, Italy;

INP G-SCOP Laboratory, Industrial Engineering School, Institute Poly technique de Grenoble, France;

IMéRA, Institut d’Études Avancées, laboratoire MAP Marseille;

MuCEM, Musée des Civilisations de L’Europe et de la Méditerranée, Marseille, France;

F.A.B.R.I.CATORS, Milan, Italy;

MEDIARTECH Tuscany Region, Province of Florence, Italy;

Electronic Visualization Laboratory [EVL], School of Art & Design University of Illinois at Chicago, USA;

Tuscany Hi Tech Network, municipality of the region of Tuscany, Italy;

Medienmuseum ZKM|Zentrum für Kunst and Design, Karlsruhe, Germany;

High Performance Networks Laboratory, Bristol, Network Interaction Laboratory, University of Essex, UK, University of Bristol, Faculty of Engineering, UK;

EU-FP7 VISIONAIR -VISION Advanced Infrastructure for Research;

Marco and Luisita Angiolini, among others.

COMPLEMENTATY INFORMATION

FRANZ FISCHNALLER SHORT BIO (IT)

Interdisciplinary scholar, polymath and practitioner (author, artist, designer, project manager), whose work spans the field of arts, design, technology, science, ecology, digital humanities and cultural heritage. FF’s artistic hybridism gravitates around unconventional combinations of multiple perspective and multi-layer creativity. His approach embraces a permeable interdisciplinary applied research methodology, across intuitive interconnection and cross-fertilization through different art forms, disciplines and shared practices with minded knowledge creation.

DEEP SPACE 8K – ARS ELECTRONICA CENTER (DS-8K-AEC)
DS-8K-AEC large-format projection space for interactive, stereoscopic and high-definition content. A VR multi-user interactive storytelling of 16×9-meter wall and floor projections with laser tracking and 3D 8K resolution images, that can host 90 persons simultaneously

In DS-8K-AEC , groups of up to 90 persons can experience simultaneously a broad spectrum of visuals: artistic productions on the basis of various different imaging techniques, interactive narratives featuring new approaches to interaction, and art historical, architectural and astronomical visualizations.
DS-8K-AEC serves as both an exhibition space for members of the public touring the AEC as well as an R&D platform for AEC Future lab staff. The program is articulated by several sections upon field and thematic, including to Art, culture, heritage, society, ecology, science, technology, design, architecture and digital heritage through explore classical statues, structures or entire streets from new angles thanks to 3D laser scanning technology

ARS ELECTRONICA FESTIVAL Ars Electronica is one of the world’s largest media art venues, a digital music festival, a showcase for creativity and innovation and a playground for the next generation – Ars Electronica is a world-class festival for art, Science, technology and society.

ARS ELECTRONICA CENTER (AEC) IN LINZ, AUSTRIA, ALSO KNOWN AS MUSEUM OF THE FUTURE, The Ars Electronica Center (AEC) is a centre for electronic arts run by Ars Electronica situated in Linz, Austria, at the northern side of the Danube opposite the city hall of Linz. It has been built on the right side of the Nibelungenbrücke. The building is also known as the “Museum of the Future”. Ars Electronica’s activities focus on the inter-linkages between art, technology and society. It runs an annual festival, and manages a multidisciplinary media arts R&D facility known as the Future lab. It also confers the Prix Ars Electronica awards.

LSI Website