Wk 1 - The age of the image and the trends of the lens
In this lesson we first started by discussing some current trends in VFX.
I started by mentioning the use of virtual sets (such as LED screens instead of green screens). This allows for VFX artists to create a CGI background before filming, the camera is connected to the background, so if the camera moves the background does too to fit its field of view, this allows for traditional camera moves (such as the parallax effect) to be easily filmed on virtual sets. Virtual sets also help with lighting as the LED screens give better lighting and reflections off of surfaces, they also help with the actors performance as they can see what kind of environment they are in instead of just being in a sea of green all around them. Virtual sets can be a lot too invest in, but in the long run saves both time and money as they do not have to keep building new sets, travel to new locations or spend more money/time in post-production.
'The mandalorian' used LED screens in their filming. See the video below on how/why they did this.
The age of image 'A new reality' - Dr James Fox
BBC iPlayer (2020) Age of the Image. Available at: https://www.bbc.co.uk/iplayer/episodes/m000fzmc/age-of-the-image (Accessed: 21 September 2020).
This episode discusses time, movement (4th dimension) and artists view on this.
Gunning, T. (2006) 'The Cinema of Attractions: Early Film, Its Spectator and the Avant-Garde', in Strauven, W. (1.) The Cinema of Attractions Reloaded. Amsterdam: Amsterdam University Press, pp. 381-388.
Tom Gunning introduces a new theory which states that instead of audiences focusing on the narrative of a film, audiences will focus on the animation and the images that are there in front of them. This is known as the 'Cinema of attractions'. In the beginning, films were very short and audiences were less focused on the narrative then amazed at the moving images right in front of them. In this scenario we can unite Lumiere and Melies, they are fascinating because of their illusionary power. For example the realistic illusion of motion by Lumiere brothers or the later on magical illusions by Melies.
Eadwerd Muybridge was one of the most important photographers as he started the path for filmmakers with his experiments in motion photography. In 1872 he first experimented with motion photography when he wanted to prove that while galloping a horse takes all four legs off the floor. He did this by setting up 12 consecutive cameras along a racing track which were activated by the horse running across a trip wire. These cameras had a faster shutter speed making it easier to capture motion.
Harold Edgerton, famous for many of his motion photography pieces (especially the milk drop cornet), took another path to motion photography by adding a stroboscope to the end of his camera, this would flash up to 120 times per second, capturing movement that wouldn't be visible. He photographed many fast moving things, such as a bullet going through an apple, birds flying and much more.
John Gaeta is famous for the bullet time sequence from the matrix, he would not be where he is today without the work from Eadwerd Muybridge and Harold Edgerton. The technology used was much more advanced but many of the techniques were the same from Muybridge and Edgerton. In order to do this shot, he set up a large camera rig that completely surrounded Keanu Reeves, each camera then went off in a sequence while the action took place.
Gunning's Cinema of Attractions - think about how this theory can this be related to contemporary visual effects in cinema do you see any connections? Can you provide visual examples?
Gunnings cinema of attraction can be related to contemporary visual effects in cinema as it makes audiences focus more on the image in front of them and the illusions of film. VFX have become so important in cinema that it can make or break a film, if the VFX are bad then the whole movie goes down despite it having good narrative (the same goes for the opposite).
Reflect upon the relationship between Visual Effects and the lens, do you see a paradox here?
The relationship between VFX and the lens is very important....
The photographic truth claim: can we believe what we see?
Plato had a theory called the allegory of the cave. In this theory there are prisoners chained up in a cave and all they see are shadows of figures passing on the wall (for example a bird). These prisoners see the shadow of the bird as a real bird when in fact it is an illusion, Plato says that in order to become free one must realise that it is an illusion not reality. When one prisoner escapes the cave he is blinded by the sunlight, however when he adjusts he is amazed by the reality of the world and returns to try and set the others free. Having his eyes now accustomed to sunlight he is blinded again by the darkness of the cave, the other prisoners see this and fear the reality outside the cave, not believing it could be better, they then kill anyone who tries to make them leave.
Here Plato is saying that everything we see isn't always real and our senses can easily fool us. We can see examples of this in photographs (photoshop), film (vfx) and other media forms.
Below shows an image depicting the theory.
What you think is meant by the theory: The Photographic Truth-Claim?
Gunning, T. (2004) ‘What’s the Point of an Index? or, Faking Photographs’, Plenary Session II, Digital Aesthetics, 1(25), pp. 39-48
A photograph combines both index and icon. The traditional idea of photography is that it is an imprint of reality, it is the only medium that can do this. A photograph doesn't have to be exactly what it looks like (it could be blurry or very zoomed in that we cannot see the object entirely etc.) but it is still a record of that thing. However, digital photography can be easily manipulated, and therefore it can be argued that this can disrupt the truth claim - when we use photoshop etc. to alter an image the index is altered and created. It can also be argued that the truth claim is questionable not only with digital but traditional photography as well as there are photographers that used to alter images to create illusions. "Traditional and digital photography can both lessen, ignore or undo the indexical"
The image on the left shows an alteration in digital photography using photoshop (as we can see the final photo isn't really an imprint of reality).
The image below is an old photo using traditional photography which captured fairies.
This shows that both traditional and digital photography could have been altered, however digital photography is more easily and realistically altered.
Faking photographs: image manipulation and computer collage.
Write a definition of VFX compositing. What is it and how does it work?
Mitchell, W. J. (1998) The Reconfigured Eye. Cambridge: MIT Press.
Casetti, F. (2011) 'Sutured Reality: Film, from Photographic to Digital.' October, Volume 138, pp. 95-106.
Compositing is the technique used to layer different elements together to create one final image.
VFX composting has many component parts, mainly CGI elements and live action footage. These different components are taken and layered together, now we use technology (computers) to do this but there were ways of doing this in the past using film, light and chemicals. The purpose of the composite is to create a final new image, usually a realistic one, by combining different elements into one. This is all to create an impression of reality. The composite is creating a believable image, it needs to look optically correct, the light, shadows and CGI need to have the right detail. Compositing is not an imprint of reality but an impression of reality. This is because it does not represent exactly what the reality was because the image we see was never a reality, instead it was constructed to be this. Thus it is in an impression of reality as it tales multiple different realities to make one, this could also be seen as a fake reality since it is not truly representative, linking back to the photographic truth claim.
Finding the properties' of Photorealism in Visual Effects
Photorealism with CGI in VFX.
TED (2019) 'Why most visual effects suck—but some don't? Available at: https://www.ted.com/talks/wren_weichman_why_most_visual_effects_suck_but_some_don_t (Accessed: 20 October 2020)
Flueckiger , B. (2015) 'Photorealism, Nostalgia and Style. Material Properties of Film in Digital Effects', North et al. Special effects: new histories/theories/contexts. London: Bloomsbury, pp. 78-98.
Photorealism is a form of art when we use different mediums to try and reproduce the image as close to reality as possible. With the advanced technology of today VFX can do this well, the Computer generated images are very realistic. However we must be careful as if they become too realistic it will break this illusion of reality and fall into the 'uncanny valley'. Realism in VFX can either be with the composite or CGI, the illusion of reality can break if both these things are done incorrectly. Lets look at CGI, for example The Lion King (2019), this movie is done completely CGI, it was groundbreaking for the technology they used and the way they did this. However, was it too realistic? the characters look real, their movements feel real, the environment is realistic, the movie could almost be a documentary based on how realistic it is. But the characters talk and even though they look realistic when you look at them there is something off, something that doesn't feel real. Perhaps there is too much detail, more than our eye usually captures (did you know that they had to diminish the quality if the movie by a lot before they released it). We can say that because of our modern technology VFX can create photorealism very well and use it well but it can also use it 'too much'.
A Digital index: bringing truth into VFX via the capture of movement.
Blog post: Compare motion capture to Key frame animation.
Allison, T. (2011) 'More than a Man in a Monkey Suit: Andy Serkis, Motion Capture, and Digital Realism', Quarterly Review of Film and Video, Iss P. 325–341.
Johnson, O. and Thomas, F. (1981) The Illusion of Life. New York: Disney Editions.
Key frame animation and motion capture are both techniques used in film and VFX industry today mainly for animation, both of these techniques have their advantages and are used for different reasons.
Key Frame animation is when there are key points of motion which defines the animation and then these are transitioned between to create the illusion of movement. Key frame animation can bring inanimate objects to life, whether it be a drawing animation or one made in a 3D program. In 3D programs the object/ character will be automatically moved along according to its keyframes, where as with drawing animations the artist may sometimes have to make in-between shots to make the movement smoother. For example, the original Disney animations were all done using the key frame technique. Another example of an animation using a keyframe technique is one I did for my Character animation class where I animated a ball using the 3D software Maya.
Motion capture is where movements are recorded using special suits and this data is used by computers to link the recorded motion to a computer generated character/object. This can make movements more fluid as the data captured is "real" once coming from the original movements. This can be used to turn actors into characters that were not actually real, for example in planet of then apes the apes were actually actors in motion capture suits. It can be said that this is a. more modern approach as you do not need to hand animate the characters or objects. However, both methods are used today.
Digital index: reality capture (LIDAR) and VFX
Blog post: Case study on Lidar lounge
Cotterall, S. (2019) The Study Skills Handbook, Red Globe Press, London.
UWLVFX.(2020) The Industry Interviews: Tamara Mitchell & Ross Clark Co-Directors of Lidar Lounge Available at:https://uwlvfx.wixsite.com/uwlvfx/tamara-ross-interview (04.11.2020).
Reality capture is an umbrella term that encompasses different techniques, it is also a software. It uses photogrammetry and/or laser scanning/LIDAR to create 3D models. It can create scans of objects, environments and more to be used in film, tv, architecture, VR, gaming and much more. LIDAR lounge is a company that specialise in 3D scanning using laser technology and photogrammetry. They do character cyber scanning, set, location, vehicle, object and prop scanning as well as texture and roundshot photograph, it is also used for data-mapping They are mainly hired to work on films however they also do other things too.
The benefits of doing 3D scanning is that it is quicker than modelling from scratch, it also gives a more accurate representation of what you want to show, it is also more photorealistic.
At LIDAR lounge a typical day for them (if working on a project) would be to go out and scan certain locations or a set or, if scanning a character, they would do it in-house on their 360 rig. After scanning they would back it up and then start aligning the data in software (such as point cloud), this is then passed on to modellers, using the scan as a base, they would add more detail and make it even more photorealistic.
LIDAR lounge has worked on many well known films. For example they took panoramas of the London skyline for the movie Hobbs and Shaw, this was used in many shots as a background. The way the company did this was by taking there equipment onto a window washers rig, climbed up a building and took data to make a panorama background for the film.
Final Essay
From this point on everything that is posted is research towards my final essay topic. The topic I chose was the second option, I included a screenshot below of the question.
I chose to focus my essay more on 3D scanning v.s. 3D modelling. I started my research by writing down some points for each type of modelling. Such as:
3D scanning can be beneficial as it can be more photorealistic, as it also captures very realistic textures from the photos it takes, it can be quicker to produce and is useful for creating CG doubles of actors. Modelling is still used to fix any mistakes scans may have and also to retopologise the mesh. 3D modelling is also better when creating fantasy characters that don't exist as these are not possible to scan, for example the Davy Jones character in Pirates of the Caribbean 3: at worlds end. They created this main character as well as 17 of his crew member by modelling them in ZBrush.
Below are two links to how they created some of the vfx in this movie, within these it mentions slightly how they modelled the characters.
https://www.cgw.com/Publications/CGW/2007/Volume-30-Issue-5-May-2007-/All-Hands-on-Deck.aspx
https://cgi.tutsplus.com/articles/a-look-at-hollywoods-biggest-vfx-part-2--ae-18875
I found it difficult to find much information on 3D scanning and modelling of particular films specifically. SO therefore I decided to conduct my own comparison by using photogrammetry and 3D modelling in Zbrush/Maya of different objects.
I first started by 3D scanning a rock wall lamp and this worked quite well.
The texture was very different throughout the object so I think that the program liked this as it was able to pinpoint the different pixels easily, putting together a very good 3D mesh and texture of the object, the only part that did not work so well was the bulb in the middle as well as some overexposure to the texture on one side, but this was most likely due to the lighting as the inside of the lamp did not get much natural light and one side was a bit too well lit due to the window being right next to it.
After waiting a while and finally getting the final object, I imported into Maya to begin retopologising it.
ASSESMENT 2 - PRESENTATION
For part of the assignment we had to create a presentation. I took screenshots of each slide, shown below. The presentation was a brief summary of what my essay will include.
FINAL ESSAY
NEW TRENDS OF CAPTURE AND REAL-TIME FILMMAKING LEAD US TO CONSIDER IF THE TRADITIONAL APPROACHES TO 3D MODELLING, ANIMATION AND RENDERING ARE STILL NEEDED.
This question evokes the words of Paul Delaroche in 1840, who upon seeing the first photograph supposedly claimed that from that moment on painting as a medium was dead.
Can we say the same for 3D modelling, animation and/or rendering? New and established trends of motion capture, reality capture and virtual filmmaking are impacting how 3D models are produced, animated, rendered and indeed how we perceive them.
In 1840, upon seeing his first photograph, Paul Delaroche supposedly claimed that from that moment on painting was now a dead medium. This statement can lead to the question if the traditional approaches to 3D modelling, animation and rendering are still needed, due to New trends of capture and real-time filmmaking? Can we say that because of these new trends the traditional form is dead? No, however, they are impacting how 3D models are produced, animated, rendered and perceived. These trends include motion capture, reality capture and virtual filmmaking. I will be focussing on the new trends of reality capture and how this impacts traditional 3D modelling in Visual effects.
Reality capture includes photogrammetry and LIDAR, these are both 3D scanning techniques. Photogrammetry is when you take several images of the object/environment you wish to scan and then upload them to a software which pinpoints similar vectors, guessing and putting together the mesh. LIDAR is a form of laser scanning using a laser light to measure different distances putting the mesh of the area/object together. These techniques are used in film, games, tv shows and more and usually help with VFX modelling, but never fully replacing it.
Lidar lounge is an example of a company that does 3D scanning. They do LIDAR scanning as well as photogrammetry, a service mainly provided to the film and television industry but also other industries such as Architecture and Design. Lidar lounge does character cyber scanning, set, location, vehicle, object and prop scanning as well as texture and roundshot photography. The benefits of doing 3D scanning include speed, quicker than modelling from scratch, it also often gives a more accurate representation of the object shown. The scan captures high quality and detailed textures as they come from the real life object, environment/characters, assisting in the overall photorealism of the scan. Although 3D scanning can save a lot of time as well as money, saying that it replaces the use of 3D modelling is inaccurate as it is still used in 3D scanning. For example, after 3D scanning the artist must retopologise the whole model (which is a form of remodelling it), usually they also have to add extra details or fix some parts that weren’t picked up correctly.
Even if 3D scanning is being used more often, it only provides a base for 3D modellers to start their work from, making the overall process cheaper and quicker, but not replacing it completely. 3D modelling is not only being used to fix/alter 3D scans, but also has other benefits that cannot be found in 3D scanning, making it irreplaceable by new technologies of capture. 3D modelling allows a certain level of creativity that 3D scanning does not. For example, it allows artists to express and create from their imagination or from concept art, 3D scanning cannot be used to create fantasy characters as they are objects of the imagination, not objects of the real world. 3D scanning may assist in some developments but 3D modelling will never be neglected as it allows for innovative design, creativity and a wider range of content. We can see this in many films, such as Pirates of the Caribbean and the character Davy Jones, this character does not exist in real life, he is half man half octopus, a human with an octopus head. There is no possibility that he could be 3D scanned as he never existed and was a false reality, the artists that worked on him sculpted this creation in Zbrush as shown in a video interview about it (mouseketweeter, 2010).
Another issue with the process of 3D scanning is that it is not always successful and can often be difficult to achieve a good quality scan. Scanning plain coloured, smooth textures things such as a plain white wall does not always work as there is not much difference between the different points so the program finds difficulty putting it together. This can cause holes in an environment giving extra work to fix after the scan is complete. Hence, resulting in the artist depending more heavily on 3D modelling later in the process.
Plato’s ‘Allegory of the cave’ theory discusses reality and signs, in the cave the people see a shadow of a bird and think it is an actual bird, in reality it is an illusion and in fact they are referencing a sign to a real bird (it is an appearance of reality). This can link back to CGI and modelling/scanning, the model that we see is not reality itself, it is an appearance of the reality, it is a sign. For example if there is a 3D scan/model of a building, as realistic as it looks this is an appearance of the reality of that building. From this we can talk about Gunning and his photographic truth claim theory. According to Gunning (2004, pp. 39-48) a photograph is considered to be both indexical and iconic, it is an imprint of reality as it captures exactly what that reality is, these photographs are icons, but with the use of programs they can be altered and manipulated, disrupting the truth claim and creating an indexical representation of reality. We can say the same for 3D modelling and scanning. 3D scanning is an indexical representation of reality, it captures exactly what that reality is by scanning the actual thing itself, whether it be an object, person or environment. Photogrammetry uses multiple photographs to create the 3D model, here we are essentially using multiple representations of a reality to create another one. A 3D model is more of an iconic representation of reality as it references a specific reality but is not an exact representation of it. When modelling something from scratch you are creating a new reality from nothing, although a real world reference may be used it is not that actual thing that you are creating, no matter how realistic it may be. For example, I conducted my own research where I both scanned and modelled things, first I 3D modelled a speaker that I have in my room (this is an iconic representation of that speaker), then I 3D scanned the same speaker by using photogrammetry and the software Metashape (this is an indexical representation of the speaker). One is something that, with the inspiration of looking at the speaker, I tried to recreate and the other is exactly what the speaker is as it was made from multiple photographs of the object. As Fox states “Over the last few decades images have become more like reality than ever before, and reality has become more like an image than ever before the result, like Plato’s prisoners before us, is we can’ t always tell the difference between them” (Fake Views, Age of the Image, 2020). Realism in CGI has become more and more important and popular. In the recent show 'the crown’ Framestore created a digital stag for the show, note that they had to use 3D modelling for this as they cannot 3d scan an animal, many people could not tell that this was a digital animal, nor did they even think about it. The level of realism was achieved by thinking about small details you wouldn’t think would matter, like how the hair clumps in the rain or fine pieces of grass on the antlers. We can link this back to Fox’s quote as more often modern vfx are becoming so realistic that we cannot tell them apart from reality. Thus making the line that separates the two even finer. Susan Sontag states “Needing to have reality confirmed and experience enhanced by photographs is an aesthetic consumerism to which everyone is now addicted it would not be wrong to speak of people having a compulsion to photograph: to turn experience itself into a way of seeing. Ultimately, having an experience becomes identical with taking a photograph of it.” (Sontag, 1977, p. 3). This can be linked to VFX, the audience has a need for the reality, of the tv show/film/advert and more, to be confirmed and enhanced. With these new trends people also become more used to films with vfx creating a new reality or view on reality for them that if the vfx shots are slightly off or not as realistic, then they are taken from the story and it can ruin the whole experience. Looking back at the first computer generated shots in movies to the ones today there is much more realism however because audiences have gotten so used to them, they expect a certain level of realism, one which can be difficult to achieve. The new trends of VFX affect the way audiences take in media.
After looking at the benefits of both 3D scanning and modelling, it is safe to say that the traditional approaches are still needed. Without 3D modelling there would be no creative expression within characters/fantasy characters would not be created, 3D modelling is also involved in the process of 3D scanning. However 3D scanning also can be very beneficial. The new trends of reality capture have changed the way things are produced, but have not fully replaced the need for the original ways. Both have their benefits and combining the two can produce extraordinary models in visual effects whether it be for film, tv, advertising, or simply just for creativity.
REFERENCES:
Fake Views, Age of the Image (2020), BBC four 6 March, 03:00. Available at: BBC iPlayer (Accessed: 18 January 2020)
Gunning, T. (2004) ‘What’s the Point of an Index? Or, Faking Photographs’, Plenary Session II, Digital Aesthetics, 1(25), pp.39-48
mouseketweeter (2010) Pirates of the Caribbean: Dead Man's Chest- Creating Davy Jones at Walt Disney World Backlot. June 22. Available at: https://www.youtube.com/watch?v=UdAEYLWMY3A (Accessed: 8th January 2020).
Sontag, Susan (1977) Susan Sontag On Photography. London: Macat international ltd.
Stag that Framestore created mentioned in the essay