Category: Motion Tracking

I was looking at who uses PFTrack in the industry and found that it was used in the movie the Watchmen by the VFX house Intelligent Creatures, they also used Houdini and Maya.

In particular they used PFTrack for the face of the character Rorschach.

The VFX supervisor, Jeff Newton said;

“This required a lot of very exact head tracking as Rorschach is a major character in the movie and had a lot of close ups”

They had over 300 mask shots for the character and used PFTrack for them all. They used the software to solve the camera for all of the shots and like i have been finding with motion blur,some moving shots needed manually tracking as the tracking markers on the face would blur. But with close up shots they were able to use automatic tracking.

Reading about another shot they did it seems that tracking software doesn’t like zoom shots and they would fail at the track but PFTrack is said to be ‘famously good’ at them. It seems this is down to the lens distortion correction facility in PFTtack.

“With the help of PFTrack’s lens distortion correction facility we were able to engineer our own lens distortion correction and redistortion solution. We then imported the lens distortion corrected footage into PFTrack and with the help of its Automatic and User Feature Tracking it solved the camera move and zoom animation easily”

Jeff Newton

I defiantly need to have a look at this software in the future. I remember watching this movie when it came out and wondering how do they do that?!  I’m slowly learning how.

Time willing I would have liked to have a go with PFTrack and from what I can see they dont have a student version, or i just cant find it! If i had a spare £2000 buy it but no such luck.

So I thought I would read up on it instead.

Braught to us by Pixel Farm, a company that manufactures and markets image processing technology for the professional film, TV and Broadcasting market.

PFTrack is Tracking software and after reading the overview on their website, it sounds like an impresive piece of kit. They describe it as the most comprehensive 3D tracking, match moving and scene preparation toolset thats available. So what can you do with it?

Node-based Flowgraph Architecture

‘The Tracking Tree controls the flow of data as nodes are connected to perform all of the various tasks in PFTrack 2012 such as image processing, feature and geometry-based tracking, camera solving, image modelling and file export. Nodes may be infinitely branched allowing multiple techniques to be used to achieve the most accurate result.’

Geometry Tracking

‘Geometry Tracking can be used to track either the camera or a moving object, using a triangular mesh instead of tracking points, which avoids many of the typical pitfalls that plague conventional tracking such as glints, highlights and motion blur. In PFTrack 2012, Geometry Tracking has been enhanced so that it may be used to track a deformable object like a talking face. This can be achieved by creating one or more deformable tracking groups, assigning some of the triangles in the mesh to those groups, and specifying how the groups can transform relative to the rest of the mesh.’

Image Modelling

‘Image Modelling can be used to construct 3D polygonal models that match elements viewed by a tracked camera. A set of modelling primitives are provided that can be positioned in 3D space and edited to match the image data, or new models can be constructed by connecting 3D vertex positions to form a polygon mesh. Z-Depth can be used to estimate the distance of every pixel in an image from the camera frame, producing a grey-scale depth map image encoding z-depth, and a triangular mesh in 3D space. Texture UV maps can be created and edited for any object, and both static and animated textures can be mapped onto geometry for export.’

Stereoscopic Tracking

‘When tracking a stereoscopic camera in PFTrack 2012, auto and user features are tracked simultaneously on both the left and right eye images. When solving the camera, artists have full access to the data defining the rig including interocular distance, convergence, etc.’

Image Processing

‘Already benefiting from the Enhance, Shutter Fix and rotoscoping capabilities of PFMatchit, PFTrack 2012 adds a new Optical Flow tools to calculate dense optical flow fields describing the apparent motion of objects relative in the camera plane. It will also retime clip and motion data to increase or decrease the apparent frame-rate of the camera.’

Mocap Solver

‘The Mocap Solver node can be used to calibrate the motion of individual tracking points viewed from two or more camera positions. This is often used to track the motion of an actor’s body or face, where tracking points have been identified using physical markers. In contrast to standard object tracking, the Mocap Solver node does not assume that the object is moving in a rigid fashion. The motion if each tracking point is completely independent and can therefore represent movement of non-rigid objects.’

Reading through the features of PFTrack i realise how little i have touched the surface of tracking software. A lot of this has gone over my head but it is a program i would like to experiment with and look more into depth into match moving and motion tracking.
Since the hiccup with the Faceware plugin not being compatible with Mac, i have been checking other software compatabilitles before hand. So PFTrack is compatible with Mac and Windows.

After the Millennium disaster Annebeth took me through what I needed to do to get better footage from a DSLR for what I wanted to do with it. We used a bigger space again, but used a faster shutter speed and wider aperture to eliminate the blurriness I got in the previous test.

Well this worked much better, I got a much better track of which I did manually, Annabeth also showed us that we can place the grid manually in Matchmover if the camera solve is off.

I’m lucky I exported the camera when i did because when I tried to reopen the Matchmover file it had corrupted! Lesson learnt MAKE MORE BACK UPS! Especially as Match mover has a habit of crashing on you mid camera solve and any other time it feels like it.

I totally forgot that the camera was still set to tungsten but I just colour corrected it again. But Mat showed me the tint effect in Aftereffects that sort of matched the boxes in the seen quite nicely. But i prefer the colour corrected one.

I’m happy i did a bigger track involving surroundings and not just some dots on paper. Maybe I shouldn’t have made such a massive leap from the dots on the paper to a very large open space and thought about an inbetween track like this one. but thats me aiming high…again.

The millennium square shoot wasn’t great unfortunately but it was a test after all.

1. The lighting was set to tungsten on Andy’s DSLR so when we got back and took a look at the footage it was blue. but that wasn’t a major issue as its about tracking not visual effects. So I did a rough color correction for when I composite my cube in.

2. The shutter speed was too low that a lot of frames were blurred making it harder to track. So it needs a high aperture and shutter speed to eliminate motion blur. The only problem with this is the footage can look CG but you can add a bit of blur in post to take that perfect edge off it.

3. Although the buildings tracked alright, the floor barely tracked, with no tracks above 100 frames. I tried to resolve this by using the threshold effect in AfterEffects.

Although the lines on the floor stood out really well using threshold the footage was very noisy. Taking it into Matchmover and trying to manually track the floor with the corrected footage was possible but it wasn’t accurate enough. The pixels within the bounding box weren’t consistent, instead of a nice neat lines the while and black pixels would change to white as the camera panned round. I think this is why I got a very bad track, Matchmover was doing it’s job I just hadn’t provided it with good enough footage. Because of the changing pixels it thought the camera was moving and tracking it, when it wasn’t. I highlighted what i mean in the clip below, you can see the color of the pixels changing on and area i tried to track.

I also tried turning the image to black and white, sharpened it and upped the brightness. But because of the blurred frames it wasn’t possible to maintain a solid track. I tried manually tracking the cracks in the pavement and the corners where the different colored tiles meet.

4. Because I was having trouble with the camera solve Annabeth said to take out any lens distortion and see if that helps. Turns out there was quite a degree of lens distortion, so using AfterEffects we tried to correct that using Optics Compensation under Distortion Effect.

So here is my first export…

It’s pretty bad, with the skills I have learn over the last few weeks I just couldn’t get a better track. I do think that threshold is to blame for the shakiness but i couldn’t get a track on the floor without it. I think it also didn’t help that the tracks on the buildings in the background were so far away, confusing the camera solve.

I tried making the cube float but that was only slightly better, the shadow made it very obvious that it moves in the scene.

So i took the shadow out…

Still bad but this test was not a total loss! I have learned so much to be prepared for the next shoot. I thought that there would be enough to track but next time I’ll be sure to put some markers down. I think I was worried about masking them out but I could have just hidden them under the cube that I wanted to composite in. oh well next time I know what to do!


Mocha ( add video)

I came across Mocha while looking up tracking software. I had a look at this tutorial to see what it was about.

It focuses on planar tracking and could open up a lot more options for VFX compositing. In this tutorial they track out the fore ground from one sequence and place it in another. This would be so good for adding buildings to a back ground or hills and mountains.

So when I had trouble tracking my interface idea in AfterEffect I thought I would try planar tracking it in Mocha.

Well… I literally opened and closed it. Someone showed me how to put in a bounding box but after that i was just lost. I felt a this late stage in the module i didn’t have time to get wrapped up in another program as i still have Motion capture to cover and some performance capture. But hopefully if there is time I will follow a tutorial for it and have a go. If not there is plenty of time over the summer.

I like to watch films and tutorial while i’m working to give me inspiration and tips.

I watched this tutorial on tracking.

It gave me a heads up on the understanding on Motion tracking.

In my interim crit Mat mentioned AfterEffects Mocha, so i looked up some information to give me an insight into the software.

Andy’s shoot

Saturday 19th I helped Andy with a shoot based on the final shot from the video Eye of the storm.

I was a bit concerned about the color of the tracking points as we were using green tape on a green screen. From what i had been learning about tracking you need high contrasting points and green on green wasn’t high contrast. But Andy was confident it would be fine and it was. Putting the footage into AfterEffects and applying the Effect Threshold to prep the footage for tracking and the tracking points stood out brilliantly. I’m not 100% sure how threshold works, I have a feeling it is something to do with light and luminosity, but I can’t find a clear definition of how it works to gain a better understanding.

We put up the set and had to make shift a dolly as the proper one was on loan.

We got some really nice footage and while we had the set up and the studio all day we did some extras shots to play with on our own time. So we had maximum greenscreen coverage we clipped the greens screen to the chairs.

As this is for Andy’s project I wont be using this footage for part of my negotiated studies but we have been putting our heads together to try and come up with the best way of getting the effect.

Looking at types of tracks I could do, I wanted to try and see how I could achieved this effect they did in Avatar. These interactive panels are transparent and has an interface composited onto the plain.

I recently bought a reading panel that has a hard plastic transparent sheet, this was perfect to use as a prop for my interface test.

I took two sets of footage and tried to do a perspective corner pin track straight away. Both sets of footage had problems tracking. With the shot of Andy holding the prop it had trouble tracking when the colors around the tracking dots changed and it would loose the track.

With this shot, as I pretended to interact with the plane to add interactions later. The problem with this was my hand would obscure the tracking points and then lose the track.

After leaving it for a few days while getting on with other tracking in MatchMover, i wondered if, like Matchmover, I could re-position the track when it gets lost, so I gave it ago. While it was tracking forward i would stop it when it was going of track and re-position the bounding box over the tracking point. For one tracking point I had to re-position frame by frame when my hand would cover the dot and do it by eye until the dot was visible again and would pick up the track. It worked! I guess this is a lesson to prevail and problem solve using experience gained from other programs.

I made a new solid and did a very basic animation on the layer. I wanted to make sure i applied a moving image to the source of a tracked plane as i hand done it yet. This was very simple, i just had to make sure the animation was on the right section of the time line. Added a little blur for the illusion of emitting light and a back light of added effect.

So i did it! I worked out a way of creating a similar effect to that of the Avatar picture using knowledge I gained in this module!

It looks good, its very basic but it’s done the job and would be able to do it again!

Third time lucky

So my indipendant track went well… the third time round. I was having trouble getting a good camera solve from matchmover. I was getting very frustrated as both tutor Matt and friend Andy both had a go at the same footage and got a really good solve. I tried it 3 times and on the third time i got it. but i am not sure what i was doing different. I have a feeling that it could have been the bounding box around the track. I just kept re-sizing them and everntually i got a good , mostly green solve. After inputing the co-ords etc i opened it in maya and dropped a cube in the scene.

With this play blast you can see it is a really nice track. The cube looks firmly set on the table, i am very happy with that Pretty confident that i will be able to do it again.

After trying to mask out the tracking markers and failing (explained in a previous post) i decided to simply place some cubes over each one to cover them up. A bit sneaky i know but it worked nicely.

I did two render passes in maya. One of the shadows only and one of the blocks only, with a 2 pixel blur. Annabeth suggested bluring it in Maya as it would be a 3D blur rather than adding a 2D one in AfterEffects.

By rendering out the shadows separately this allowed me to soften them in an AfterEffect layer to match them with the lighting in the footage, which I think has worked quite nicely. They look very much like they are on the table. Really pleased and will be using this as 5 seconds of my 10 second motion tracking demonstration.

So the more I learn about digital film, games and animation, the more I spot and analyse stuff on the telly and ads. I spotted this late rooms add ,

And immediately thought I know how they did that!! Well i know how i could do that with what I have learnt about tracking. That could have easily been done using point track in AfterEffects.

I found a good tutorial from video copilot on corner pin track and point tracking and the point track part of the tutorial is pretty much how the advert did it.