Green Screen Gesture Tracking for After Effects
Have you ever watched a movie where an actor uses their phone or tablet but the underlying animations just didn't feel right? I've seen plenty, so when the time came for us to film a video commercial for our app, I came up with the idea for TouchTracker to avoid having to manually track interactions and ensure perfect animations.
TouchTracker is a simple web app for mobile devices that displays a greenscreen and records all gestures performed on the screen.
It outputs a json file that can be imported into AE with the included .jsx script. The script then generates a new composition with the exact screen resolution and imports each gesture as an animated "Null Object".
Say goodbye to poorly synced screen replacements with TouchTracker 🪄
Disclaimer: This has only been tested on iOS but should work on Android as well.
- Go to https://marcocanc.github.io/TouchTracker/ on your mobile device
- Press the Share button and then "Add to Homescreen"
- Open the app/bookmark and film your device while interacting with the screen
- When done, touch the screen with 3 fingers at the same time (might take a few tries). A window will open
- Transfer the file to your computer
- In AE, select "Run Script" and select the
touchtracker.jsxfile - Select the file you have transferred from your mobile device
- A new comp should open up with the screen and null objects representing each gesture.
- To attach a layer to a null objects, you can simply make it its parent
- Perform a screen replacement on the video recording of your device and sync the timing of your null object timeline to the video