A fully automated system that transforms Twitch clips into gaming compilations

Video explanation (YouTube)

Authors: Christian C., Moritz M., Luca S.
Related Projects: Neural Networks, YouTube Uploader, YouTube Watcher

---

About

This is our approach of automatically creating compilations of twitch clips (focused on gaming). We are very interested in YouTube, Twitch and Programming so we started this as an experiment, now a few months later we have to say it worked!

What did we achieve?

• We automatically uploaded over 100 videos to our YouTube channel without any human interaction
• One video got quite some traction by getting 2155 views and 48 likes at that point we had 0 Subscribers and it was the 8th video of the channel
• All our uploads combined got over 10k views
• We didn't receive a single comment indicating the use of bots

So why do we publish this now instead of making money with it?

• We came to the conclusion that we'd need to further optimize the videos to grow the channel which would require sophisticated and computationally expensive neural networks.
• We do not own the copyright of the clips so theoretically somebody could claim the video. To legally monetize the content, we'd need to add value to the clips like funny comments with a good text-to-speech framework.
• The language of some clips is declared as English by Twitch, even though the spoken language is not English. To filter those clips, a computationally expensive neural network for language-detection is required.

It was only an experiment without the direct goal of making money with it. Additionally, we think it is more valuable for us as a reference. If you are encountering any problems when running the code, feel free to open an Issue or a Pull Request with a possible fix.

Results

Example Video Screenshot	Example Thumbnail

Example Description

[Fortnite] Twitch Highlights #10 | Fails, Satisfying/Funny Moments & Epic Wins!

🔔 SUBSCRIBE FOR MORE FORTNITE FAILS/WINS!
https://www.youtube.com/channel/UCqq27nknJ3fe5IvrAbfuEwQ?sub_confirmation=1

Welcome back to episode 10 of Daily Gaming Highlights | Fortnite!
We have some insane clips this episode.
If you liked and enjoyed the video be sure to leave a like and subscribe for more Fortnite content!
We upload a Fortnite video every monday once a week.
Also be sure to comment letting us know which your favorite clip was 😄

Featured Playlists:
• Daily Gaming Highlights: https://www.youtube.com/playlist?list=PLhOg0bYb-2IKbbV4rHerrtlWArm14bXdI
• Fortnite Highlights: https://www.youtube.com/playlist?list=PLhOg0bYb-2IICWc4MB-ZetkKGMTSvanye

🎬Featured clips/streamers:
Streamer1: https://clips.twitch.tv/AwkwardProtectiveDoritosChip
Streamer2: https://clips.twitch.tv/WimpyGleamingTTours

00:00 Streamer1
00:11 Streamer2

Golden Gorilla does not have the copyrights for the used clips.
If you are within a video and would like to get it removed please email us:
[email protected]

#GoldenGorilla #Fortnite #Fails #FunnyMoments #EpicMoments #Wins #EpicWins
#fortnitebattleroyale #season4 #twitch

Requirements

This project requires Poetry to install the required dependencies. Check out this link to install Poetry on your operating system.

Make sure you have installed Python 3.10! Otherwise Step 3 will let you know that you have no compatible Python version installed.

Additionally, this project requires ffmpeg and imagemagick.

There are a few additional steps depending on the operating system that are listed below:

Windows

• Poetry
  • installing might throw errors which can be resolved by following this solution
  • you should use pip version ≥ 20.2.3 to avoid additional problems
• ffmpeg
  • has to be added to the path, so it can be accessed directly with the streamlink command
• imagemagick
  • you have to change the config_default as described here

Linux

• imagemagick
  • make sure the line <policy domain="path" rights="none" pattern="@*"/> is NOT present in the imagemagick policy config (/etc/ImageMagick-6/policy.xml on Debian/Ubuntu). Comment out this line if it is found in the file to ensure that imagemagick is not restricted by the path access policy.*

Setup

1. Clone/Download this repository
2. Navigate to the root of the repository
3. Run poetry install to create a virtual environment with Poetry
4. Follow the Twitch Api Docs until you get your own CLIENT_ID and CLIENT_SECRET
5. Optional: Follow the YouTube Data API Docs until you receive an API key
6. Insert your Twitch(CLIENT_ID, CLIENT_SECRET) and optionally your YouTube(YT_API_KEY) into the config.py. It should look like the following

HINT: The example keys do not work.

# For the Twitch API
CLIENT_ID = "o42tebpdzmj5z811iycgcpmd82on24"
CLIENT_SECRET = "owdgx64t1dodleiuo49e5rg7rvk5iq"

DIRECTORIES = dict(raw_clips_dir="rawClips", compilation_dir="compilation")

# For the YouTube API
# Optional: Only required if you want continuous numbering (e.g. Example Video Title #1, #2, ..., #N) for your videos
# Note: The number (#N) is determined by the number of items in a given YouTube playlist (declared in metadata_config)
YT_API_KEY = "h5UYdzJHeZNWLTgGteh0J68k7icp9jp9vPJlbzF"

7. Run poetry run python main.py to run the program. Alternatively you can run poetry shell followed by python main.py
8. Enjoy :)

Script Explanations

APIHandler.py

This class contains static methods to handle all API request for the Twitch and YouTube API.

Clip.py

This dataclass is used to convert json data from the Twitch API to Clip objects. This makes the data easier accessible within the python code.

ClipHandler.py

This class determines which Twitch clips the compilation will be made of and process them. It will do the following:

• call the Twitch API to get the clip metadata
• check if clips are ingame by using a neural network
• process 100 clip metadata at a time and fetch more metadata if necessary
• download the clips one-by-one

ClipCompilationCreator.py

This class will use the clips from the ClipHandler to create a compilation of all clips with a logo and the name of the clip creator as overlay. It will do the following:

• load all clips based on the metadata file
• create an overlay text for each clip
• composite all clips with their text and the logo
• compress the audio of the compilation so the sound is equaly high for the whole compilation
• render the finished compilation

MetadataHandler.py

This class will handle/create all metadata that is needed for a YouTube upload. It will do the following:

• handle the loading, saving, and storage of the metadata
• on every first load it will refresh the metadata. If clips got deleted in the raw_clips_dir it will remove them automatically from the metadata
• use a template to create YouTube title and description based on the metadata_config of the game and the metadata of that specific compilation
• create a thumbnail using a frame of one of the clips, a random emoji, the number of the playlists_items, a basic half transparent overlay, and the game logo

parser.py

This script is only for the ArgumentParser that will be used in the main.py. It simplifies the usage of the script by defining default values for every parameter aswell as types and help information. All parameters are optional so the script runs without parameters but they all work and can be used. To read more about this checkout the Run Parameters section.

utils.py

As the name indicates this script offers only utility functions. Those functions are used in basically every other script or class. It contains functions performing the following tasks:

• formatting a string into the correct string format
• adding any amount of seconds on top of a datetime.time value
• getting start and end datetimes for a given timespan (e.g. last hour, last week, ...)
• get a valid filename which removes all characters that may conflict with file handling operations
• get the full path to the game folder in our own directory structure for a given game
• get the full path of the previous compilation of a given game
• create our directory structure
• delete the whole directory structure
• saving and loading txt as well as json files

Run Parameters

All of these parameters are optional and a default value will be used if they are not defined. You can also get these definitions by running main.py --help

-h, --help            show this help message and exit
-g GAME, --game GAME  
                      Declares for which game the compilation should be created. It uses fortnite as default
-ap ASSET_PATH, --asset_path ASSET_PATH
                      Path to the assets folder. If not declared it uses './assets' as default
-noc NUMBER_OF_CLIPS, --number_of_clips NUMBER_OF_CLIPS
                      How many clips should be used. For most use cases -ml will fit better since the length of clips can be between 1-60 seconds so a -noc 5 compilation could be 5 or 300 seconds long
-ts {day,week,month}, --timespan {day,week,month}
                      ['hour', 'day', 'week', 'month'] - timespan from when the clips should be taken. Default is week
-la LANGUAGE, --language LANGUAGE
                      Language of the clips. Default is en
-ml MIN_LENGTH, --min_length MIN_LENGTH
                      Length of the compilation in seconds. Default is 360 (6 minutes)
-mcc MAX_CREATOR_CLIPS, --max_creator_clips MAX_CREATOR_CLIPS
                      Number of clips used from a single creator. Default is 2
-mcd MIN_CLIP_DURATION, --min_clip_duration MIN_CLIP_DURATION
                      Minimal clip length. Default is 10
-o OUTPUT_PATH, --output_path OUTPUT_PATH
                      Output path - default is './TwitchClips'. This should not start with a '/', otherwise it will use it as an absolute path