Datamosh Studio For Mac

GNU datamash is a command-line program which performs basic numeric, textual and statistical operations on input textual data files. ZXPInstaller is a user-friendly installer for Adobe extensions. It replaces the Extension Manager which Adobe no longer supports as of CC 2015. Avidemux is a free tool commonly used for datamoshing. You can also do basic video editing with it, though many people edit the original videos in some other software (e.g. Here are some good introductions to Avidemux: How to datamosh videos Datamoshing.

This tutorial is going to be about creating your own datamosh-like shader, and will be going over two very useful techniques for Unity shader effects: motion vectors & pixel recycling. Datamoshing refers to the artifacts produced by modern digital video compression techniques, where small motion vectors control displacement of blocks of pixels instead of sampling a fresh frame on every single frame. It’s quite an interesting technique, and it’s fun to try reproducing the artifacts produced by it.

Datamosh Studio is a datamosh video editor. You can use this program to perform datamoshing on videos in a quick and easy manner. Find out more about datamoshing here. This software is currently in beta and it may contain bugs.

QUICK NOTES:

Unity’s internal motion vectors only work on some platforms, and will require RG16-format (or RGHalf) rendertextures to be working on the target platform to render. Some platforms, like WebGL, do not support this at the time of writing, though it may be possible to write your own motion vector system.

Pixel recycling (or feedback, or continuous computation, etc…) is incredibly useful for producing interesting/dynamic feedback loop effects, as well as extremely valuable for simulating physics and other complex computations with accumulative data (as opposed to data that’s disposed of at the end of each frame).

A number of projects of mine make extensive use of this technique, and because it relies only on preventing Unity from clearing data from the pixel buffer, it’s pretty easy to get working on all platforms.

There are a number of ways you can set this up in Unity to read back pixels from the previous frame (another method is explored here in a tutorial by Alan Zucconi), but here we’ll be exploring a simple method that uses Unity’s GrabPass function to do this entirely in the shader script.

To start, I’ve created a scene with geometry and a first person character controller. Feel free to use whatever you want as long as you have some way of moving your camera when playing to check the motion vectors, and geometry/meshes in your scene. Also make sure that whatever materials you have in your scene either use shaders that use Unity’s standard surface shader system or write to depth/motion vectors.

First, let’s get the core of the script out of the way.

The Start function includes a line to tell Unity to generate a motion vector texture for the current camera, and then a line in OnRenderImage to render with an image effect material.

To get started with the shader, let’s try to visualize the motion vectors so we can get a better understanding of what kind of data it will provide us with. The fragment shader is as follows:

Add this shader to a material, then add the script to your camera and the material to the DMmat variable on the script. Now when you move your camera in scene space (or if objects in view have movement relative to the camera), you should see motion vectors visualized as color, but only during movement.

Motion vectors provide you with a texture with two channels that calculate the positional difference objects render in camera space between this frame and the previous frame.

Motion Vector value chart:

(values in relation to previous frame)

G- represents Y-
R- represents X- | R+ represents X+
G+ represents Y+

R@0.0 & G@0.0 represents no motion

Now that we have access to motion vectors for each onscreen pixel, let’s use it to create UV displacement.


This is actually the basis for the datamoshing effect. All it’s actually doing it displacing pixels from the previous frame based on vector motion, similar to how video compression codecs allow us to skimp on providing samples for every pixel every frame and instead just displace previous pixels based on computed motion vector blocks to simulate motion at much smaller file sizes. The only thing that’s missing is recycling pixels from the previous frame.

Before we start, let’s go back to the script and add control for a global shader variable so that we can enable and disable using recycled pixels at any time.

Now back in the shader, we need to add a GrabPass pass just before the main shader. This will grab all onscreen pixels before rendering the effect, and is usually useful for displacement effects and for materials that render on objects that will need to read and modify pixels the object obfuscates. Here, we’ll be repurposing it for recycling modified camera pixels from the previous frame.

StudioMac

Just before the main pass, add this:

This will write the grabpass texture to _PR (pixel recycling). Now, in the fragment shader:

Now you have your basic datamosh effect! The shader will start recycling the same pixels from the previous frame once you hold down the left mouse button, displacing them further each frame by the motion vectors, and will clear by rendering the _MainTex texture when you let go.

For fun, we’ll add two more functions: Flooring the motion vector UV & using noise to determine what parts of the screen will update!

BLOCKING WITH UV POSITION ROUNDING

In video compression, you don’t have a motion vector for every single pixel since that would be very expensive in size. Instead, motion vectors are assigned to a small blocks of the image that move chunks of pixels according to the general motion. Over a stretch of numerous P frames, this causes more noticeable artifacts, but in it’s intended use with regularly refreshing I frames, it produces similar results to the original file with a huge benefit in compression size.

How To Datamosh

To simulate this, we’ll round the UV positions for reading the motion vector UV. Modify the first lines of the previous shader to redefine how we’ll sample for ‘mot’.

PER-BLOCK NOISE

For basic noise in shaders, I like to use this function:

And implement it into the fragment shader as follows:

I’m going to add one more line to the shader that will cause the motion vector data itself to be lossy. After defining ‘mot’, add this line:

Now you’ll get some very interesting lossy moshy artifacts.

If you enjoyed this tutorial or found it insightful, please consider supporting my Patreon so I can continue to explore more techniques and effects and have the time to write these kinds of tutorials!

You’ll also gain access to exclusive patron-only tutorials and writeups, teasers to upcoming projects, and much more in the near future.

most examples above use Apollo Pavillion model by Rory Peace, used for demonstration purposes
Made by Phil Tessier-Ashpool','source':'

Made by Phil Tessier-Ashpool

Datamosh download'},'hSize':null,'floatDir':null,'html':'','url':'https://www.youtube.com/watch?v=NjVLTX-gY7U','width':854,'height':480,'providerName':'YouTube','thumbnailUrl':'https://i.ytimg.com/vi/NjVLTX-gY7U/hqdefault.jpg','resolvedBy':'youtube'}'>

Datamosh Studio is a datamosh video editor. You can use this program to perform datamoshing on videos in a quick and easy manner. Find out more about datamoshing here.

This software is currently in beta and it may contain bugs. If you find a bug please report it here.

Features

  • Supports importation of AVI, WMV, MP4, FLV and MOV formats

  • Allows frame by frame editing of video

  • Exports to MP4 format for easy playing

  • Takes care of placing an IFrame at the beginning of the exported video

  • Allows customization of the format imported videos are imported to

You can't mix videos without audio with videos that have audio. Adding a silent audio track to the video before importing will allow you to use it with videos that have audio.

Videos made with Datamosh Studio

Made by Phil Tessier-Ashpool','source':'

Made by Phil Tessier-Ashpool

Datamosh Studio For Mac'},'hSize':null,'floatDir':null,'html':'','url':'https://www.youtube.com/watch?v=indzb4J66us','width':854,'height':480,'providerName':'YouTube','thumbnailUrl':'https://i.ytimg.com/vi/indzb4J66us/hqdefault.jpg','resolvedBy':'youtube'}'>

Wiki

Datamosh Studio Tutorial

Coming Soon

Bugs

You can't mix videos without audio with videos that have audio. Adding a silent audio track to the video before importing will allow you to use it with videos that have audio.

If any bugs are found please contact me here.

Datamosh Studio For Macbook Pro

Change Log

BETA v1.0
- version one created
BETA v1.0.1
- small fix to the way failed creation of projects are handled
BETA v1.0.2
- took wild stab in the dark to fix project creation error
BETA v1.0.3
- r̶e̶w̶r̶o̶t̶e̶ ̶p̶a̶r̶t̶s̶ ̶o̶f̶ ̶A̶V̶I̶ ̶c̶o̶n̶v̶e̶r̶s̶i̶o̶n̶ ̶c̶o̶d̶e̶ ̶t̶o̶ ̶u̶s̶e̶ ̶o̶w̶n̶ ̶c̶o̶d̶e̶ ̶i̶n̶s̶t̶e̶a̶d̶ ̶o̶f̶ ̶M̶e̶d̶i̶a̶T̶o̶o̶l̶k̶i̶t̶
- A̶V̶I̶ ̶c̶o̶n̶v̶e̶r̶s̶i̶o̶n̶ ̶s̶h̶o̶w̶s̶ ̶i̶n̶ ̶n̶e̶w̶ ̶w̶i̶n̶d̶o̶w̶ ̶n̶o̶t̶ ̶i̶n̶ ̶e̶m̶b̶e̶d̶d̶e̶d̶ ̶c̶o̶n̶s̶o̶l̶e̶ ̶d̶u̶e̶ ̶t̶o̶ ̶r̶e̶w̶r̶i̶t̶e̶s̶/̶
- Found an error with using frame rates with decimals in ffmpeg on some computers
BETA v1.0.4
- Constricted input of frame rate to integers only

Downloads

Datamosh Studio For Macbook

(Read the bugs section before downloading) Windows 64 bit download: Click Here
By downloading or using this software you agree to the terms and conditions of its EULA and the licenses of the 3rd party software it uses. Read the licenses below before downloading.

Datamosh Download

If you liked this program please consider donating.

.Net Framework 4.5.2 is required to run this program. Most computers should already have this installed. If not you can download it here.

Licenses

The EULA of this software can be found here.

This software uses FFmpeg licensed under the GPLv3.0 License, its source can be downloaded here.
This software uses MediaToolkit licensed under the MIT License, its source can be downloaded here.
MediaToolkit uses FFmpeg as well and it is licensed under the LGPLv2.1 License, its source can be downloaded from here.

FFMPEG was compiled with the following external libraries:
bzip2 1.0.6
Fontconfig 2.11.94
Frei0r 20130909-git-10d8360
GnuTLS 3.3.22
libiconv 1.14
libass 0.13.2
libbluray 0.9.2
libbs2b 3.1.0
libcaca 0.99.beta18
FreeType 2.6.3
Game Music Emu 0.6.0
GSM 1.0.13-4
iLBC 20141214-git-ef04ebe
Modplug-XMMS 0.8.8.5
libmfx 1.17
LAME 3.99.5
OpenCORE AMR 0.1.3
OpenJPEG 1.5.2
Opus 1.1.2
RTMPDump 20151223-git-fa8646d
Schroedinger 1.0.11-2.1build1
Snappy 20160108-0800b1e
libsoxr 0.1.2
Speex 1.2rc2
Theora 1.1.1
TwoLAME 0.3.13
vid.stab 0.98
VisualOn AMR-WB 0.1.2
Vorbis 1.3.5
vpx 1.5.0
WavPack 4.80.0
WebP 0.5.0
x264 20160118-git-a01e339
x265 1.9
XAVS svn-r55
Xvid 1.3.4
z.lib 2.0.4
XZ Utils 5.2.2
zlib 1.2.8
Licenses for each library can be found in 'licensesffmpeglicenses' in the download for this software.

Comments are closed.