MVTools

Abstract

Table of contents

• I) About MVTools
• II) Function descriptions
• III) Examples
• IV) Disclaimer (don't skip that part, but I don't force you to learn it either)
• V) Revisions

I) About MVTools

...

II) Function descriptions

Common parameters

Filters that use motion vectors have common parameters. Those are the scene-change detection thresholds, and the mmx / isse flags. They also use one or several vectors stream, which are produced by MVAnalyse.

int thSCD1 : threshold which decides whether a block has changed between the previous frame and the current one. When a block has changed, it means for me that motion estimation for him isn't relevant at all. It occurs for example at scene changes. So it is one of the thresholds used to tweak the scene changes detection engine. Default is 300. Raising it will lower the number of blocks detected as changed. The threshold is compared to the SAD ( sum of absolute difference, a value which says how good the motion estimation was ) value. If you use 4x4 blocks, you should divide that value by 4.

int thSCD2 : threshold which sets how many blocks have to change for the frame to be considered as a scene change. It is ranged from 0 to 255, 0 meaning 0 %, 255 meaning 100 %. Default is 130 ( which means 51 % ).

bool mmx : flags which allows to disable mmx optimizations if set to false. Default is true. If your computer doesn't support mmx optimizations, it will be disabled by default anyway ( and you won't be able to activate them )

bool isse : flags which allows you to disable isse optimizations. Default is true, and it behaves just like mmx.

MVAnalyse

MVAnalyse (clip, int blksize, int pel, int level, int search, int searchparam, bool isb, int lambda, bool chroma, int delta, int idx)

blksize : Size of a block. It's either 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.

pel : it is the accuracy of the motion estimation. 1 means a precision to the pixel. 2 means a precision to half a pixel. Value can only be 1 or 2. Default is 1.

level : it is the number of levels not used in the hierarchal analysis made while searching for motion vectors. The lower the better. It is kept variable for study's purposes only. Default : 0.

search, ssearchparam : search decides the type of search, and searchparam is an additionnal setting for this search :

• search = 0 : 'OneTimeSearch'. searchparam is the step between each vectors tried ( if searchparam is superior to 1, step will be progressively refined ).
• search = 1 : 'NStepSearch'. N is set by searchparam. It's the most well known of the MV search algorithm.
• search = 2 : Logarithmic search, also named Diamond Search. searchparam is the initial step search, there again, it is refined progressively. It's the default search ( with searchparam = 1 )
• search = 3 : Exhaustive search, searchparam is the radius. It is slow, but it gives the best results, SAD-wise.

isb : allows to choose between a forward search ( between the current frame and the previous one ) and a backward one ( between the current frame and the following one ). isb = false means forward search ( isb stands for is backward ).

lambda : set the coherence of the field of vectors. The higher, the more coherent. However, if set too high, some 'true' motion vectors can be missed. Default is 0, but values around 400 - 2000 are strongly recommended.

chroma : set to true, it allows to take chroma into account when doing the motion estimation. Default is false.

delta : set the frame interval between the reference frame and the current frame. By default, it's 1, which means that the motion vectors are searched between the current frame and the previous ( or next ) frame. Setting it to 2 will allow you to search mvs between the frame n and n-2 or n+2 ( depending on the isb setting ).

idx : allows the filter to store the interpolation he made during the motion estimation, in order them to be reused by another instance of the filter on the same clip. It allows for example, when doing a forward & backward search on the same clip, to avoid to compute twice the bilinear interpolation, if pel = 2. By default, a unique negative number is given ( unique for each filter ). If you use it, you should always use positive values, and you should only use the same value for filters which work on the same clip ( else, the analysis won't work properly ).

MVCompensate

MVCompensate (clip, clip vectors, bool scbehavior, int mode, int idx)

Do a full motion compensation of the frame. It means that the blocks pointed by the mvs in the reference frame will be moved along the vectors to reach their places in the current frame.

scbehavior ( by default true ), decide which frame will be kept on a scene change. If true, the frame is left unchanged. Else, the reference frame is copied into the current one.

mode can be either 0 ( default ), 1 or 2. 0 means it uses the compensation made during the mvs search. 1 means it recomputes that compensation ( because you may want to apply vectors to a different clip that the one on which you searched ). 2 means it recomputes the compensation, but it does it in-loop, meaning that the vectors will be applied to the last frame computed. Results will be ugly, and that mode shouldn't be used except if you know what you're doing.

idx works the same way as idx in MVAnalysis. It is used only with mode = 1.

MVDenoise

MVDenoise (clip, clip mvs [,...], bool Y, bool U, bool V, int thT, int thSAD, int thMV)

Makes a temporal denoising with motion compensation, only on the luma. Reference frames are motion compensated and then merged into the current frame.

The first threshold, thT, decides whether the pixel which comes from the previous or the following frame has to be taken into account. If this pixel differs more than thT from the pixel of the current frame, it is not used.

The second one, thSAD, decides whether the block has changed or not ( same principle as thSCD1 ). If it has changed, the pixels aren't merged with those of the previous or following frame.

thMV is the vector's length over which the block isn't used for denoising.

Finally, Y, U and V tell which planes should be denoised.

Defaults are : Y, U and V are true, thT = 10, thSAD = 200 and thMV=30.

MVMask

MVMask (clip, clip vectors, int ml, float gamma, bool showsad, int Ysc)

Creates a motion mask from the motion vectors' length. It builds a better mask than MotionMask ( MaskTools ) because motion vectors are a lot more reliable than the algorithm of MotionMask.

The mask is output only on the luma plane. 0 means no motion at all ( the length of the motion vector is null ), whereas 255 means maximum motion, and is defined by ml. When the vector's length is superior or equal to ml, the output value is 255. gamma is used to defined the relation length / value. gamma = 1.0 implies a linear relation, whereas gamma = 2.0 gives a quadratic relation.

showsad allows to build a mask of the SAD values instead of the vectors' length. And finally, Ysc is the value taken by the mask on scene change

Defaults are : ml = 100, gamma = 1.0, showsad = false and Ysc = 0.

MVShow

MVShow (clip, clip vectors, int scale, int sil, int tol, bool showsad)

Shows the motion vectors. scale allows to enlarge the motion vectors, in order for example to gain in accuracy ( when pel > 1 and scale = 1, you can't see a variation of less than one pixel ). sil allows to see a different level of analysis ( when searching for motion vectors, a hierarchal analysis is done, and it may be interesting to see what happens at higher levels ). tol is a tolerance threshold. If the distorsion induced by the motion vector is over tol the vector isn't shown. Finally, showsad allows to show the mean SAD after compensating the picture.

Defaults are : scale = 1, sil = 0, tol = 20000 and showsad = false ( which shows all vectors ).

MVChangeCompensate

MVChangeCompensate (clip vectors, clip)

Allows to change the compensation stored into the mvs stream.

Deblock

Deblock (clip, int quant, int aOffset, int bOffset)

It does a deblocking of the picture, using the deblocking filter of h264. It doesn't use motion vectors, however, it's usefull to deblock the result of MVCompensate.

quant : the higher the quant, the stronger the deblocking. It can range from 0 to 51.

aOffset : quant modifier to the blocking detector threshold. Setting it higher means than more edges will deblocked.

bOffset : another quant modifier, for block detecting and for deblocking's strength. There again, the higher, the stronger.

Defaults are quant = 25, aOffset = bOffset = 0. If quant + aOffset is inferior to 16, the filter does nothing at all. The same goes for quant + bOffset.

MVIncrease

MVIncrease (clip, clip vectors, int horizontal, int vertical, int idx)

It allows to use vectors computed on a reduced version of the clip in order to make a compensation on a clip with the original size.

horizontal is the horizontal ratio between the width of the clip and the width of the reduced clip.

vertical is the vertical ratio between the height of the clip and the height of the reduced clip.

idx works the same as in MVAnalyse

Corrector

Corrector (clip analyse_0, clip output_0, clip analyse_i[, ...], clip output_i[, ...], int mode, int th)

This filter allows to construct a clip, pixels per pixels, from the output clips, based on the analyse clips. There's two mode.

mode = 0 : the output pixels whose collocated analyse pixels differs from the analyse_0 clip by less than th are averaged to form the new pixel. If none are found, the output_0 pixel is chosen.

mode = 1 : the output pixel whose collocated analyse pixels is the closest from analyse_0 clip is chosen. If none differs by less than th, the output_0 pixel is chosen.

Defaults are mode = 0 and th = 10.

MVDepan

MVDepan (clip, clip vectors, bool zoom, bool rot, float pixaspect float error, bool info)

Get the motion vectors,  estimate global motion  and put data to output frame in special format for DePan plugin (by Fizick).

Inter-frame global motion (pan, zoom, rotation) is estimated by iterative procedure, with good blocks only.

Rejected blocks: 1) near frame borders; 2) with big SAD (by thSCD1 parameter); 3) with motion different from global.

zoom and rot parameters switch zoom and rotation estimation, pixaspect  is pixel aspect (1.094 for standard PAL, 0.911 for standard NTSC),  error is maximum mean motion difference.

The frame estimated global motion is switched to null for big motion error or at scene change  (by thSCD1, thSCD2 parameters).

info parameter allows to type global motion info for debug.

Defaults are : zoom = true, rot = true, pixaspect = 1.0, error = 15.0, info = false.

III) Examples

To show the motion vectors ( forward ) :

vectors = source.MVAnalyse(isb = false)
return source.MVShow(vectors)

To show the backward one :

vectors = source.MVAnalyse(isb = true)
return source.MVShow(vectors)

To use MVMask :

vectors = source.MVAnalyse(isb = false)
return source.MVMask(vectors)

To denoise :

backward_vectors2 = source.MVAnalyse(isb = true, lambda = 1000, delta = 2)
backward_vectors1 = source.MVAnalyse(isb = true, lambda = 1000, delta = 1)
forward_vectors1 = source.MVAnalyse(isb = false, lambda = 1000, delta = 1)
forward_vectors2 = source.MVAnalyse(isb = false, lambda = 1000, delta = 2)
return source.MVDenoise(backward_vectors1, backward_vectors1, forward_vectors1, forward_vectors2, tht = 10, thsad = 300)

To deblock the compensation stored into a mvs stream

vectors = source.MVAnalyse(isb = false, lambda = 1000)
compensation = source.MVCompensate(vectors, mode = 0)
compensation = compensation.Deblock()
vectors = vectors.MVChangeCompensate(compensation)

To denoise with pel = 2, efficiently :

backward_vectors2 = source.MVAnalyse(isb = true, lambda = 1000, delta = 2, pel = 2, idx = 1)
backward_vectors1 = source.MVAnalyse(isb = true, lambda = 1000, delta = 1, pel = 2, idx = 1)
forward_vectors1 = source.MVAnalyse(isb = false, lambda = 1000, delta = 1, pel = 2, idx = 1)
forward_vectors2 = source.MVAnalyse(isb = false, lambda = 1000, delta = 2, pel = 2, idx = 1)
return source.MVDenoise(backward_vectors1, backward_vectors1, forward_vectors1, forward_vectors2, tht = 10, thsad = 300)

To use MVIncrease :

vectors = source.reduceby2().mvanalyse(isb = true)
return source.MVIncrease(vectors, horizontal = 2, vertical = 2)

To use MVDepan with Depan plugin (DepanStabilize function example):

vectors = source.MVAnalyse(isb = false)
globalmotion = source.MVDepan(vectors, pixaspect=1.094, thSCD1=400)
DepanStabilize(source, data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094)

IV) Disclaimer

This plugin is released under the GPL license. You must agree to the terms of 'Copying.txt' before using the plugin or its source code.

V) Revisions

0.9.9.1

• No need anymore of stlport_vcxxxx.dll

0.9.9

• Filter added : Corrector
• Filter added : MVIncrease
• New available blocksize : 16
• New parameter in MVAnalyse : chroma
• Changes in the core

0.9.8.5

• MVCompensate changed : a new parameter, idx, which works as idx in MVAnalysis, and which allows speed up when compensating the same frames several times.

0.9.8.4

• ME takes into account the chroma now, as requested by tsp.

0.9.8.3

• Added Corrector function, as requested by scharfi.

0.9.8.2

• New function MVDepan (added by Fizick) for Depan plugin.

0.9.8.1

• Several bugfixes

0.9.8

• Yet another little changes in the filters' syntax. The core changed a lot, in order to gain speed were it was possible. However, by default, the speed gain won't be visible, you'll need to configure correctly the analysis filter through its "idx" parameter in order to gain speed ( in the mode "pel" = 2 ).
• Bugfixes in MVDenoise, and chroma denoising in MVDenoise.
• Now, the filters down the filter's chain tell to the analysis filter if they need the compensation, so you don't have to worry about that at the analysing stage.

0.9.7

• Yet again, a lot of rewriting. Interpolating filters are disabled ( for the moment ), all the other filters work and should be considered as stable. Syntax has changed a lot, and will change again before reaching 1.0 ( if it's reached one day ). Changes mainly affect MVAnalyse. Two new filters : Deblock and MVChangeCompensate.

0.9.5

• Huge rewritting of the core engine. Vectors are now searched with a precision up to the pixel ( because no other filters can use yet more precise vectors, except MVShow ). The search engine is now fast ( which doesn't mean necessarily that the filters which use it are fast )
• A new parameter for MVAnalyse : lambda. See the documentation of the filter to see how it works
• MVDenoise works better.

0.9.4

• Vectors can be saved to a file. In order to do so, add the parameter filename="C:\foo.bar" to the filter MVAnalyse. If the file doesn't exist, vectors will be saved into it. If it exists, vectors will be read from it. But, be warned :
  • The file for a whole movie will be around 500 MB
  • Saving / reading from a file need for the moment a linear access to the frames, so it has to be used only when encoding the movie, not when doing random access in it.
  • The speed gain is not as great as one may hope, because SADs can't be saved ( it would take too much space ) and so have to be recomputed.
• The filter MVDenoise now works on 5 frames, and its parameters are now "thT" and "sadT" ( have a look in the documentation to see how they work ). It works nice ( very good for heavy denoising )
• The scene change detection thresholds have slightly changed. Now, a block has changed if its SAD it over thSCD1. The default for thSCD1 is 300, and for thSCD2 it is 130. It orks well ( better than the previous SCD engine ).
  

0.9.3

• Last cleanings in the search of the motion vectors. It should be slightly faster
• More search parameters can be set by the user, especially the search algorithm. See the documentation
• Server / client implemented. You now first have to use MVAnalyse, and then the filter you want. Look at the documentation and at the examples I'll give alter.
• MVCompensate is separated from MVShow ( it's more logic that way ). For the moment, it doesn't move the chroma ( same behavior as MVShow in the latest releases )
• Some cleaning in MVBlur / MVInterpolate / MVConvertFPS, but still some work to do. Now, MVBlur blurs around the frame, not between the frame and the previous one.
• Half of the work is done for writing vectors to a file. But the resulting file will be large ( around 500 MB - 1 GB I guess ).
• MVDenoise is slightly faster ( at least it should )
• Copies are optimized inside the filter, thanks to avisynth's copy functions.
• MVShow can display the mean SAD of the compensation ( using showsad = true )

0.9.2.1

• MVInterpolate makes its come back.
• MVConvertFPS should work on the last few frames of the clip

0.9.2

• MVInterpolate doesn't exist anymore.
• MVBlur and MVConvertFPS have been improved. They also have got new parameters, have a look at the documentation.
• MVShow gets back its compensate mode ( MVShow(cm = true) )