Topic: Psychovisual RD & Psychovisual Trellis & Trellis & Deblocking...

I found this part of your forum which seems open to general discussion.  When you have a few minutes, could I please bother you for your opinion about what these settings (enhancements?) do and how they work?

I know the default setting for Psychovisual RD is '1'  I've read many differing opinions about how very subjective this is.  Also with Psy Trellis.  The default there is '0', however when a person tunes for 'Film', that setting goes to .15.  Some people use .20.  Now, from what I understand, Psy Trellis has something to do not so much specifically with preserving detail, but more with giving the 'impression' of more detail (or something like that...)

My understanding of Trellis itself is a bit more vague.  I really don't understand the difference between trellis 1 and trellis 2, except that whatever it does, I think with '2' it does it more continuously or comprehensively, I'm not sure.  I believe that you are supposed to use it, but I don't really understand what it does.

Also, I guess this is related, but Deblocking for tune--film is -1.-1.  But, some people who SEEM to be really experienced at encoding and want to preserve a lot of detial use -2,-2 or even -3,-3.

Now, generally speaking for myself, this is always going to pertain just to HD.  And, for my personal taste, I like a sharper, more detailed picture for the most part (the exception is MAYBE when a film is EXTREMELY grainy, like Criterion's Blu-ray of 'THE THIRD MAN' for example, which looks like it is bathed in snow, BUT it does indeed have amazing detail - so, that circumstance is a bit of a quandary to me, but I think I'm leaning toward more detail)  So, I generally set encodes (or re-encodes) for at least tune--film.  I'm currently trying deblock,-2,-2 to see how I like the results.  I've only done one so far, the new Blu-ray print of 'THE ART OF WAR', bringing it down to fit on a BD25.  The result looked really good, very defined.  I also use Trellis 2 and Psy Trellis .20 as a rule because it generally LOOKS like it comes out very detailed and sharp.

So, since you are VERY experienced at this, I just wanted to get your opinions about how these particular settings work and how you personally feel about them, if that would be okay.

Oh, BTW... I also use the setting that YOU suggested (and I THINK is incorporated in Hybrid, if I'm not mistaken) and that is setting the qcomp to .5 instead of the default .6.  I remember you saying that you preferred that.  In doing some further reading, it appears that that would take more bits from fast moving scenes and place them in more calm, static scenes, where I ASSUME the idea is that it would be more noticeable there.

I KNOW that all of these are somewhat interrelated with each other, but it would be nice to understand better how they work together.  BTW, I don't care anything about smaller file sizes or getting the most out of compression, which I know some of these things do help to do.  I almost ALWAYS am doing full Blu-rays on BD-25s, or in some cases, to force a re-encode of a bad encode.  So, I GUESS from what I understand, the bit rate will always be reasonably high (which may make a lot of this pointless, I don't know smile )

And, I DO remember you saying a number of times that the settings are not as vital as the filters one uses before encoding.  You gave me a LOT of very helpful advice in using AVS to alter the sharpness, contrast, and brightness in some cases which REALLY helped!

Thank you so much for whatever insight you can give me.


Re: Psychovisual RD & Psychovisual Trellis & Trellis & Deblocking...

I guess this is about x264 encoding.

About psychovisual rate distribution:
The main idea is to adjust the bit rate distribution (inside a frame) in a way that might hurt objective metrics  (like SSIM/PSNR/...) but should help with quality perception of a human.
Psy RD strength is meant to avoid blurring and trying to keep roughly the same image complexity in the input as in the output.

About deblocking:
The deblocking parameter in x264 is a modifier to the inloop deblocker.
Note that:

  • a inloop-deblocker and a normal deblocker are not the same thing.

  • it is really unwise to disable the inloop deblocker, since it is an integral step of the H.264 compression algprithm

  • so unless you disable the inloop deblocker, changing the deblock values only modified the strength of the deblocker

  • both the H.264 encoder and the H.264 decoder are aware of the in-loop deblocking settings

  • the effect of the in-loop deblocking is more visible on non intra frames, because inter frames are the frames with most of the references

the main idea of the in loop deblocker is that when higher quantizers are used it might be a good idea to blur the content a bit before using it as reference since higher quantizers might produce artifacts which otherwise would be used as reference for frames that refer to the current content. The in-loop deblocker has two parameters which are roughly mapped to a strength and a threshold parameter. The threshold modifies the minimum quantizer which will be affected by the in loop deblocking where as the strength parameter modifies how strong the deblocking should be if it gets applied.
So when would it make sense to heighten the threshold&strength?
It would make sense if you do not care that much about small details, which is mainly the case if a. you aim for low bit rates b. your content doesn't contain fine details to begin with (like with classic cartoons and animes) or c. you in general think that H.264 tries to preserve to many details when using high quantizers.
Okay, so when would it make sense to lower the threshold&strength?
It would make sense if you think that H.264 should try harder to preserve details even in inter frames. '--film' is trying to get tell x264 to try to preserve as more fine details even if this means lowering the general quality a bit.
Personally I normally only use values between -1/-1 and 1/1 when encoding, since I like the default inloop deblocking sthrength and threshold.

about trellis:
Trellis quantization is a method to adjust the quantization results, to some higher trellis values look better and to some lower trellis values are more pleasant. So if you like higher trellis values that is fine. smile

adaptive quantization:
This is the main option to tell x264 to move bits more to flat areas or edges. smile

All the values here are mainly for tweaking the results so be aware that what might be a good choice for source A might not be that optimal for source B and yes, tweaking the filtering of the input is normally more important than tweaking these settings.

Cu Selur

3 (edited by latheofheaven 2015-01-26 02:45:03)

Re: Psychovisual RD & Psychovisual Trellis & Trellis & Deblocking...

Hey thanks Selur; I sure appreciate you taking the time to explain this in detail AND to give me your opinions on it!

So, what you are saying about Trellis (1 or 2) then is that is it COMPLETELY subjective...?  And, is the same also true with Psy-Trellis (.00 - .20)



Re: Psychovisual RD & Psychovisual Trellis & Trellis & Deblocking...

Trellis does an additional optimization of the quantization so if you use trellis I would recommend to use trellis 2.
Also the effect of trellis is most visible on anime content and when lower bit rates are used, since then the additional optimization helps more.
Psy trellis is helps to additionally save some fine details, which most people will probably not notice during playback. For sources with fine grain, fog or gradients trellis and psy-trellis should be worth to playing with but for general reencoding settings I would probably save the encoding time (trellis expensive). (As a side note, for normal content I would, if I use psy-trellis probably try values > 0.5.)

Cu Selur


Re: Psychovisual RD & Psychovisual Trellis & Trellis & Deblocking...

Wow, interesting, thanks!