1 (edited by Kdmeizk 2016-05-29 15:39:02)

Topic: [SOLVED] AVI file: superindex entry limit + index entries per block

Hi Selur,

I saw a few answers from you about this kind of question but outside of this forum. I prefer ask to you before others.
I'm really confused with these settings from VirtualDub. I want to find something which allows me to set PROPERLY these both settings, but excepted all answers "should be this value" without arguments, I didn't find my happiness...
I tried several things and it seems "superindex entry limit" isn't a maximum because the size of the output file is higher with a high value than a lower value.
For "index entries per block", I don't understand what is this.



SOLUTION:

  • Superindex: One superindex every 2GB of the file involved. One superindex every 1GB is possible but less isn't useful, it grows the file for no benefit. One superindex more than 2GB is a bad idea because it can smother the reader/computer.

  • Index entries per block: In AVI structure, there are index chunks. This setting limits the number of index entries per chunk (here VirtualDub uses the word "block"). More index entries than necessary for the file is completely useless. You have to choose the minimum required as possible.
    This setting is directly linked with Superindex. Example: 4 Superindex, 500 Index entries = 4*500 = 2000 Index entries in the file.
    One index entry is needed per video frame. But for VirtualDub, this rule is slightly different.
    In this setting, you need (here is an example): Number of video frames of all video streams/Number of 2GB block of your file = 7898/3 = 2632,xxxxxx (if it's a float number, you need +1, integer numbers are accepted only) = 2633.
    In the case where you have 2 video streams, you need the number of video frames of stream1 + number of video frames of stream2. Same reasoning for more than 2 video streams.
    Audio streams don't affect directly this setting. There is a problem only if audio streams are big. In this case, VirtualDub asks for more Superindex. You have to add more index entries only, don't touch the Superindex. You can try values to find the minimum required or put a larger value if you don't want to waste your time, but the file will become "larger" (1 index entry = 12 bytes).

To help you:

---------------------------
VirtualDub Error
---------------------------
AVIOutput: Not enough superindex entries to index AVI file.  (5 slots required, 4 slots preallocated)
---------------------------
OK   
---------------------------

means for our first example, the number of index entries to add is between 1 to 500.
We are in the case where audio data isn't interleaved (Audio -> Interleaving). By the way, Interleaving is very strongly discouraged for compatibility problems.

2

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Moved the thread, since it has nothing to do with Hybrid.

Unless you create humongous out files the defaults of these settings should be fine.

This is the Virtual Dub description:

For AVI files larger than 2GB, approximately one superindex entry is needed for every 2GB block of file, and 1-2 entries for every video frame. The default settings of 256/8192 are good for about 0.5TB and nine hours. Each additional superindex entry adds 16 bytes per stream to the header.

So for every 2GB block you need 1 superindex, with the default 256 you should be fine for 500GB. (note that each superindex increases the header size by 16 byte)
Inside each block, you need up to 2 index entries per frame.
So the default 256 and 8192 mean that you can save ~500GB and 256/2*8192 = 1 048 576 frames with 30 fps this would mean 34952 sec = ~582 min = ~9 1/2 hours of video.

Depending on what you do you need to increase these settings.

A general warning: Not all avi container handlers fully support the specification, so messing with these settings might cause problems with some applications.

Cu Selur

Ps.: http://www.alexander-noe.com/video/docu … on/avi.pdf might also be interesting.

3 (edited by Kdmeizk 2016-05-29 15:56:57)

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Sorry for the place of this thread.

Inside each block, you need up to 2 index entries per frame.

Block, one block = one superindex here ?
Have you an idea what "index entries" is for the .AVI ? Because, up to 2 per frame, I don't understand.

(I assumed only ->) I thought 1 per frame ok, for the seeking. But up to 2, it seems impossible with my assumption.


With my researches yesterday, I found your link, don't worry!

4

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Block, one block = one superindex here ?

Yes.

Have you an idea what "index entries" is for the .AVI ?

avi has multiple indices's to address frames and depending on the frame type you need one or two index entries.

5 (edited by Kdmeizk 2016-05-29 13:47:46)

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Ok. You know when exactly use 1 or 2 index entries? I thought for key-frame?

And, more than 2 index entries per frame would give what?
It's probably a rule to follow said by developers, but I would like to be sure.

6

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

I knew some time ago, but haven't looked into avi for years and forgot. smile
iirc. alex noe wrote about it inside his pdf, not sure whether this was frame or muxer dependent,... the whole avi container wasn't that well documented. smile
Iirc it was mainly a compatibility thing.

Cu Selur

7 (edited by Kdmeizk 2016-05-29 13:48:15)

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Did you remember if each subindex handle one stream per subindex? Or more like video + audio streams?
If I understood, one subindex is needed per video frame to go to this frame for example.

So if I'm right, the sense of these settings is here to let the choice to the user if the seeking is possible to all frames or less.

8

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

iirc you need one subindex per frame (independent if it's audio or video frames), but the setting in Virtual Dub is meant per stream.

9 (edited by Kdmeizk 2016-05-29 13:46:48)

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

In his source code, the setting for "Index entries per block" is related to "xxxxxxx.xxxSubIndexLimit" something like this.
For superindex, it's superindex.

So, I assume subindex what you said it's "Index entries per block".
If it's right, so yes VirtualDub and you said:

and 1-2 entries for every video frame.

---------------------------------------------------------------------------------------------------------------------------------------------------------------

but the setting in Virtual Dub is meant per stream.

So if I have one video + 4 audio channels, I need 5 or 10 index entries per video frame ?

10

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

I need 5 or 10 index entries per video frame ?

Not unless:
a. each channel is a separate stream
b. each audio frame has the same size as a video frame (which normally is not the case)

normally just using the potential video requirements as goal should be fine

11 (edited by Kdmeizk 2016-05-29 16:01:59)

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Can you explain a bit more for the b. point ? Why ?

Or, may be, any audio streams have no links with this index setting in avi. The seeking for audio stream would be different?
I think this because I find NOTHING about the seeking and audio streams for avi file...

12

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

I think this because I find NOTHING about the seeking and audio streams for avi file...

Seeking and synching always works around of indexes. -> Indexing in .avi files is the interesting thing.

Can you explain a bit more for the b. point ? Why ?

Why audio frames have a different size/interval than video frames?
Because they are designed that way. big_smile
It really gets tricky if you have audio formats which have changing frame sizes/intervals like newer audio formats have, but those are normally not supported inside .avi; or most splitters can't handle them. (opus, ogg, aac,..)
For audio frames normally contain a specific number of audio samples, and for video a frame normally contains a fixed number of pixels. Since the audio sample count differs with a. the compression format and b. the sample rate (newer audio codecs don't always use a fixed sample rate wink), the duration of an audio frame normally differs from the display duration of a video frame, which is why a/v cutting isn't as simple as most people think and why you normally have indices an a/v synch infos inside different containers. wink

Cu Selur

13

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Oh, not this kind of why. I wonder, in the case of there is a link with this index in avi file, why you agree with me only if an audio frame = a video frame hmm

14

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Because only if audio frame size = video frame size it seems accurate to assume that they will need the same account of indices, like you did.

15 (edited by Kdmeizk 2016-05-29 16:08:55)

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

I think, like the author of VirtualDub said, one super index is needed around each 2GB of the file. It's a rule and could produce bad things if this rule wasn't respected. But, VirtualDub seems to forbid the violation of this rule with:

---------------------------
VirtualDub Error
---------------------------
AVIOutput: Not enough superindex entries to index AVI file.  (3 slots required, 1 slots preallocated)
---------------------------
OK   
---------------------------

If I'm right, so it remains "Index entries per block".
I think this "block" is these each 2GB around in a file.
So, this setting seems to be a low accuracy, because the number of index entries can be very often lower for the last block of the whole file than others blocks in the file, which are fully blocks (2GB around, because it's the rule).
But, seemingly the total number of index entries can be entered in the attribute higher than the true total of index entries. It's probably related with the super index (I will look again but not at 2:00 A.M...) which must have a number of index entries per block of the file, like this setting in VirtualDub.
Btw, the size of a chunk seems be split with a "compatibility value", so we haven't the right to change a size of chunk, which would change the number of index entries.


Finally, I don't know why I was "stubborn" with these both settings. At the end, it seems so evident...

16

Re: [SOLVED] AVI file: superindex entry limit + index entries per block

Yes, if you don't have enough indices you run into an overflow and create a broken file. Having too many indices only wastes a bit of overhead.