[vdr] Re: Recordings menu sluggish - *the* solution.

[Klaus Schmidinger <Klaus.Schmidinger@cadsoft.de>]

Carsten Koch wrote:
> 
> Klaus Schmidinger wrote:
> > Carsten Koch wrote:
> >
> >>Hi,
> >>
> >>we have discussed this subject here before.
> >>I believe we consent that the main reason for the big delays
> >>when opening the recordings menu is the fact that vdr reads
> >>and writes huge amounts of data from/to the nnn.vdr files,
> >>the kernel tries to cache these, thus moving much more useful
> >>stuff (like cached directory entries and cached small files)
> >>out of the memory cache.
> >>It usually makes no sense at all to cache the nnn.vdr files,
> >>as they normally are read/written only once.
> >>
> >>I recently found out that kernels > 2.4.9 seem to have exactly
> >>the solution we were looking for.
> >>
> >> From "man 2 open":
> >>
> >>        O_DIRECT
> >>               Try to minimize cache effects of  the  I/O  to  and
> >>               from  this file.  In general this will degrade per­
> >>               formance, but it is useful in  special  situations,
> >>               such  as  when  applications  do their own caching.
> >>               File  I/O  is  done  directly  to/from  user  space
> >>               buffers.  The I/O is synchronous, i.e., at the com­
> >>               pletion of the read(2)  or  write(2)  system  call,
> >>               data   is  guaranteed  to  have  been  transferred.
> >>               Transfer sizes, and the alignment  of  user  buffer
> >>               and  file offset must all be multiples of the logi­
> >>               cal block size of the file system.
> >>               This flag is supported on  a  number  of  Unix-like
> >>               systems;  support  was  added under Linux in kernel
> >>               version 2.4.10.
> >>               A semantically similar interface for block  devices
> >>               is described in raw(8).
> >>
> >>I guess a patch that uses this should be trivial.
> >>I have not tried it yet.
> >>Is there anything obvious that speaks against this being a nice
> >>solution fo our problem?
> >
> >
> > The only thing that might be a problem is
> >
> >         Transfer sizes, and the alignment  of  user  buffer
> >         and  file offset must all be multiples of the logi­
> >         cal block size of the file system.
> >
> > This might make things more complicated than just setting
> > the O_DIRECT flag.
> 
> Oops, you are right.
> What would be involved in order to align everything to, say, 4k boundaries?
> I guess every I-frame should start at such a 4k boundary as well, which would
> make it unnecessary to special-case the treatment of the first block when using
> the fast forward/backward and jump functions.

I'm afraid this would require some major "surgery", because as it
is now VDR assumes that it can simply read a frame no matter what
size it is, and at any file offset.

> Of course the limitation that data must be aligned to 4k boundaries gives me
> the feeling that vdr performance will actually *improve* quite a bit when you
> use O_DIRECT, as the kernel can then DMA directly between the device and VDRs
> buffers in user space, right?

Maybe, but we would probably need an additional layer of data
handling inside VDR for this.

> > Apart from that the idea seems good, since then the system
> > probably wouldn't lose the directory information.
> 
> It would be wonderful if that would work.
> I had equipped my vdr with 512MB of RAM when I first noticed that its original
> 128MB were always full. Of course, as we know now, that did not help at all...

The whole problem, AFAICS, is that Linux uses the entire available
RAM (or at least most of it) as disk buffer. So when large files are
handled, all other things are dropped.

Maybe it would help if there was a way to tell the kernel to
use only that much RAM for buffering a particular file? Don't
know, though, whether there actually is such a feature...

The recording directory handling will be changed, anyway, to
keep the information readily available. Maybe then this whole
thing won't be a problem any more...

Klaus


-- 
Info:
To unsubscribe send a mail to ecartis@linuxtv.org with "unsubscribe vdr" as subject.