Video Compression Breakthrough discovered! (POLL)

[tvshowman]

Now that I have everyones attention heh...

First I would like to say that I am shocked that not a single person has replied to my request for some 2 pass mencoder command line help after 59 views
http://forums.nextpvr.com/showthread.php?t=41968

I suspect either people don't know much about FMP4 and Mencoder or people are much to busy to reply.

So I have decided to get more people hooked on the codec that I have been using for more than a year now.

To get you interested I have uploaded Video Samples of my compression tests using some of the popular formats that are being used by members of the board.

Many posts have hinted that it is better to buy larger hard drives rather than compress your video files... Well I have a few Huge hard drives and they get filled up rather quickly especially if you record the movie channels 24/7 so that you can watch whatever you want whenever you want. The problem is that Mpeg recorded files are huge and take up to much space for a single 2 hour movie. For me the file size for a typical movie is around 4 gigs using PVRX2's Medium encoding settings for my hauppage 500 card.

I had been searching a year ago for the ultimate conversion utility, script or auto program and man let me tell you that nothing compares to the FMP4 codec for sheer speed and thankfully it has an above average picture quality.

FMP4 is a Video format by FFmpeg and a description can be found at this wiki

After trying numerous frontends for FFmpeg.exe and Mencoder.exe and trying to read and learn about the command line scripts they used for various profiles. One conversion program used an FMP4 script that I stumbled upon and fell in love with the speed and was quite happy with the results. I then started experimenting with changing the values to the line to see what the heck it does. (I know nothing about video codes just a hobby)

So this is my current compression script that I auto convert all recording to.

Code:

mencoder.exe %1 -ovc lavc -oac mp3lame -lavcopts vcodec=mpeg4:vqscale=3:vqmin=2:vqmax=31:keyint=250:me_range=1000 -lameopts preset=standard -o "%~dp1%~n1.avi"

vqscale=3 - a lower value raises the video quality resulting in a higher file. higher number = smaller file and worsening video quality...

If you want to deinterlace the picture simply add
-vf pp=md

To rescale the video add:
-vf scale=624:352

To deinterlace and scale:
-vf pp=md,624:352

Now my findings....

Test rig Q6600 quad core XP 3 gigs ram

I recorded a 30 second commercial using Variable bitrate 3500 - 7200 @ 720x480

Filename-----------------------Resolution-------FPS_EST_Enc--------time2encode--------------Filesize----
seaworld.mpg-------------------720x480----------029.99fps-----------0:30.00s----------------16,625k
seaworld_xvid_720x480.avi------720X480----------023.82fps-----------0:24.12s----------------06,774k
seaworld_h264_720x480.avi-----720X480----------016.99fps-----------1:10.40s----------------07,471k
seaworld_fmp4_720x480.avi-----720X480----------105.00fps-----------0:08.75s----------------10,637k
seaworld_fmp4_624x352.avi-----624X352----------100.00fps-----------0:09.28s----------------07,303k
seaworld_fmp4_448x336.avi-----448X336----------119.00fps-----------0:07.92s----------------05,696k

By not resizing the movie resulted in a much larger file in my test however usually in a 2 hour movie the results were about 2 gigs and it is important to understand that the codec adjusts to the colors and movements in the movie so the file size will rise or lower depending on the content. More black like a talk show results in much smaller files.

Resizing to 624x352 resulted in file sizes similar to H264 and Xvid 2 pass encoding. I decided to even go even smaller to 448x336 and the files are way smaller and I was happy with the picture on my 42inch LCD TV @ 1920x1080 resolution. Its only TV and if I was to convert a DVD I would use something else. I am planning on getting HDTV cable and purchasing the HD-PVR at some point.

Note-
Encoding using h264 or Xvid requires much to much cpu horsepower and PVRX2 becomes much slower during encoding to those formats using my pvr rig which is a dual core with 2 gigs ram. once comskip kicked in you can firget about doing anything in pvrx2 until comskip finishes.

The FMP$ encoding is much less cpu intensive and on my dual core rig I see about 60 - 70fps encoding using mencoder. I am also able to encode 2 files at the same time with only a 10%fps loss and no sluggish in PVRX2.

On any given day my pvr is recording 2 shows at once while encoding 2 shows at once while I am watching a recorded show lol. Also I am able to download and load a huge tvguide.xml without problems.

So I want you to judge for yourself my findings so that you can determine if FMP4 encoding will be worth it to you if you are currently encoding to xvid or h264.

Please keep in mind that the encoding settings are only for standard tv and if you wanted to convert a DVD I would only recommend that you use the Xvid or H264 methods to obtain the highest quality video.

But if you don't care about an exact copy of your recordings and want to have the fastest encoding than you might want to try out FMP4...

Here are the Video examples...
52,613k
http://www.4shared.com/file/102625718/dd...sults.html

You can use Video Lan player to view the videos without having FMP$ installed on your computer.
http://www.videolan.org/vlc/

If you want to view the videos in PVRX2 then you need to install FFDshow,
http://www.afterdawn.com/software/video_..._20081219_

WARNING #1 - The video will freeze PVRX2 sometimes using Windows XP SP2 and the only cure is to have SP3 installed and there is no work around. Installing SP3 and you will have no further issues.

WARNING #2 - FFDshow should be configured in your config.exe settings for PVRX2 and you must select the checkbox to use FFDshow always.
When playing the file in PVRX2 you may only see a black picture. You will need to configure PVRX2 to be in window mode so that you can view the FFDshow popup window and select always use FFDshow and stop bugging me option and then the files will always play. After that you can go back to the config.exe and select full screen exclusive again. I use Nvidia's MPEG decoders for best picture quality in PVRX2 however you can any decoder you have installed..

Use this version of Mencoder for the fastest encoding.
http://sourceforge.net/project/showfiles..._id=678601
Direct
http://sourceforge.net/project/downloadi...a=20200751

H264 encoding script used - (Post number Two) posted by sgilani
http://forums.nextpvr.com/showthread.php?t=40978

Xvid 2 pass script - (Post number 35) posted by BTJustice
http://forums.nextpvr.com/showthread.php?t=41060

If you took the time to view the Video examples please vote in the poll!

Have fun watching TV

Example
Late night with Jay Leno
Original Mpeg - 2,355 megs
using FMP4@448x336 - 578 megs

:thumbs up!:

If you are recording tv with a snowing picture you can forget about using this to convert as the files will be much bigger...
chit in chit out

[zehd]

I can only speak for myself, but I haven't been replying because I don't have the time to be bleeding edge. I'd rather refine existing processes than search for and play with new ones, that may or may not have a limited life or interest.

Of course the best way to get a codec popular is to build interest, and its great you're trying to do that. Good luck

[Reddwarf]

It IS interesting, yes, but unfortunately the MENCODER tends to cut my recordings down to 27.5 seconds when it converts them no matter how long they are, FFMPEG does not... Never figured out why.

Can this codec be used with ffmpeg?

[tvshowman]

It IS interesting, yes, but unfortunately the MENCODER tends to cut my recordings down to 27.5 seconds when it converts them no matter how long they are, FFMPEG does not... Never figured out why.

Can this codec be used with ffmpeg?

Thanks for your replies

I am not familiar with ffmpeg as I had always gotten faster encoding times with mencoder for some reason and the build that I posted on the first post is a turbo release as it gave me 20+ more frames in encoding speed over any previous version I used

It is weird that mencoder stops the encoding on your files and the only thing that comes to mind would be some bad frames in your videos but I think that would only happen to digital ts files ripped from the broadcast streams as I cannot see that happening on analogue capture cards.

if you are on digital how is your reception? any similarities to the files such as recordings made around the same time example evenings versus mornings?

@zed - thanks so much for your reply. I use your guide enhancer daily and I really enjoy the work you do to help us all in and around the site and wiki.

off-topic - I posted about 8 months ago looking for a way to bypass my mpeg encoder so that I can tryout direct software encoders like divx etc... and now I feel the FMP4 encoding is faster than real time and would be a great hobby to try to get the encodings straight from the anolgue to FMP$ bypassing the mpeg encoding process altogether.

most machines are fast enough to support real time encoding and it is fun to think of new things to pass the time

Has anyone viewed my Video encoding examples yet? I really think FMP4 is winner when it comes to speed and quality.

Compression video results
http://www.4shared.com/file/102625718/dd...sults.html

Filename-----------------------Resolution-------FPS_EST_Enc--------time2encode--------------Filesize
seaworld.mpg-------------------720x480-----------29.99fps-------------30.00s----------------16,625k
seaworld_xvid_720x480.avi------720X480-----------23.82fps-------------24.12s-----------------6,774k
seaworld_h264_720x480.avi-----720X480-----------16.99fps-----------1:10.40s-----------------7,471k
seaworld_fmp4_624x352.avi-----624X352---------*100.00fps--------------9.28s-----------------7,303k

[Reddwarf]

It does not happend to all my recordings, just now and then, but too often to live with. Since I have one Ati550Pro and one PVR150 i suspected that it was the old timespamp bug on the PVR150 that caused the recordings to be truncated, but it happend to some recordings from the ATI card as well. I spent much time trying to figure out why, but I gave up in the end. The signal is very good, about 58-60 dB. I couldn't find any pattern in the phenomena, so it remains an "unsolved mystery" to this day :confused:

[mvallevand]

I didn't bother looking because for me the only reason to compress is for permanent archive, normally I would use native and then delete, and FMP4 is not ubiquitous enough for long term retention. Everyone will have personal preferences but I think it is better to stick with standards like xvid or h264 with standard profiles.

Martin

[csy]

I used MeGUI for a while to convert to H.264 and it did an excellent job, but I eventually worked out it was easier and cheaper (timewise) to simply buy a 1TB hardrive and archive in native format.

[-stattik-]

Hi tvshowman,

I took a look at it but just got around to posting now.

You mention that your x264 transcodes interfere with PVRX2 and make it slow. Have you tried running the transcode as a low priority? You can also set comskip as belownormal so it will get priority over transcoding but it won't step on PVRX2. I do it all the time and I have no problems with PVRX2 getting slow. If your GPU supports it you could enable DXVA in PVRX2. That will offload playing of h264 files.

Like Martin mentioned above, if you are archiving your content, you might want to stick to a standard like h264 or xvid since they are also supported in hardware appliances like the PCH.

The content you provided was little fuzzy and had a little artifacting in the source video so it's hard to get a good idea of the quality of compression when the source is in that state.

I ran the mpg through a new script I'm writing and came up with two output files that I think look fairly good. One uses "average settings" and the other uses fast settings. Here's the link:

http://rapidshare.com/files/228474596/seaworld.zip.html

seaworld-default.mkv--x264--mp3--12fps
seaworld-fast.mkv-----x264--mp3--30fps

The processor was an AMD x2 4200 which is pretty old. Yours is most likely faster. I can post the encode parameters if needed. The videos are DVXA and PCH compatible.

[tvshowman]

thanks everyone for your replies. I am forced to compress everything as I seem to record way too much and watch far too less. I don't record to archive but rather so that I can watch at a later date. The problem with recording all the time is that the hard drive over time can fail. I am very please with the auto compress I am using with FMP4 and the encoding times of 60FPS means a 30 minute recording is encoded in 15 minutes. I tried to use the XVID and H264 scripts and found I could not even use the pvr much since it was encoding all the time.

TV quality to begin with is not the greatest and I have found the perfect balance that works best for my pvr and hard drive space. I have about 3 drives and 1.3tb of space but I always seem to have 18 gig of space free on any of them and still have to delete shows.

I am happy to compress a 5 gig movie down to 1.5 gigs so I can watch it at a later date. I usually delete the shows after I watch them or a season ends...

Cheers