As it says in the title, since the 1980s I've accumulated about 10TB of mixed videos, past rips, and home movies on my file server, as well as DVDs. Looking at them now, about 6 TB of them (~30 days actual play time) are large files at ridiculously high bitrates that I just don't need. A quick calculation suggests that if I recode all files with > 150MB and > 2500 kbps video, to 2000 kbps HEVC, and rip all DVDs I haven't yet ripped, at the same video bitrate, I'll save about 1.2TB of file space, which is quite a lot for me and worth it.
I use a mix of FreeBSD and Windows (including Windows Server), and because HEVC is sloowwwww, I have not one, but 3 decent machines I can split the files across and run 24/7, to cut the time right down. (One of these machines would be a background task on my mostly-idle FreeBSD server, the other 2 are unused machines and can run ffmpeg under FreeBSD or Windows). I've already ripped the physical DVDs to disk as .VOB's and .TS, in preparation. My tool of choice is ffmpeg x64 CLI.
I'd like it ideally done within about a week, if possible, or 2 at most. But at the monent I'm getting about 0.1x ~ 0.3x, even after moving from "slower" to "slow". Probably because there's no graphics cards or encoder hardware assist in any of these three machines.
So I'm posting to ask advice on 2 aspects:
1) What hardware might help to seriously accelerate HEVC encoding?
Ideally I'd like at least one option that will also work with ffmpeg on FreeBSD, because that wqay I can also do any future conversions on my file server from console. But mostly it's a one-off task, so cheapest is best, and if that means Windows for driver availability thats okay.
Also what I've read, suggests that hardware HEVC encoding acceleration can be a problem, because the options supported by cards are seriously limited compared to those supported by software. I might need 10 bit not 8 bit, or specific colour spaces - I don't know what settings I need. if the card doesn't support them, I don't know how big a deal it is. Insight and card advice needed badly :)
My available hardware platforms (all use motherboard VGA, none have builtin graphics):
2) Are there any specific settings or formats that I am strongly recommended to use, or that the card needs to support?
The files are a complete mishmash of existing formats and rips over the years, going back to the 1990s, off many devices, codecs, containers and sources, so I'd like to pick one format that will generally deliver well, wherever I go in future. I'm happy to use HEVC for all of it, regardless. No reason not to, easier to write the script. I've historically used mp4 for max portability but finding of course that most subtitle streams need converting to text_mov, which I know nothing about. Also there's some subtleties with HEVC options, which I don't know much about either.
My current command I'm trialling is: ffmpeg -i {FILENAME} -map 0 -c:a copy -c:s copy -c:v libx265 -preset slower -b:v 2000k {FILENAME}.RECODED.mp4
I've run 1 pass simply because 2 pass doubles the time, for uncertain quality benefits. I also now know I need to add -c:s text_mov for mp4. But I've seen stuff about using 10 bit not 8 for HEVC, and stuff about using -pix_fmt yuv422p10 or similar for compatibility and best colour spaces, and I don't have a clue if those are significant benefits, or what exactly is best. In fact I don't even know if mp4 is still the most widely supported format (I'd like to never have to recode again, so I want widest support, hence unsure if I want to use mkv for example). So my settings questions are:
SUMMARY
As you can see, its a bit of a messy question, because I need to consider both hardware acceleration cards, and software args, not just one or the other. Encoding really isn't my skill area, so any advice or information from the many gurus and enthusiasts here, would be so appreciated! :)
I use a mix of FreeBSD and Windows (including Windows Server), and because HEVC is sloowwwww, I have not one, but 3 decent machines I can split the files across and run 24/7, to cut the time right down. (One of these machines would be a background task on my mostly-idle FreeBSD server, the other 2 are unused machines and can run ffmpeg under FreeBSD or Windows). I've already ripped the physical DVDs to disk as .VOB's and .TS, in preparation. My tool of choice is ffmpeg x64 CLI.
I'd like it ideally done within about a week, if possible, or 2 at most. But at the monent I'm getting about 0.1x ~ 0.3x, even after moving from "slower" to "slow". Probably because there's no graphics cards or encoder hardware assist in any of these three machines.
So I'm posting to ask advice on 2 aspects:
1) What hardware might help to seriously accelerate HEVC encoding?
Ideally I'd like at least one option that will also work with ffmpeg on FreeBSD, because that wqay I can also do any future conversions on my file server from console. But mostly it's a one-off task, so cheapest is best, and if that means Windows for driver availability thats okay.
Also what I've read, suggests that hardware HEVC encoding acceleration can be a problem, because the options supported by cards are seriously limited compared to those supported by software. I might need 10 bit not 8 bit, or specific colour spaces - I don't know what settings I need. if the card doesn't support them, I don't know how big a deal it is. Insight and card advice needed badly :)
My available hardware platforms (all use motherboard VGA, none have builtin graphics):
- File server = Xeon E5-1620 v4 (4 core+HT, 3.5GHz, haswell generation), 256 GB 2400 MHz RAM. (FreeBSD only)
- Spare #1 = Xeon E5-1680 v4 (8 core+HT, 3.4 GHz, Haswell generation), 128 GB 2400 MHz RAM. (FreeBSD or Windows)
- Spare #2 = Intel i5-2300 (4 core, 2.8 GHz + turbo, Sandy Bridge generation), 8 GB 1600 MHz RAM but can add more or faster. (FreeBSD or Windows)
2) Are there any specific settings or formats that I am strongly recommended to use, or that the card needs to support?
The files are a complete mishmash of existing formats and rips over the years, going back to the 1990s, off many devices, codecs, containers and sources, so I'd like to pick one format that will generally deliver well, wherever I go in future. I'm happy to use HEVC for all of it, regardless. No reason not to, easier to write the script. I've historically used mp4 for max portability but finding of course that most subtitle streams need converting to text_mov, which I know nothing about. Also there's some subtleties with HEVC options, which I don't know much about either.
My current command I'm trialling is: ffmpeg -i {FILENAME} -map 0 -c:a copy -c:s copy -c:v libx265 -preset slower -b:v 2000k {FILENAME}.RECODED.mp4
I've run 1 pass simply because 2 pass doubles the time, for uncertain quality benefits. I also now know I need to add -c:s text_mov for mp4. But I've seen stuff about using 10 bit not 8 for HEVC, and stuff about using -pix_fmt yuv422p10 or similar for compatibility and best colour spaces, and I don't have a clue if those are significant benefits, or what exactly is best. In fact I don't even know if mp4 is still the most widely supported format (I'd like to never have to recode again, so I want widest support, hence unsure if I want to use mkv for example). So my settings questions are:
- In 2019 is mp4 still "acceptable" and should I still use it? If not, what is more widely fully supported, and what issues does mp4 have that I could avoid?
- My subtitles contain formatting and unicode/non-English characters. What args/containers/formats to use?
- 8 bit or 10 bit? Add -pix_fmt or not, and which one? Any other args that are recommended?
SUMMARY
As you can see, its a bit of a messy question, because I need to consider both hardware acceleration cards, and software args, not just one or the other. Encoding really isn't my skill area, so any advice or information from the many gurus and enthusiasts here, would be so appreciated! :)