Doubt that it is related to a Ryzen specifc problem, since I use a 1800x.
Looking at the debug output:
E:\Hybrid\Temp\encodingTempSynthSkript_17_02_17_5010.avs: Cannot allocate memory
the avisynth script itself looks okay:
SetMTMode(5,16) # change MT mode
# loading source: E:\Hybrid\YUN00001.mp4
# input luminance scale pc
# current resolution: 3840x2160
SetMTMode(2) # change MT mode
but my guess is the 'Cores=32' is causing avisynth trying to allocate to much memory. (keep in mind that Avisynth is 32bit)
=> either use Vapoursynth, which is probably a better choice for your Ryzen, since:
a. it scales better in regard of the core count
b. it's only limited by the RAM on your system
or stick with Avisynth and use another core count ('Cores'; default is 0 which translates to 2xlogical core count).