Incredible long shader compilation time (HLSL)
category: general [glöplog]
Motivated by the current ray-marching trend I decided to try do some myself. So far I just have some very basic sphere tracing of a cube, sphere and a plane and a routine for calculating the normals.
Now, this all runs pretty smooth on the fast ATI HD 4870. The ironic thing though is that the time taken by the shader compilation is unbelievable. It currently takes around 90 seconds. When trying to implement a simple shadow algo it risez to 5 minutes just to tell me there is an error in the code. It also consumed a lot of memory during compilation. The simple shader takes around ~220 MB, the more complex one with the shadow experiment took ~600 MB.
I have the latest drivers installed and I am using the latest DX SDK (august 2009.)
I compile the shader like this:
Sorry for the ugly layout but the copy/paste fucked it up.
I tried with flags = 0 as well (I normally don't use the flags.)
To make sure the main loop is not unrolled I pass the threshold (length) as a parameter.
I noticed the excellent 'Rudebox' intro loads quite fast.
What do you guys think? Should I switch to OpenGL? Any ideas?
If you think the shader code is relevant I will post it as well.
Thanks you for any help!
Now, this all runs pretty smooth on the fast ATI HD 4870. The ironic thing though is that the time taken by the shader compilation is unbelievable. It currently takes around 90 seconds. When trying to implement a simple shadow algo it risez to 5 minutes just to tell me there is an error in the code. It also consumed a lot of memory during compilation. The simple shader takes around ~220 MB, the more complex one with the shadow experiment took ~600 MB.
I have the latest drivers installed and I am using the latest DX SDK (august 2009.)
I compile the shader like this:
Code:
HRESULT result =
D3DXCompileShaderFromFile(
sourceFile, //filepath
NULL, //macro's
NULL, //includes
"main", //main function
"ps_3_0", // shader profile
D3DXSHADER_SKIPOPTIMIZATION | D3DXSHADER_SKIPVALIDATION, //flags
&code, // compiled operations
&errorMessages,//errors
NULL); //constants
Sorry for the ugly layout but the copy/paste fucked it up.
I tried with flags = 0 as well (I normally don't use the flags.)
To make sure the main loop is not unrolled I pass the threshold (length) as a parameter.
I noticed the excellent 'Rudebox' intro loads quite fast.
What do you guys think? Should I switch to OpenGL? Any ideas?
If you think the shader code is relevant I will post it as well.
Thanks you for any help!
opengl seemed even worse when it came to compilation time from what i saw...
since you didnt specify what you are making: is there a practical reason for not using precompiled shaders? (=4k?)
since you didnt specify what you are making: is there a practical reason for not using precompiled shaders? (=4k?)
Try to add the D3DXSHADER_PREFER_FLOW_CONTROL-flag.
actually i never had such shader compilation times under opengl... there must be something (very) wrong about what you're doing..
Gargaj:
I don't care about the compilation time for a final 4k/64k :) It's a problem when coding.
I am not experienced with shadow algos so for me it requires a lot of adjusting code and hitting F5. Waiting 5 minutes every time doesn't make it a lot of fun :|
Actually I gave up some weeks ago but decided to give it another spin and also to open this thread :)
I don't care about the compilation time for a final 4k/64k :) It's a problem when coding.
I am not experienced with shadow algos so for me it requires a lot of adjusting code and hitting F5. Waiting 5 minutes every time doesn't make it a lot of fun :|
Actually I gave up some weeks ago but decided to give it another spin and also to open this thread :)
Also sorry for my crappy English. I don't know what's up with that..
Kusma:
Tried that but didn't help. Thanks though. It seems the flags are ignored completely.
I will put the shader file on my server. Let me clean it up a bit first :)
Tried that but didn't help. Thanks though. It seems the flags are ignored completely.
I will put the shader file on my server. Let me clean it up a bit first :)
Okay,
Shader code can be found here: http://www.lelyskate.nl/code/example.cpp.
The output of the shader is: http://www.lelyskate.nl/code/example.jpg.
The code is not optimized and probably contains some very stupid stuff. Also, most of it comes directly from the several pouet topics about distance fields.
Shader code can be found here: http://www.lelyskate.nl/code/example.cpp.
The output of the shader is: http://www.lelyskate.nl/code/example.jpg.
The code is not optimized and probably contains some very stupid stuff. Also, most of it comes directly from the several pouet topics about distance fields.
try turning the "while (length(...))" statement into "[loop] while (length(...))" and see whether that helps.
ryg:
Done, no difference :(
Done, no difference :(
I don't have a very fast CPU btw. It's a Intel Core 2 Duo E6600. Not horribly slow either though.
Compiles within 1 second here. I've got an i7, but it shouldn't be THAT much faster ;)
Code:
float3 rayDirection = normalize(float3(input.TexCoords.x * 1.7777 7777 7777 77777f, input.TexCoords.y, 1));-remove the "f"s as they are not used in HLSL anyway.
-remove one "7" as there are 17 after the dot.
dunno if that helps anything, but who knows ;)
rest looks good by a fast look-over to me !
i do it as ryg suggests aswell btw:
do{
stuff
} while ( );
Kusma:
Thank you very much for testing. This means there is something seriously wrong on my machine...
c0c00n:
Thank you for your suggestions. I know the "f"s are not necessary anymore but it is somewhat burned into my soul ;)
The rayDirection doesn't seem to be the bottleneck, both during compilation and during running.
Thank you very much for testing. This means there is something seriously wrong on my machine...
c0c00n:
Thank you for your suggestions. I know the "f"s are not necessary anymore but it is somewhat burned into my soul ;)
The rayDirection doesn't seem to be the bottleneck, both during compilation and during running.
weird, i tested with fxc.exe, and if i don't specify a target profile it indeed takes ages, but if i set /Tps_3_0, it compiles under a second.
Gargaj:
Interesting, let me try that as well.
Interesting, let me try that as well.
"fxc /EMain /Tps_3_0 example.cpp"
Still took incredibly long :(
Still took incredibly long :(
I have updated the example.cpp. It now has a do .. while loop because it indeed makes much more sense.
Now: this has increased the framerate but compilation time is still eons :(
Now: this has increased the framerate but compilation time is still eons :(
which version of fxc are you using? i use "Microsoft (R) D3D10 Shader Compiler 9.24.949.2307"
Quote:
C:\>fxc /EMain /Tps_3_0 example.cpp
Microsoft (R) Direct3D Shader Compiler 9.27.952.3012
I'm on XP so no D3D10 for me.
I'm on XP but I use the August 2008 SDK
I am sorry for not mentioning this before: working with D3D 9.
All right, thanks to the great help from Gargaj I can now compile it in less than a second.
Somehow this flag: D3DXSHADER_USE_LEGACY_D3DX9_31_DLL helps.
It doesn't allow me to return from an if statement though anymore. But that is solvable with a simple workaround so no problem.
Thank you all :)
Somehow this flag: D3DXSHADER_USE_LEGACY_D3DX9_31_DLL helps.
It doesn't allow me to return from an if statement though anymore. But that is solvable with a simple workaround so no problem.
Thank you all :)
time to file a bug report with ms.
since this is pretty relevant to topic: has anyone noticed that on Vista (at least 64bit) GLSL shaders (all of them) compile noticeably too long when running program under debug mode on vs2005/08? regular executing of opengl program and no duration issues occur...
