Writing, compiling and linking a plain shader for Maya Mental Ray.




Intro


This tutorial is only about the basic steps you need to follow in order to write and link a simple MentalRay shader into Maya.
This includes a basic explanation of the .mi declaration file, a basic source code, how you compile, how you link and load the shader into Maya and how you can batchrender with it. A sample plain shader is also included with all the explanations you will need.
Throughout the tutorial, two main cases are described. One for Windows users and one for Linux who don't have access to the root directory where Maya is installed. I guess the process would be quite similar for Windows users who don't have access to the installation directory as well.
I assume you have a basic understanding of programming and use of Maya. It would also be good to have access to MentalRay's help file, which comes with Maya 6.0 help.

Note - I am referring to paths where Maya is installed or where the user's Maya directory is located. Those are the specific paths for my case. In your case, they may be different so you will have to adjust them.




Get Started


*  Create a directory somewhere convenient for you. Let's call it mentalShaders. There you will store all the source files and everything else you will need.
*  Copy shader.h from
C:\Program Files\Maya6.0\devkit\mentalray\include\
/usr/aw/Maya6.0/devkit/mentalray/include/
*  Copy shader.lib from
C:\Program Files\Maya6.0\devkit\mentalray\lib\nt

Note: you can avoid coping the files, but then you will have to tell to the compile to go and find them.




Shader Declaration


*  Create a text file and name it plain.mi. This is the file where the shader node is declared. You can declare more than one shaders in each file.

1 . # A plain shader
2 . declare shader
3 . color "plain"
4 . (
5 .   color "color",
6 .   scalar "brightness", #: default 1.0 min 0.0 softmax 1.0
7 . )
8 . version 1
9 . apply material
10. end declare


Lines Explanation

1      Use a single # to comment out the line.
2,10 Every shader declaration starts and ends like this.
3      The name and the output of the shader
5,6    All the input attributes of the shader. Need to have the same parameters with the shader structure which we will make later on. For all the types available, refer to the help file. After #: you can specify the default value and the slider range for the attribute editor. Soft min and max mean that a greater or lower values can be typed it and the slider will be adjusted.
8      This is there for mental ray to check that this .mi file match the current version of your source code.
9      There you specify what type of shader it is. If apply is not used, the shader will be a simple texture. With apply material the shader can be assigned to objects, and the node will be under Materials in the HyperShade. For all the types available check the help file.



Shader Source


*  Create an another text file and name is plain.c. This is the source code of the shader and again you can include more than one shader in each file.

1 . #include "shader.h"
2 .
3 .
4 . struct plain
5 . {
6 .   miColor color;
7 .   miScalar brightness;
8 . };
9 .
10. DLLEXPORT int plain_version(void) {return 1;}
11.
12.
13.
14. DLLEXPORT miBoolean plain
15. (
16.   miColor *result,
17.   miState *state,
18.   struct plain *paras
19. )
20. {
21.   miColor *color = mi_eval_color(¶s->color);
22.   miScalar *brightness = mi_eval_scalar(¶s->brightness);
23.
24.
25.   result->r = color->r * (*brightness);
26.   result->g = color->g * (*brightness);
27.   result->b = color->b * (*brightness);
28.
29.   return miTRUE;
30. }


Lines Explanation

1      shader.h provides all the functionality you will need to write the shader.
4      You need to declare a data structure from each shader. This structure should have the attributes that the shader will have as inputs from Maya's Attribute Editor and must be the same with those that you declared in the .mi file. You can also put the structure declaration in its own .h file and include it in this source.
10     This is the version of your source code which must match the version in the .mi. The DLLEXPORT can be ignored when compiling under Linux.
14     The shader main function, which always takes those three parameters and returns a boolean whether the function call was successful or not. Again ignore the DLLEXPORT if it is Linux.
16     Reference for the output result of the shader.
17     Reference for the current state of the renderer. Out of this structure, you can get all the information you need for the point which is sampled and many more.
18     Reference to a copy of the structure you declared previously, which holds the values of the shader's attributes.
21     This is how you take the attribute values from the shader structure. There is a different function call for each type. Make this call when and only it is needed to save computation time.
25     That's the point where the shader actually does something and modifies the result colour of the sample which is being calculated.
29     The return statement.



Compiling



Windows

*  To compile and link the shader under Windows using Visual Studio, type those commands in a command promt inside your mentalShaders directory.

cl /c /MD /nologo plain.c
link /nologo /DLL /nodefaultlib:LIBC.LIB plain.obj shader.lib

Note: For other compilers or systems, refer to the mentalRay help file)

*  Make sure you have added the path of the Visual Studio compiler in your Windows enviroment variables. Then just move the .dll in C:\Program Files\Maya6.0\mentalray\lib and the .mi in C:\Program Files\Maya6.0\mentalray\include directory.
*  Now you need to edit the maya.rayrc which is located under C:\Program Files\maya6.0\mentalray\ by adding 2 lines somewhere in the middle:

link "{MAYABASE}/lib/plain.{DSO}"
mi "{MAYABASE}/include/plain.mi"


Hopefully the shader should be now ready to use in the HyperShade and render.



Linux

*  To compile and link the shader under Linux, the commands are:

gcc -c -O3 -fPIC -Bsymbolic plain.c
ld -export-dynamic -shared -o plain.so plain.o

Note: For other compilers or systems, refer to the mentalRay help file)

*  To load the shader into Maya, there are two approaches depending if you have permissions into the root directory /usr/aw/maya6.0/ of Maya or not.
*  Typically you need to add your files under /usr/aw/maya6.0/mentalray/ and edit the maya.rayrc file which is located in the same directory. The .so file goes to /lib/ and the .mi goes to /include/. You can add a command to your compile shell script to do the copy of the .so and .mi automaticly. There are also 2 lines you need to add in the maya.rayrc file, which would be something like:

link "{MAYABASE}/lib/plain.{DSO}"
mi "{MAYABASE}/include/plain.mi"


After that the shader will be in the hyperShade and ready to use.

*  But if you don't have permissions there, you need to follow some other steps. First make a copy of the maya.rayrc file under your local maya preferences directory ~/maya/6.0/prefs which is going to be the one that Maya will read. In this case the 2 lines will be something like:

link "/(full path from root)/mentalShaders/plain.{DSO}"
mi "/(full path from root)/mentalShaders/plain.mi"


*  Now to use the shader in Maya you need to manually load the .mi from the menus Windows/Rendering Editors/mental ray/shader manager. But this can get a bit annoying because you will have to load it every time you open Maya, and also you can't use the shader if you batch render from the command line. To prevent this you need to add a couple of Environment Variables in your local ~/maya/6.0/Maya.env file which will point to your own mentalShaders directory. Those would be something like:

MI_CUSTOM_SHADER_PATH = /(full path from root)/mentalshader
MAYA_MRFM_SHOW_CUSTOM_SHADERS = /(full path from root)/mentalShaders


This way, Maya will automatically pick up and load all the shaders that are stored under mentalShaders directory, either when loading the UI or a batch render.



Rendering


*  Finally in order to batch render using MentalRay for all cases from the command line, use a script called mayarender_with_mrM which is located under /usr/aw/maya6.0/bin or C:\Program Files\Maya6.0\bin. If you want, you can create an alias to your .bashrc which will directly run the script.
*  While developing the shader, you will need to update the .so or .dll file many times. Windows won't even let you update the .dll while Maya is running and even if Linux doesn't complain, it won't update the shader node it self. The most convinient way is to first unload the MentalRay plugIn from the menu windows/settings-preferences/plugIn manager, compile the shader, copy the library to the proper directory and reload the MentalRay plugIn. Spend a bit of time to write a shell script or a .bas file for dos, which will compile and move the files in one go. Mine for Linux looks like that:

gcc -c -O3 -fPIC -Bsymbolic $1.c
ld -export-dynamic -shared -o $1.so $1.o
rm $1.o




End


Download sample plain.mi
Download sample plain.c

Note: For any queries or correction, please email me.
Thanks to Sander Van Der Steen for his help.




www.ch3.gr