For my senior thesis at RCAD I made a 3D coloring book. This involved making my own pipeline coordinating between a master material, meshes, tools, and blueprints for each model in the game.
All art in the game was made by me except the Unreal mannequin, and non-linear blending was handled by Mixbox.
Create mesh and materials
One UV channel has to be designated for painting, meaning no overlapping UVs. If a mesh needed overlapping UVs for another texture, Unreal’s Generate Lightmap UVs option works perfectly for setting the paintable version automatically.
Blueprint has a variable for which UV channel is for painting.
Create empties/locators for the mesh
Naming convention is SOCKET.{section}.###
Unreal converts . into _ and removes the SOCKET_ prefix automatically.
Blueprint function creates three maps based on {section} in alphabetical order. First one is Section Data, second is socket locations, and third is socket colors.
Vertex Color/Texture for sections
Color based on alphabetical order of sections. This way the vertex color’s number doesn’t have to be manually typed into Section Data.
Click button to generate Section Data
The surface area of each section is input manually. I used a plugin in Blender to measure the surface area of selected polygons.
If a mesh has more than one material the MID index array needs to be manually set.
If a material is used for more than one section, SectionMask needs to be manually set.
Blueprint accessible, so the game could tell which section was selected, if the current target matched, and communicate with material.
Material accessible, so the paint could visibly be masked out.
Avoid splitting the mesh into multiple or use multiple materials to define
I originally decided to use a texture because:
It could be detailed - for example, foliage with flowers, leaves, and branches on the same atlas.
I planned to use grayscale, capable of infinite sections (but only able to paint on one at a time), and pack it with other textures.
However, this method meant going into a drawing program to create the section mask texture, then packing it with other textures. With roughness and ao being on the same image, this quickly meant everything needed unique materials and I needed to find a better way.
I switched to vertex data.
Red (0), green (1), blue (2), white (3), and black (4) are used for a total of 5 separate sections. As sections need to be mesh-specific and vertex data can be accessed from the material, this worked well.
Blueprints can detect which section is selected by enabling a MCP to make vertex color the base color and reading a single pixel from the player’s cursor. It then reads the Section Data variable of an actor which coordinates with materials to show areas being painted.
Additionally, this method keeps the option to use a texture instead of vertex data if the extra detail is really needed.
My first thought was to get the color directly from the texture, however the blueprint nodes that “Read Render Target” are far too slow to be used.
Instead of using the texture, I decided to make an approximation of painted colors stored in the blueprint.
My first test tried to take advantage of line trace’s Find Collision UV. By creating an array of coordinates and colors based on where uv islands were, I could store approximate colors.
This had a few problems:
With bigger brush sizes, I could either choose to let it “paint” less than it actually did, or leak onto other places it didn't paint in 3D but was close to in UV space.
I wasn’t actually defining where uv islands were- I was using a very low resolution version of the mesh’s texture, using the Read Render Target node, and creating array items based on which pixels were not black. This had similar issues with my first go with section masks.
I moved the array of coordinates into 3d space.
The biggest deal was making a tool to dictate where the coordinates were. I decided to place them myself, as the point of approximating this meant there weren’t many to place anyways.
While Unreal has a 3d widget toggle on public location variables, I found it more convenient to place the points on the actor in Maya/Blender. Unreal’s mesh importer accepts locators/empties with the prefix “SOCKET_” as sockets, and I named each socket according to section.
I wrote a function to store the socket and section information, and afterwards the sockets can be safely removed from the mesh.
Outcome
I created a good enough, fast estimate of how much the player had painted the level.
