Disconnected base mesh handling in NRR

General
1

Hi,

At the moment I’m using a Daz Genesis model as my base.
Version 1.3.3 works nicely now and gives me an exception warning when required.
The ONLY non-connected geometry I require are the eye balls (lashes, gums and mouth etc won’t be seen/used in my final obj)

Am I right in thinking that it IS NOT just a case of attaching them (using attach command in 3ds MAx for example) to the base mesh?
The problem is that they are physically isolated?

I see it is suggested to use wrap.subset.

From what I understand from the reference file, isn’t wrap.subset just creating a edited obj (minus disconnected mesh)? Something I can do externally anyway?

What do I do to handle the unwrapped remaining geometry - in my case the eyes?

Perhaps I am missing something?

Thank you in advance.

2

Hi,
You have a model from Daz and it has eyes included in the obj file and you want to wrap the body/face minus the eyes?
Can’t you extract out the eyes before hand?

If not, then take a look at the docs:
http://www.russian3dscanner.com/docs/WrapX/WhatsNew132.html

List out the polygroups so you know which ones to work on.

daz = wrap.Geom("Daz_model.obj")
print daz.polygroups
['Eyes', 'Body']

bodyPolys= daz.selectPolygonsByPolygroup('Body')

bodyOnly, verts = wrap.subset(daz, bodyPolys)
wrapped = wrap.nonRigidRegistration(bodyOnly, wrapToMesh, bodyPoints, wrapToPoints)

# Apply positions back to the source
wrap.applySubset(daz , wrapped, verts )

Hope that helps
-Sean

3

Hi Sean

Many thanks for the reply!

Yes, it is no problem to strip out the eyes externally.

So, as I understand it there is no requirement to use the method you kindly listed in the script? (extracting the eye poly group in WrapX)

However, I am wondering how to re-combine the eyes subsequently with the new Wrapped morphed bod yother than manually adjusting them into position?

Unless I’m missing something, ideally I would need to run the NRR somehow for each polygroup. I can’t see how this could be done using selectPoints though.

My scans (of which there are many!) are of children in all clothing sizes from 3 to 12 year olds. So there is a lot of variance.

Your final line in the script
“Apply positions back to the source”

Does that re-orientate the unwrapped poly group (eyes) somehow?

As I say, I’m rather confused as to a suitable method.

4

So wrapX can only work on 1 mesh at a time. If you have sub meshes (polyGroups) then they have to be treated separately.

As for the applySubset command above, it essentially is taking the results of the wrap which is an extracted version of the original mesh minus the eyes and is snapping the vert positions back to the original mesh leaving the point order the same leaving the eyes untouched.

As far as dealing with the eyes by themselves is tricky. You could use the rigidRegistration on just the extracted eyes so they will be in the place of where you need them to be from the target mesh.

5

Ah!

YES!

Good point about rigidRegistration, of course!

That’s got to be worth a go.

Thanks, I’ll let you know how it goes!

6

Grrr.

Nope. Not got anything usable with this software yet after messing with it for over a week now!

Rigid registration needs a minimum of 3 points where as I can only provide two for the eyes. Result - eyes twisted or rotated. Easier to do by hand.

Free polygons selections still get deformed so the result is a hideous unusable mess.

Thank you for your help Sean, least there is one person willing to help on this forum!

Just fed up now.

Hopefully we can get our money back.

7

Hello Adrian,

I’m very sorry, I was working like crazy on Wrap3.0 last week so I didn’t find time to help you. I have no excuse.
A refund has just been made.

As for the problem - yes, WrapX can’t really help with eyes alignment. In most cases the eyes will require manual refinement in third-party software. To make this easier I can only give few ideas.

When it comes to set of facial expressions of the same person I would suggest to add some markers on the top of the head (so that they does not move relative to a skull while the actor is performing) and pre-align all the scans according to this markers. The goal is to align them so that the skull does not move between frames. Stabilizing the skull should make eyeballs quite steady.

In case of different person scans, as snolan suggested, one can extract a subset with head geometry only and wrap it. Having two subsets (original and wrapped) one can apply lattice deformer with large radius parameter(http://www.russian3dscanner.com/docs/WrapX/wrapFunctions.html#wrap.latticeDeform) to deform the eyeballs, teeth, eyelashes etc. However depending on radius the eyeballs may become not precisely spherical.

8

Sorry you’re having such a hard time. I did take a bit time to get comfortable with the interface and api. Once I did I was able to batch process over 130 shapes from Maya to WrapX back to Maya. Took about an hour to process but I’m only doing it on one connected mesh at a time. Since then I’ve found a method to do what I need in Maya alone so I don’t need WrapX now (sorry guys) but it has been a great learning process and understanding the tool set.

-Sean

9

Hey Sean,

If it is possible, could you please share your approach? If there is a better work-around to wrapping I would be glad to implement it in next version.

10

Hi Andrew!

FIRSTLY - thank you for reply & refund.

I must say I wasn’t expecting that at this stage!

I must apologise for my negative comments as I think I made them when I’d been having a frustrating day.

I sent that when I’d just discovered that MakeHuman also has disconnected geometry. I had hoped that if a Daz base wasn’t an ideal option I could use something else.

I have now used Wrap X for a first set of 10 scans of very varied body shapes. I seem to have much more success with fewer points (3 actually) than the original set of around 50. I select ALL the head, hands and feet as free polygons as I want those as unaffected as possible.

I’m not sure I fully understand the latticeDeform. So can target_to_deform geometry include disconnected geometry?
This might work. I know Ralph(Vintorix) mentioned surface deformers. I use 3DMax so it may be possible. Manually positioning eyes is quite easy except it is not precise which means my generated morphs have moving eyeballs!

Sean - thank you again for your help! I’m impressed at your workflow rate! My script wasn’t fully automated but thye wrapping took around 10 mins per mesh, 130 in an hour, that is FAST! Any further clues of your solution would be most welcome!

Thanks for the help - I’m certainly not finished with WrapX quite yet.

cheers

Adrian

11

Hi Adrian, that’s ok. I have created a sample script that demonstrates how to handle teeth, eyelashes, eyes etc. using lattice deformer. The archive includes pre-selected control points and a basemesh model.

http://www.russian3dscanner.com/Scripts/WrapLatticeExample.zip

To make it clearer you can run it step-by-step. Select a portion of the code and run it with SHIFT+F5

http://www.russian3dscanner.com/images/Lattice.gif

http://www.russian3dscanner.com/images/InnerParts.jpg

import wrap
scan = wrap.Geom(wrap.demoModelsPath + '/AlexNeutral_3DHumanity.obj',scaleFactor = 100)
scan.wireframe = False

basemesh = wrap.Geom('ManWithDetailsPolygroups.obj',scaleFactor = 10)

# Rigid Alignment
(pointsScan, pointsBasemesh) = wrap.selectPoints(scan,basemesh,wrap.loadPoints('pScan.txt'),wrap.loadPoints('pBasemesh.txt'))
rigidTransformation = wrap.rigidAlignment(basemesh,pointsBasemesh,scan,pointsScan,matchScale = True)
basemesh.transform(rigidTransformation)
basemesh.fitToView()

# Take a subset of good connected manifold head geometry
headPolygons = basemesh.selectPolygonsByPolygroup('Head') + basemesh.selectPolygonsByPolygroup('Sockets')
head, indices = wrap.subset(basemesh,headPolygons)
basemesh.hide()

# Non-rigid registration
(controlPointsScan,controlPointsHead) = wrap.selectPoints(scan,head,wrap.loadPoints('cpScan.txt'),wrap.loadPoints('cpHead.txt'))
freePolygons = wrap.selectPolygons(head,head.selectPolygonsByPolygroup('Sockets'))
initialRadiusMultiplier = 2.0
smoothnessFinal = 0.2
headWrapped = wrap.nonRigidRegistration(head,scan,controlPointsHead,controlPointsScan,freePolygons,minNodes = 50,initialRadiusMultiplier = initialRadiusMultiplier,smoothnessFinal = smoothnessFinal,maxIterations = 20)
head.hide()

# Bring changes of head geometry back to basemesh
wrap.applySubset(basemesh,headWrapped,indices)
headWrapped.hide()
basemesh.show()

# Extract hair, eyelashes, eyes, teeth into subset
otherPolygons = basemesh.selectPolygonsByPolygroup('Other')
other, indices = wrap.subset(basemesh,otherPolygons)
basemesh.hide()
scan.hide()

# Lattice deformer
nNeighbours = 150
otherDeformed = wrap.latticeDeformAdaptive(other,head,headWrapped,nNeighbours)
otherDeformed.color = (128,0,255)
other.hide()
headWrapped.show()

# Bring changes to original geometry
wrap.applySubset(basemesh,otherDeformed,indices)
otherDeformed.hide()
headWrapped.hide()
basemesh.show()
12

Hello Andrew

Aha! This looks interesting!

Thank you very much for taking the time to make this.

An example helps so much while I’m still learning things with WrapX.

I’ll try it later today certainly

Thanks so much for the help!

13

So let me give you the full scope of what I was trying to achieve. I have a high res head with a few LOD heads and needed to transfer the blendshape targets from high head to each respective LOD head. This was not an easy process which required a quite of bit of per-processing for performance optimizations in WrapX.

Mapping
Mapping between the meshes (High res → LOD1 and so on). I used the (X,Y,Z) vert positions on the LOD to get a bary coord on the high res. This creates a big data set but gets truncated later in the process.

Target Extraction
Each of the high res blendshape targets get extracted from Maya as objs. This is where blendshape data is acquired in Maya. I find all the points that are affected by the blendshape delta’s on the high res. These points get projected onto the LOD and converted to face ID’s. Now some of these faces are disconnected which causes an issue in WrapX. So I create a temp UV map on each island of faces to get UV shells. These shells get converted to face IDs and then I pass in each of these islands into WrapX so it’s a continued mesh. This is where the points get truncated. I do a test in WrapX for each of the source and target points. If the points on the subset mesh (the island) falls within the large data set then use this point. I iterate over each island in WrapX doing a wrap and applying the results back to the source mesh (applySubset). Since I’m using a smaller section of the mesh, this process is very fast. Under 1 min per shape. Each target is getting kicked off from Maya using a subprocess. Once the subprocess is over and all targets have been created they are imported and the blendshape deformer is created. Now at one point, I was using mult-threading so I was able to kick off multiple instances of wrapX which sped up the process even more.

My process broke down once I got to the teeth. The models were modeled in a way that wouldn’t make this process work so I ended up having to come up with different solution.

Here is a bit of the code I used for mult-processing:

import multiprocessing
from subprocess import Popen, PIPE, list2cmdline
import time

def executeInParallel(commands):
    """
    Exec commands in parallel in multiple process
    (as much as we have CPU)
    """
    # empty list
    if not commands:
        return

    def done(p):
        return p.poll() is not None

    def success(p):
        return p.returncode == 0

    def fail():
        sys.exit(1)

    # Get number of physical cpu's (half of them)
    max_task = int(multiprocessing.cpu_count() / 4)
    processes = []
    while True:
        while commands and len(processes) < max_task:
            task = commands.pop()
            print list2cmdline(task)
            processes.append(Popen(task))

        for p in processes:
            if done(p):
                if success(p):
                    processes.remove(p)
                else:
                    fail()

        if not processes and not commands:
            break
        else:
            time.sleep(0.05)

# Gather up a bunch of commands
commands = []
cmd = ['WrapX.exe', script, lodObj, targetObj, target, lodPnts, targetPnts, lodOutputDir, blendShapeDataList[t], "False"]
commands.append(cmd)

# Send them off to be processed
executeInParallel(commands)

This is a small example but hopefully shows the capabilities. One thing to note that is not in the docs when using the commandline option, it works best if you add the WrapX.exe to your environment path. That way you can call the commandline without having to give an absolute path.

-Sean

14

Hey Sean,

Thank you so much for sharing this! Mind blowing pipeline :o

15

Wow - looks a very clever solution.

My task should be a doddle compared to this.

Thank you so much for sharing Sean!