After Google surrendered to Sceneform, this developer built a better way to make AR faces to life

When SceneForm and its Armas were stopped in 2020, this made it more difficult android applications. Inspired by the original scene applications programming and based on Arcore samples, I have created AugmentedFaceFragment and AugmentedFaceListener A interface to be able to create Arcore UPderFaces easily on Android.

I will release 3 different upderses features using AugmentedFaceFragment and AugmentedFaceListener Interface.

What will you build?

Part 1 of the series will cover the basics AugmentedFaceFragment and AugmentedFaceListener As well as an overview of all the chapters of the assistant. At the end of this article, you will build a simple offer by writing a few lines of code.

Project preparation

Download the Starter project

Reproduction of the warehouse:

git clone https://github.com/droid-girl/arfaces_labs.git

What is our starting point?

Our starting point is a modified version of the Arcore SDK sample for reinforced faces. The code has been modified in a way that we can now add different textures and objects to the face object.

Project structure

• Helpers – The original Java files of the symbol
• Rendering – The original Java files of a symbol formula dealing with the presentation of AR and background objects

Additional files for the warehouse:

• AugmentedFaceFragment.kt – Display and tracking handles, enhanced faces
• AugmentedFaceListener.kt – The handles are adding and updated the reinforced faces
• AugmentedFaceNode.kt He deals with the width of the face, including texture and three -dimensional models associated with it
• AugmentedFaceRenderer.kt Provide the texture of his face
• FaceRegion.kt – It offers a three -dimensional model in FACELANDMARK specific

Preparing the main activity

class MainActivity : AppCompatActivity(), AugmentedFaceListener {

    private lateinit var binding: ActivityMainBinding

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        binding.faceView.getFragment().setAugmentedFaceListener(this)
        setContentView(binding.root)
    }

    override fun onFaceAdded(face: AugmentedFaceNode) {}

    override fun onFaceUpdate(face: AugmentedFaceNode) {}

}

in activity_main.xml Add AugmentedFaceFragment As a major view, and we can be able to receive events from this fragment, we need to determine and appoint AugmentedFaceListener To our part.

onFaceAdded The method will be called when Arcore discovers a new face onFaceUpdate It will be called in each frame update.

Let’s add the face tissue as the next step.

Add the face tissue

We will use arcore assets for the first face mask. You can find assets/models Volume in your project. Let’s add freckles.png As a facial texture:

in MainActivity.kt Adjust onFaceAdded The method as follows:

override fun onFaceAdded(face: AugmentedFaceNode) {
   face.setFaceMeshTexture("models/freckles.png")
}

Add 3D models

Introduction to UpertedFacenode

The category is used with the face network and the middle medium to determine the areas of the face. These areas are:

• Left_ForeHead
• The right front (right – Fahid)
• Advice for the node (nose_tip)

This project includes AugmentedFaceNode season. Inspired by the scene, it is a knot that will provide visual effects on a discovered face using Arcore. AugmentedFaceNode It defines the same areas as the face AugmentedFace In the accompanying object:

companion object {
   enum class FaceLandmark {
       FOREHEAD_RIGHT,
       FOREHEAD_LEFT,
       NOSE_TIP
   }
}

Later on this tutorial, we will extend FaceLandmark enum class And add our areas.

AugmentedFaceNode Includes facial signs HashMapWhich connects facelandmark with a three -dimensional model.

The source code can be found for this project here.