Search

Speech-driven 3D facial animation technology has been developed for years, but its practical application still lacks expectations. The main challenges lie in data limitations, lip alignment, and the naturalness of facial expressions. Although lip alignment has seen many related studies, existing methods struggle to synthesize natural and realistic expressions, resulting in a mechanical and stiff appearance of facial animations. Even with some research extracting emotional features from speech, the randomness of facial movements limits the effective expression of emotions. To address this issue, this paper proposes a method called CSTalk (Correlation Supervised) that models the correlations among different regions of facial movements and supervises the training of the generative model to generate realistic expressions that conform to human facial motion patterns. To generate more intricate animations, we employ a rich set of control parameters based on the metahuman character model and capture a dataset for five different emotions. We train a generative network using an autoencoder structure and input an emotion embedding vector to achieve the generation of user-control expressions. Experimental results demonstrate that our method outperforms existing state-of-the-art methods.

The talking head generation recently attracted considerable attention due to its widespread application prospects, especially for digital avatars and 3D animation design. Inspired by this practical demand, several works explored Neural Radiance Fields (NeRF) to synthesize the talking heads. However, these methods based on NeRF face two challenges: (1) Difficulty in generating style-controllable talking heads. (2) Displacement artifacts around the neck in rendered images. To overcome these two challenges, we propose a novel generative paradigm \textit{Embedded Representation Learning Network} (ERLNet) with two learning stages. First, the \textit{ audio-driven FLAME} (ADF) module is constructed to produce facial expression and head pose sequences synchronized with content audio and style video. Second, given the sequence deduced by the ADF, one novel \textit{dual-branch fusion NeRF} (DBF-NeRF) explores these contents to render the final images. Extensive empirical studies demonstrate that the collaboration of these two stages effectively facilitates our method to render a more realistic talking head than the existing algorithms.

This article is dedicated to discuss the sliding stability and the uniqueness property for the 2-dimensional minimal cone YXY in R4. This problem is motivated by the classification of singularities for Almgren minimal sets, a model for Plateau's problem in the setting of sets. Minimal cones are blow up limits of Almgren minimal sets, thus the list of all minimal cones gives all possible types of singularities that can occur for minimal sets. As proved in [16], when several 2-dimensional Almgren (resp. topological) minimal cones are Almgren (resp. topological) sliding stable, and Almgren (resp. topological) unique, the almost orthogonal union of them stays minimal. Hence if several minimal cones admit sliding stability and uniqueness properties, then we can use their almost orthogonal unions to generate new families of minimal cones. One then naturally ask which minimal cones admit these two properties. Among all the known 2-dimensional minimal cones, YXY is the only one whose stability and uniqueness properties were left unsolved. We give affirmative answers to this problem for the stability and uniqueness properties for YXY in this paper: we prove that the set YXY is both Almgren sliding stable, and Almgren unique; for the topological case, we prove its topological sliding stability and topological uniqueness for the coefficient group Z2. This result, along with the results in [16, 18, 17], allows us to use all the known 2-dimensional minimal cones to generate new 2-dimensional minimal cones by taking almost orthogonal unions., Comment: 56 pages

We prove that the k-medial axis of an arbitrary closed set in Rn is n-k+1-rectifiable (and hence of dimension at most n-k+1). This result gives a first stratification for medial axis of any closed set, which has been widely studied and used in pure and applied mathematics. This also answers a question proposed by Erdos[4], and leads to more further interesting investigations (see the end of the article)., Comment: 8 pages

In this paper, we prove that the product of a paired calibrated set and a set of codimension 1 calibrated by a coflat calibration with small singularity set is Almgren minimal. This is motivated by the attempt to classify all possible singularities for Almgren minimal sets--Plateau's problem in the setting of sets. In particular, a direct application of the above result leads to various types of new singularities for Almgren minimal sets, e.g. the product of any paired calibrated cone (such as the cone over the $d-2$ skeleton of the unit cube in $\R^d, d\ge 4$) with homogeneous area minimizing hypercones (such as the Simons cone)., Comment: 36 pages

In this article we prove that for a large class of 2-dimensional minimal cones (including almost all 2-dimensional minimal cones that we know), the almost orthogonal union of any two of them is still a minimal cone. Comparing to existing results for minimality of almost orthogonal union of planes \cite{2p,2ptopo}, here we are dealing with unions of cones with non isolated singularities, which results in a series of essential difficulties, and new ideas are required. The proof in this article can be generalized to other types of minimalities, e.g. topological minimality, Reifenberg minimality, etc.., Comment: 78 pages

In this article we treat two closely related problems: 1) the upper semi continuity property for Almgren minimal sets in regions with regular boundary, which guanrantees that the uniqueness property is well defined; and 2) the Almgren (resp. topological) uniqueness property for all the 2-dimensional Almgren (resp. topological) minimal cones in $\mathbb{R}^3$. As proved in \cite{2T}, when several 2-dimensional Almgren (resp. topological) minimal cones are measure and Almgren (resp. topological) sliding stable, and Almgren (resp. topological) unique, the almost orthogonal union of them stays minimal. As consequence, the results of this article, together with the measure and sliding stability properties proved in \tb{\cite{stablePYT} and \cite{stableYXY}}, permit us to use all known 2-dimensional minimal cones in $\mathbb{R}^n$ to generate new families of minimal cones by taking their almost orthogonal unions. The upper semi continuity property is also helpful in various circumstances: when we have to carry on arguments using Hausdorff limits and some properties do not pass to the limit, the upper semi continuity can serve as a link. As an example, it plays a very important role throughout \cite{2T}., Comment: 35 pages. arXiv admin note: text overlap with arXiv:1808.09687, arXiv:1808.09691

In this article we prove the measure stability for all 2-dimensional Almgren minimal cones in $\mathbb{R}^n$, and the Almgren (resp. topological) sliding stability for the 2-dimensional Almgren (resp. topological) minimal cones in $\mathbb{R}^3$. As proved in \cite{2T}, when several 2-dimensional Almgren (resp. topological) minimal cones are measure and Almgren (resp. topological) sliding stable, and Almgren (resp. topological) unique, the almost orthogonal union of them stays minimal. As consequence, the results of this article, together with the uniqueness properties proved in \cite{uniquePYT}, permit us to use all 2-dimensional minimal cones in $\mathbb{R}^3$ to generate new families of minimal cones by taking their almost orthogonal unions., Comment: 39 pages

We prove that, without any assumption on lower density bound or codimension, any 1-dimensional stationary varifold on any Riemannian manifold admits unique tangent behaviour everywhere., Comment: 33 pages

We prove that Hausdorff limit of topological minimal sets (with finitely generated coefficient group) are topologically minimal. The key idea is to reduce the homology group on the space to the homology group on the sphere, and reduce the homology group on the sphere to a finitely representable one, by "glueing" grids with small measure to block local elements in the homology group.

In this article we prove the topological minimality of unions of several almost orthogonal planes of arbitrary dimensions. A particular case was proved in arXiv:1103.1468, where we proved the Almgren minimality (which is a weaker property than the topological minimality) of the union of two almost orthogonal 2 dimensional planes. On the one hand, the topological minimality is always proved by variations of calibration methods, but in this article, we give a continuous family topological minimal sets, hence calibrations cannot apply. The advantage of a set being topological minimal (compared to Almgren minimal) is that its product with $\R^n$ stays topological minimal. This leads also to finding minimal sets which are unions of non transversal (hence far from almost orthogonal) planes; On the other hand, regularity for higher dimensional minimal sets is much less clear than those of dimension 2, hence more efforts are needed for higher dimensional cases., Comment: 44 pages. arXiv admin note: substantial text overlap with arXiv:1103.1468

In this paper we discuss various minimality properties for the orthogonal product of two 1-dimensional $\Y$ sets, and some related problems. This is motivated by an attempt to give the classification of singularities for 2-dimensional Almgren-minimal sets in $\R^4$., Comment: 41 pages

We discuss the global regularity of 2 dimensional minimal sets that are near a union of two planes, and prove that every global minimal set in R^4 that looks like a union of two almost orthogonal planes at infinity is a cone. The main point is to use the topological properties of a minimal set at a large scale to control its behavior at smaller scales., Comment: 30 pages. arXiv admin note: substantial text overlap with arXiv:1112.3565

We discuss the global regularity for 2 dimensional minimal sets that are near a $\T$ set, that is, whether every global minimal set in $\R^n$ that looks like a $\T$ set at infinity is a $\T$ set or not. The main point is to use the topological properties of a minimal set at large scale to control its topology at smaller scales. This is the idea to prove that all 1-dimensional Almgren-minimal sets in $\R^n$, and all 2-dimensional Mumford-Shah minimal sets in $\R^3$ are cones. In this article we discuss two types of 2-dimensional minimal sets: Almgren-minimal set in $\R^3$ whose blow-in limit is a $\T$ set; topological minimal sets in $\R^4$ whose blow-in limit is a $\T$ set. For the first one we eliminate an existing potential counterexample that was proposed by several people, and show that a real counterexample should have a more complicated topological structure; for the second we construct a potential example using a Klein bottle., Comment: 38 pages

In this article we introduce a definition of topological minimal sets, which is a generalization of that of Mumford-Shah-minimal sets. We prove some general properties as well as two existence theorems for topological minimal sets. As an application we prove the topological minimality of the union of two almost orthogonal planes in $\R^4$, and use it to improve the angle criterion under which the union of several higher dimensional planes is Almgren-minimal.

In this article we prove that the union of two almost orthogonal planes in R4 is Almgren-minimal. This gives an example of a one parameter family of minimal cones, which is a phenomenon that does not exist in R3. This work is motivated by an attempt to classify the singularities of 2-dimensional Almgren-minimal sets in R4. Note that the traditional methods for proving minimality (calibrations and slicing arguments) do not apply here, we are obliged to use some more complicated arguments such as a stopping time argument, harmonic extensions, Federer-Fleming projections, etc. that are rarely used to prove minimality (they are often used to prove regularity). The regularity results for 2-dimensional Almgren minimal sets ([5],[6]) are also needed here., Comment: 67 pages