added TMM

sysu-imsl · May 9, 2024 · dcac764 · dcac764
1 parent 5179362
commit dcac764
Show file tree

Hide file tree

Showing 5 changed files with 42 additions and 1 deletion.
diff --git a/en/projects_2Dshape.html b/en/projects_2Dshape.html
@@ -681,12 +681,32 @@ <h2>Multimedia Processing & 3D Rendering and Modeling</h2>
                         <br>
 
                         <table width="100%" align="center" border="0" cellspacing="0" cellpadding="10">
+                            <tr>
+                                <td width="30%">
+								    <img src="../files/images/projects/tmm2024.png" alt="" width="100%" border="1"/>
+                                </td>
+                                <td width="70%">
+								    <h3> <a href="">DanceComposer: Dance-to-Music Generation Using a Progressive Conditional Music Generator</a></h3>
+                                    <p>
+                                        Xiao Liang, Wensheng Li, Lifeng Huang and Chengying Gao
+                                        <br>
+                                        <br>
+                                        <strong>Intro: </strong>
+                                        A wonderful piece of music is the essence and soul of dance, which motivates the study of automatic music generation for dance. To create appropriate music from dance, cross-modal correlations between dance and music such as rhythm and style, should be considered. However, existing dance-to-music methods have difficulties in achieving rhythmic alignment and stylistic matching simultaneously. Additionally, the diversity of generated samples is limited due to the lack of available paired data. To address these issues, we propose DanceComposer, a novel dance-to-music framework, which generates rhythmically and stylistically consistent multi-track music from dance videos. DanceComposer features a Progressive Conditional Music Generator (PCMG) that gradually incorporates rhythm and style constraints, enabling both rhythmic alignment and stylistic matching. To enhance style control, we introduce a Shared Style Module (SSM) that learns cross-modal features as stylistic constraints. This allows the PCMG can be trained on extensive music-only data and diversifies generated pieces. 
+                                        <br>
+                                        <br>
+                                        <em>IEEE Transactions on Multimedia (<strong>TMM</strong>, 2024) &nbsp;<strong><font color="#FF0000">(中科院 1区/CCF-B)</font></strong></em>
+                                        <br>
+                                    </p>
+                                </td>
+                            </tr>
+
                             <tr>
                                 <td width="30%">
 								    <img src="../files/images/projects/icme24-piano.png" alt="" width="100%" border="1"/>
                                 </td>
                                 <td width="70%">
-								    <h3> <a href="">Video-Driven Sketch Animation via Cyclic Reconstruction Mechanism</a></h3>
+								    <h3> <a href="">PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training</a></h3>
                                     <p>
                                         Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang and Chengying Gao
                                         <br>

diff --git a/en/publications.html b/en/publications.html
@@ -58,6 +58,7 @@ <h3>2024</h3>
 						</header>
 						<p>
 							<ol>
+                                <li> Xiao Liang, Wensheng Li, Lifeng Huang and Chengying Gao, <strong><a href=""> DanceComposer: Dance-to-Music Generation Using a Progressive Conditional Music Generator</a></strong>, IEEE Transactions on Multimedia, <strong>(TMM 2024)</strong>&nbsp;<em><strong><font color="#FF0000">(中科院 1区/CCF-B)</font></strong></em>
 								<li> Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang and Chengying Gao, <strong><a href=""> PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong>  &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
 								<li> Zhuo Xie, Haoran Mo and Chengying Gao<sup>*</sup>, <strong><a href=""> Video-Driven Sketch Animation via Cyclic Reconstruction Mechanism</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong>  &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
 								<li> Haoran Mo, Xusheng Lin, Chengying Gao<sup>*</sup> and Ruomei Wang, <strong><a href=""> Text-based Vector Sketch Editing with Image Editing Diffusion Prior</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong>  &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>

diff --git a/files/images/projects/tmm2024.png b/files/images/projects/tmm2024.png
diff --git a/projects_2Dshape.html b/projects_2Dshape.html
@@ -705,6 +705,25 @@ <h2 class="head-zh">多媒体处理与三维渲染建模</h2>
                         <br>
 
                         <table width="100%" align="center" border="0" cellspacing="0" cellpadding="10">
+                            <tr>
+                                <td width="30%">
+								    <img src="./files/images/projects/tmm2024.png" alt="" width="100%" border="1"/>
+                                </td>
+                                <td width="70%">
+								    <h3> <a href="">DanceComposer: Dance-to-Music Generation Using a Progressive Conditional Music Generator</a></h3>
+                                    <p>
+                                        Xiao Liang, Wensheng Li, Lifeng Huang and Chengying Gao
+                                        <br>
+                                        <br>
+                                        <strong>简介：</strong>
+                                        音乐是舞蹈的精髓和灵魂，这推动了从舞蹈自动生成音乐的研究。为了创作与舞蹈匹配的音乐，需考虑舞蹈和音乐的跨模态相关性，如节奏上的协调和风格上的和谐。然而，现有的舞蹈到音乐生成方法很难同时实现节奏对齐和风格匹配。此外，由于缺乏可用的成对数据，生成样本的多样性也很有限。为了解决这些问题，我们提出了一个新的舞蹈到音乐生成框架DanceComposer，它能从舞蹈视频中生成节奏和风格一致的多音轨音乐。DanceComposer采用渐进式条件音乐生成器 (PCMG)，可逐步添加节奏和风格约束，实现节奏对齐和风格匹配。为了加强风格控制，我们引入了共享风格模块（SSM），该模块可学习跨模态特征作为风格约束。这使得PCMG可以在大量纯音乐数据上进行训练，并使生成的作品更加多样化。定量和定性结果表明，我们的方法在整体音乐质量、节奏一致性和风格一致性方面都超越了现有的方法。
+                                        <br>
+                                        <br>
+                                        <em>IEEE Transactions on Multimedia (<strong>TMM</strong>, 2024) &nbsp;<strong><font color="#FF0000">(中科院 1区/CCF-B)</font></strong></em>
+                                        <br>
+                                    </p>
+                                </td>
+                            </tr>
                             <tr>
                                 <td width="30%">
 								    <img src="./files/images/projects/icme24-piano.png" alt="" width="100%" border="1"/>

diff --git a/publications_papers.html b/publications_papers.html
@@ -57,6 +57,7 @@ <h3>2024</h3>
 						</header>
 						<p>
 							<ol>
+                                <li> Xiao Liang, Wensheng Li, Lifeng Huang and Chengying Gao, <strong><a href=""> DanceComposer: Dance-to-Music Generation Using a Progressive Conditional Music Generator</a></strong>, IEEE Transactions on Multimedia, <strong>(TMM 2024)</strong>&nbsp;<em><strong><font color="#FF0000">(中科院 1区/CCF-B)</font></strong></em>
 								<li> Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang and Chengying Gao, <strong><a href=""> PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong>  &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
 								<li> Zhuo Xie, Haoran Mo and Chengying Gao<sup>*</sup>, <strong><a href=""> Video-Driven Sketch Animation via Cyclic Reconstruction Mechanism</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong>  &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
 								<li> Haoran Mo, Xusheng Lin, Chengying Gao<sup>*</sup> and Ruomei Wang, <strong><a href=""> Text-based Vector Sketch Editing with Image Editing Diffusion Prior</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong>  &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>