Skip to content

Commit

Permalink
updated ICME 24
Browse files Browse the repository at this point in the history
  • Loading branch information
@MarkMoHR
MarkMoHR committed Apr 15, 2024
1 parent 2a7f957 commit dc884ae
Show file tree
Hide file tree
Showing 8 changed files with 156 additions and 11 deletions.
81 changes: 78 additions & 3 deletions en/projects_2Dshape.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
<li><a class="mybuttonlow" href="#editing_and_synthesis">Image Editing and Synthesis</a></li>
<li><a class="mybuttonlow" href="#pose">3D Pose Estimation and Motion Generation</a></li>
<li><a class="mybuttonlow" href="#fashion">Garment Modeling and Virtual Try-on</a></li>
<li><a class="mybuttonlow" href="#3d_modeling">3D Rendering & Modeling</a></li>
<li><a class="mybuttonlow" href="#3d_modeling">Multimedia Processing & 3D Rendering and Modeling</a></li>
</ul>

<section id="sketch">
Expand Down Expand Up @@ -122,6 +122,59 @@ <h3> <a href="https://markmohr.github.io/JoSTC/">Joint Stroke Tracing and Corres
</td>
</tr>

<tr>
<td width="30%">
<img src="../files/images/projects/icme24-editing.gif" alt="" width="100%" border="1"/>
</td>
<td width="70%">
<h3> <a href="https://github.com/MarkMoHR/DiffSketchEdit">Text-based Vector Sketch Editing with Image Editing Diffusion Prior</a></h3>
<p>
Haoran Mo, Xusheng Lin, Chengying Gao<sup>*</sup> and Ruomei Wang
<br>
<br>
<strong>Intro: </strong>
We present a framework for text-based vector sketch editing to improve the efficiency of graphic design.
The key idea behind the approach is to transfer the prior information from raster-level diffusion models,
especially those from image editing methods, into the vector sketch-oriented task.
The framework presents three editing modes and allows iterative editing.
To meet the editing requirement of modifying the intended parts only while avoiding changing the other strokes,
we introduce a stroke-level local editing scheme that automatically produces an editing mask reflecting locally editable regions and modifies strokes within the regions only.
<br>
<br>
<em>International Conference on Multimedia & Expo (<strong>ICME</strong>, 2024) &nbsp;<strong><font color="#FF0000">(CCF-B)</font></strong></em>
<br>
<a href="">[Paper]</a>
<a href="https://github.com/MarkMoHR/DiffSketchEdit">[Code]</a>
</p>
</td>
</tr>

<tr>
<td width="30%">
<img src="../files/images/projects/icme24-animation.png" alt="" width="100%" border="1"/>
</td>
<td width="70%">
<h3> <a href="">Video-Driven Sketch Animation via Cyclic Reconstruction Mechanism</a></h3>
<p>
Zhuo Xie, Haoran Mo and Chengying Gao<sup>*</sup>
<br>
<br>
<strong>Intro: </strong>
Considering the time-consuming manual workflow in 2D sketch animation production,
we present an automatic solution by using videos as reference to animate the static sketch images.
This includes motion extraction from the videos and injection into the sketches to produce animated sketch sequences in which appearance properties from the source sketches should be preserved.
To reduce blurry artifact caused by complex motions and maintain stroke line continuity,
we propose to incorporate inner masks of the sketches as an explicit guidance to indicate inner regions and ensure component integrality.
Moreover, to bridge the domain gap between the video frames and the sketches when modelling the motions,
we introduce a cyclic reconstruction mechanism to increase compatibility with different domains and improve motion consistency between the sketch animation and the driving video.
<br>
<br>
<em>International Conference on Multimedia & Expo (<strong>ICME</strong>, 2024) &nbsp;<strong><font color="#FF0000">(CCF-B)</font></strong></em>
<br>
</p>
</td>
</tr>

<tr>
<td width="30%">
<img src="../files/images/projects/pg2022.jpg" alt="" width="100%" border="1"/>
Expand Down Expand Up @@ -617,15 +670,37 @@ <h3> <a href="https://diglib.eg.org/handle/10.2312/pgs.20141250.043-048">Automat

<section id="3d_modeling">
<header class="major">
<h2>3D Rendering & Modeling</h2>
<h2>Multimedia Processing & 3D Rendering and Modeling</h2>
</header>

<p>
Including: fast fluid surface reconstruction based on narrow band method, fabric modeling and rendering.
Multimedia Processing: generation and understanding of music and dance.
<br>
3D Rendering and Modeling: fast fluid surface reconstruction based on narrow band method, fabric modeling and rendering.
</p>
<br>

<table width="100%" align="center" border="0" cellspacing="0" cellpadding="10">
<tr>
<td width="30%">
<img src="../files/images/projects/icme24-piano.png" alt="" width="100%" border="1"/>
</td>
<td width="70%">
<h3> <a href="">Video-Driven Sketch Animation via Cyclic Reconstruction Mechanism</a></h3>
<p>
Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang and Chengying Gao
<br>
<br>
<strong>Intro: </strong>
Learning musical structures and composition patterns is necessary for both music generation and understanding, but current methods do not make uniform use of learned features to generate and comprehend music simultaneously. In this paper, we propose PianoBART, a pre-trained model that uses BART for both symbolic piano music generation and understanding. We devise a multi-level object selection strategy for different pre-training tasks of PianoBART, which can prevent information leakage or loss and enhance learning ability. The musical semantics captured in pre-training are fine-tuned for music generation and understanding tasks. Experiments demonstrate that PianoBART efficiently learns musical patterns and achieves outstanding performance in generating high-quality coherent pieces and comprehending music.
<br>
<br>
<em>International Conference on Multimedia & Expo (<strong>ICME</strong>, 2024) &nbsp;<strong><font color="#FF0000">(CCF-B)</font></strong></em>
<br>
</p>
</td>
</tr>

<tr>
<td width="30%">
<img src="../files/images/projects/cgi2020.jpg" alt="" width="100%" border="1"/>
Expand Down
3 changes: 3 additions & 0 deletions en/publications.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ <h3>2024</h3>
</header>
<p>
<ol>
<li> Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang and Chengying Gao, <strong><a href=""> PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong> &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
<li> Zhuo Xie, Haoran Mo and Chengying Gao<sup>*</sup>, <strong><a href=""> Video-Driven Sketch Animation via Cyclic Reconstruction Mechanism</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong> &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
<li> Haoran Mo, Xusheng Lin, Chengying Gao<sup>*</sup> and Ruomei Wang, <strong><a href=""> Text-based Vector Sketch Editing with Image Editing Diffusion Prior</a></strong>, Proceedings of IEEE International Conference on Multimedia and Expo <strong>(ICME 2024)</strong> &nbsp;<em><strong><font color="#FF0000">(CCF-B)</font></strong></em></li>
<li> Haoran Mo, Chengying Gao<sup>*</sup> and Ruomei Wang, <strong><a href="https://dl.acm.org/doi/10.1145/3649890"> Joint Stroke Tracing and Correspondence for 2D Animation</a></strong>, ACM Transactions on Graphics (Presented at <strong>SIGGRAPH 2024</strong>) &nbsp;<em><strong><font color="#FF0000">(CCF-A)</font></strong></em>
</ol>
</p>
Expand Down
Binary file modified files/images/home/sketch-teasers.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added files/images/projects/icme24-animation.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added files/images/projects/icme24-editing.gif
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added files/images/projects/icme24-piano.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit dc884ae

Please sign in to comment.