Yan, D, Sheng, Y and Mao, X (2019) Pencil Drawing Video Rendering Using Convolutional Networks. Computer Graphics Forum, 38 (7). pp. 91-102. ISSN 0167-7055

Preview

Text upload_version.pdf - Accepted Version Download (7MB) | Preview

Abstract

Traditional pencil drawing rendering algorithms when applied to video may suffer from temporal inconsistency and showerdoor effect due to the stochastic noise models employed. This paper attempts to resolve these problems with deep learning. Recently, many research endeavors have demonstrated that feed-forward Convolutional Neural Networks (CNNs) are capable of using a reference image to stylize a whole video sequence while removing the shower-door effect in video style transfer applications. Compared with video style transfer, pencil drawing video is more sensitive to the inconsistency of texture and requires a stronger expression of pencil hatching. Thus, in this paper we develop an approach by combining a latest Line Integral Convolution (LIC) based method, specializing in realistically simulating pencil drawing images, with a new feedforward CNN that can eliminate the shower-door effect successfully. Taking advantage of optical flow, we adopt a feature-maplevel temporal loss function and propose a new framework to avoid the temporal inconsistency between consecutive frames, enhancing the visual impression of pencil strokes and tone. Experimental comparisons with the existing feed-forward CNNs have demonstrated that our method can generate temporally more stable and visually more pleasant pencil drawing video results in a faster manner.

Item Type:	Article
Additional Information:	This is the peer reviewed version of the following article: Yan, D., Sheng, Y. and Mao, X. (2019), Pencil Drawing Video Rendering Using Convolutional Networks. Computer Graphics Forum, 38: 91-102. which has been published in final form at https://doi.org/10.1111/cgf.13819. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.
Uncontrolled Keywords:	0801 Artificial Intelligence and Image Processing
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Divisions:	Computer Science & Mathematics
Publisher:	Wiley
Related URLs:	https://github.com/Kanata-Bifang/CNN-for-video-transfer
Date Deposited:	28 Oct 2019 10:19
Last Modified:	04 Sep 2021 08:35
DOI or ID number:	10.1111/cgf.13819
URI:	https://researchonline.ljmu.ac.uk/id/eprint/11650

View Item