Generating View-Consistent 3D Sketch
via Differentiable Curve Rendering

  • 1Jilin University
  • 2CAD&CG Lab, Zhejiang University


We propose Diff3DS, a novel differentiable rendering framework for generating view-consistent 3D sketch by optimizing 3D parametric curves under various supervisions. Specifically, we perform perspective projection to render the 3D rational Bézier curves into 2D curves, which are subsequently converted to a 2D raster image via our customized differentiable rasterizer. Our framework bridges the domains of 3D sketch and raster image, achieving end-to-end optimization of 3D sketch through gradients computed in the 2D image domain. Our Diff3DS can enable a series of novel 3D sketch generation tasks, including text-to-3D sketch and image-to-3D sketch, supported by the popular distillation-based supervision, such as Score Distillation Sampling (SDS). Extensive experiments have yielded promising results and demonstrated the potential of our framework.

Text-to-3D Sketch

Flying dragon, highly detailed, breathing fire.
A fire Phoenix, mythical bird, engulfed in flames.
Higly detailed,
majestic royal tall ship...
A flower on a office
table, straight table legs.

Image-to-3D Sketch

Training Process

More Results


title = {Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Rendering},
author = {Yibo Zhang and Lihong Wang and Changqing Zou and Tieru Wu and Rui Ma},
journal = {arXiv preprint arXiv:2405.15305},

The webpage template is borrowed from Coin3D and SIGNeRF.