基于AI的材质研究

资料汇总

论文

分类并按年份排序。

期刊会收录会议上的论文，比如TOG (SG), CGF (EG)。TOG每一年有六期，第四期收录SIGGRAPH，第六期收录SIGGRAPH Asia（仅Technical Papers，不包括Conference Proceedings）。

SVBRDF

估计SVBRDF相关的解析BRDF模型的参数（如Diffuse、Specular、Roughnesss、Normal四张贴图）。

Reflectance Modeling by Neural Texture Synthesis

【TOG2016】

1
2
3
4
5
6
7
8
9
10
@article{aittala2016reflectance,
  title={Reflectance modeling by neural texture synthesis},
  author={Aittala, Miika and Aila, Timo and Lehtinen, Jaakko},
  journal={ACM Transactions on Graphics (ToG)},
  volume={35},
  number={4},
  pages={1--13},
  year={2016},
  publisher={ACM New York, NY, USA}
}

Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks

【TOG2017】【Github】

1
2
3
4
5
6
7
8
9
10
@article{li2017modeling,
  title={Modeling surface appearance from a single photograph using self-augmented convolutional neural networks},
  author={Li, Xiao and Dong, Yue and Peers, Pieter and Tong, Xin},
  journal={ACM Transactions on Graphics (ToG)},
  volume={36},
  number={4},
  pages={1--11},
  year={2017},
  publisher={ACM New York, NY, USA}
}

Single-Image SVBRDF Capture with a Rendering-Aware Deep Network

【TOG2018】【Github】

1
2
3
4
5
6
7
8
9
10
@article{deschaintre2018single,
  title={Single-image svbrdf capture with a rendering-aware deep network},
  author={Deschaintre, Valentin and Aittala, Miika and Durand, Fredo and Drettakis, George and Bousseau, Adrien},
  journal={ACM Transactions on Graphics (ToG)},
  volume={37},
  number={4},
  pages={1--15},
  year={2018},
  publisher={ACM New York, NY, USA}
}

Efficient Reflectance Capture Using an Autoencoder

【TOG2018】

OpenSVBRDF的前置工作之一。

1
2
3
4
5
6
7
8
9
@article{kang2018efficient,
  title={Efficient reflectance capture using an autoencoder.},
  author={Kang, Kaizhang and Chen, Zimin and Wang, Jiaping and Zhou, Kun and Wu, Hongzhi},
  journal={ACM Trans. Graph.},
  volume={37},
  number={4},
  pages={127--1},
  year={2018}
}

Single Image Surface Appearance Modeling with Self-augmented CNNs and Inexact Supervision

【CGF2018】【Github】

1
2
3
4
5
6
7
8
9
10
@inproceedings{ye2018single,
  title={Single image surface appearance modeling with self-augmented cnns and inexact supervision},
  author={Ye, Wenjie and Li, Xiao and Dong, Yue and Peers, Pieter and Tong, Xin},
  booktitle={Computer Graphics Forum},
  volume={37},
  number={7},
  pages={201--211},
  year={2018},
  organization={Wiley Online Library}
}

Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image

【ECCV2018】

1
2
3
4
5
6
7
@inproceedings{li2018materials,
  title={Materials for masses: SVBRDF acquisition with a single mobile phone image},
  author={Li, Zhengqin and Sunkavalli, Kalyan and Chandraker, Manmohan},
  booktitle={Proceedings of the European conference on computer vision (ECCV)},
  pages={72--87},
  year={2018}
}

Flexible SVBRDF Capture with a Multi-Image Deep Network

【CGF2019】【Github】

1
2
3
4
5
6
7
8
9
10
@inproceedings{deschaintre2019flexible,
  title={Flexible svbrdf capture with a multi-image deep network},
  author={Deschaintre, Valentin and Aittala, Miika and Durand, Fr{\'e}do and Drettakis, George and Bousseau, Adrien},
  booktitle={Computer graphics forum},
  volume={38},
  number={4},
  pages={1--13},
  year={2019},
  organization={Wiley Online Library}
}

Deep Inverse Rendering for High-resolution SVBRDF Estimation from an Arbitrary Number of Images

【TOG2019】【Github】

1
2
3
4
5
6
7
8
9
@article{gao2019deep,
  title={Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images.},
  author={Gao, Duan and Li, Xiao and Dong, Yue and Peers, Pieter and Xu, Kun and Tong, Xin},
  journal={ACM Trans. Graph.},
  volume={38},
  number={4},
  pages={134--1},
  year={2019}
}

MaterialGAN: Reflectance Capture using a Generative SVBRDF Model

【SGA2020】【Github】

1
2
3
4
5
6
@article{guo2020materialgan,
  title={Materialgan: reflectance capture using a generative svbrdf model},
  author={Guo, Yu and Smith, Cameron and Ha{\v{s}}an, Milo{\v{s}} and Sunkavalli, Kalyan and Zhao, Shuang},
  journal={arXiv preprint arXiv:2010.00114},
  year={2020}
}

Guided Fine-Tuning for Large-Scale Material Transferm Flash Images

【CGF2020】

1
2
3
4
5
6
7
8
9
10
@inproceedings{deschaintre2020guided,
  title={Guided fine-tuning for large-scale material transfer},
  author={Deschaintre, Valentin and Drettakis, George and Bousseau, Adrien},
  booktitle={Computer Graphics Forum},
  volume={39},
  number={4},
  pages={91--105},
  year={2020},
  organization={Wiley Online Library}
}

Deep SVBRDF Estimation on Real Materials

【3DV2020】

1
2
3
4
5
6
7
8
@inproceedings{asselin2020deep,
  title={Deep SVBRDF estimation on real materials},
  author={Asselin, Louis-Philippe and Laurendeau, Denis and Lalonde, Jean-Fran{\c{c}}ois},
  booktitle={2020 International Conference on 3D Vision (3DV)},
  pages={1157--1166},
  year={2020},
  organization={IEEE}
}

Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network

【EGSR2020】

1
2
3
4
5
6
7
@inproceedings{zhao2020joint,
  title={Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network.},
  author={Zhao, Yezi and Wang, Beibei and Xu, Yanning and Zeng, Zheng and Wang, Lu and Holzschuch, Nicolas},
  booktitle={EGSR (DL)},
  pages={53--66},
  year={2020}
}

SurfaceNet: Adversarial SVBRDF Estimation from a Single Image

【ICCV2021】【Github】

1
2
3
4
5
6
7
@inproceedings{vecchio2021surfacenet,
  title={Surfacenet: Adversarial svbrdf estimation from a single image},
  author={Vecchio, Giuseppe and Palazzo, Simone and Spampinato, Concetto},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={12840--12848},
  year={2021}
}

Generative Modelling of BRDF Textures from Flash Images

【arXiv2021】

1
2
3
4
5
6
@article{henzler2021generative,
  title={Generative modelling of BRDF textures from flash images},
  author={Henzler, Philipp and Deschaintre, Valentin and Mitra, Niloy J and Ritschel, Tobias},
  journal={arXiv preprint arXiv:2102.11861},
  year={2021}
}

Adversarial Single-Image SVBRDF Estimation with Hybrid Training

【CGF2021】【Github】

1
2
3
4
5
6
7
8
9
10
@inproceedings{zhou2021adversarial,
  title={Adversarial Single-Image SVBRDF Estimation with Hybrid Training},
  author={Zhou, Xilong and Kalantari, Nima Khademi},
  booktitle={Computer Graphics Forum},
  volume={40},
  number={2},
  pages={315--325},
  year={2021},
  organization={Wiley Online Library}
}

Highlight-Aware Two-Stream Network for Single-Image SVBRDF Acquisition

【TOG2021】【Github】

1
2
3
4
5
6
7
8
9
10
@article{guo2021highlight,
  title={Highlight-aware two-stream network for single-image SVBRDF acquisition},
  author={Guo, Jie and Lai, Shuichang and Tao, Chengzhi and Cai, Yuelong and Wang, Lei and Guo, Yanwen and Yan, Ling-Qi},
  journal={ACM Transactions on Graphics (TOG)},
  volume={40},
  number={4},
  pages={1--14},
  year={2021},
  publisher={ACM New York, NY, USA}
}

TileGen: Tileable, Controllable Material Generation and Capture

【SGA2022】【Github】

1
2
3
4
5
6
7
@inproceedings{zhou2022tilegen,
  title={Tilegen: Tileable, controllable material generation and capture},
  author={Zhou, Xilong and Hasan, Milos and Deschaintre, Valentin and Guerrero, Paul and Sunkavalli, Kalyan and Kalantari, Nima Khademi},
  booktitle={SIGGRAPH Asia 2022 Conference Papers},
  pages={1--9},
  year={2022}
}

Look-Ahead Training with Learned Reflectance Loss for Single-Image SVBRDF Estimation

【TOG2022】【Github】

1
2
3
4
5
6
7
8
9
10
@article{zhou2022look,
  title={Look-Ahead Training with Learned Reflectance Loss for Single-Image SVBRDF Estimation},
  author={Zhou, Xilong and Kalantari, Nima Khademi},
  journal={ACM Transactions on Graphics (TOG)},
  volume={41},
  number={6},
  pages={1--12},
  year={2022},
  publisher={ACM New York, NY, USA}
}

SVBRDF Recovery From a Single Image with Highlights using a Pretrained Generative Adversarial Network

【CGF2022】

1
2
3
4
5
6
7
8
9
10
@inproceedings{wen2022svbrdf,
  title={SVBRDF Recovery from a Single Image with Highlights Using a Pre-trained Generative Adversarial Network},
  author={Wen, Tao and Wang, Beibei and Zhang, Lei and Guo, Jie and Holzschuch, Nicolas},
  booktitle={Computer Graphics Forum},
  volume={41},
  number={6},
  pages={110--123},
  year={2022},
  organization={Wiley Online Library}
}

MaterIA: Single Image High-Resolution Material Capture in the Wild

【CGF2022】

1
2
3
4
5
6
7
8
9
10
@inproceedings{martin2022materia,
  title={MaterIA: Single Image High-Resolution Material Capture in the Wild},
  author={Martin, Rosalie and Roullier, Arthur and Rouffet, Romain and Kaiser, Adrien and Boubekeur, Tamy},
  booktitle={Computer Graphics Forum},
  volume={41},
  number={2},
  pages={163--177},
  year={2022},
  organization={Wiley Online Library}
}

Data Driven SVBRDF Estimation Using Deep Embedded Clustering

【Electronics2022】

1
2
3
4
5
6
7
8
9
10
@article{kim2022data,
  title={Data Driven SVBRDF Estimation Using Deep Embedded Clustering},
  author={Kim, Yong Hwi and Lee, Kwan H},
  journal={Electronics},
  volume={11},
  number={19},
  pages={3239},
  year={2022},
  publisher={MDPI}
}

Deep SVBRDF Estimation from Single Image under Learned Planar Lighting

【SG2023】

1
2
3
4
5
6
7
@inproceedings{zhang2023deep,
  title={Deep SVBRDF Estimation from Single Image under Learned Planar Lighting},
  author={Zhang, Lianghao and Gao, Fangzhou and Wang, Li and Yu, Minjing and Cheng, Jiamin and Zhang, Jiawan},
  booktitle={ACM SIGGRAPH 2023 Conference Proceedings},
  pages={1--11},
  year={2023}
}

DeepBasis: Hand-Held Single-Image SVBRDF Capture via Two-Level Basis Material Model

【SGA2023】【Github】

1
2
3
4
5
6
7
@inproceedings{wang2023deepbasis,
  title={DeepBasis: Hand-Held Single-Image SVBRDF Capture via Two-Level Basis Material Model},
  author={Wang, Li and Zhang, Lianghao and Gao, Fangzhou and Zhang, Jiawan},
  booktitle={SIGGRAPH Asia 2023 Conference Papers},
  pages={1--11},
  year={2023}
}

MatFusion: A Generative Diffusion Model for SVBRDF Capture

【SGA2023】

1
2
3
4
5
6
7
@inproceedings{sartor2023matfusion,
  title={Matfusion: a generative diffusion model for svbrdf capture},
  author={Sartor, Sam and Peers, Pieter},
  booktitle={SIGGRAPH Asia 2023 Conference Papers},
  pages={1--10},
  year={2023}
}

Efficient Reflectance Capture with a Deep Gated Mixture-of-Experts

【TVCG2023】

1
2
3
4
5
6
7
@article{ma2023efficient,
  title={Efficient Reflectance Capture With a Deep Gated Mixture-of-Experts},
  author={Ma, Xiaohe and Yu, Yaxin and Wu, Hongzhi and Zhou, Kun},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  year={2023},
  publisher={IEEE}
}

OpenSVBRDF: A Database of Measured Spatially-Varying Reflectance

【TOG2023】【Page】

1
2
3
4
5
6
7
8
9
@article{ma2023opensvbrdf,
  title={OpenSVBRDF: A Database of Measured Spatially-Varying Reflectance},
  author={Ma, Xiaohe and Xu, Xianmin and Zhang, Leyao and Zhou, Kun and Wu, Hongzhi},
  journal={ACM Transactions on Graphics},
  volume={42},
  number={6},
  year={2023},
  publisher={ASSOC COMPUTING MACHINERY 1601 Broadway, 10th Floor, NEW YORK, NY USA}
}

Ultra-High Resolution SVBRDF Recovery from a Single Image

【TOG2023】

1
2
3
4
5
6
7
@article{guo2023ultra,
  title={Ultra-High Resolution SVBRDF Recovery from a Single Image},
  author={Guo, Jie and Lai, Shuichang and Tu, Qinghao and Tao, Chengzhi and Zou, Changqing and Guo, Yanwen},
  journal={ACM Transactions on Graphics},
  year={2023},
  publisher={ACM New York, NY}
}

PhotoMat: A Material Generator Learned from Single Flash Photos

【SG2023】【Github】

1
2
3
4
5
6
7
@inproceedings{zhou2023photomat,
  title={Photomat: A material generator learned from single flash photos},
  author={Zhou, Xilong and Hasan, Milos and Deschaintre, Valentin and Guerrero, Paul and Hold-Geoffroy, Yannick and Sunkavalli, Kalyan and Kalantari, Nima Khademi},
  booktitle={ACM SIGGRAPH 2023 Conference Proceedings},
  pages={1--11},
  year={2023}
}

BRDF

AI压缩BRDF测量数据，基于AI的BRDF模型。

Unified Neural Encoding of BTFs

【CGF2020】

1
2
3
4
5
6
7
8
9
10
@inproceedings{rainer2020unified,
  title={Unified neural encoding of BTFs},
  author={Rainer, Gilles and Ghosh, Abhijeet and Jakob, Wenzel and Weyrich, Tim},
  booktitle={Computer Graphics Forum},
  volume={39},
  number={2},
  pages={167--178},
  year={2020},
  organization={Wiley Online Library}
}

Neural BTF Compression and Interpolation

【CGF2020】

1
2
3
4
5
6
7
8
9
10
@inproceedings{rainer2019neural,
  title={Neural BTF compression and interpolation},
  author={Rainer, Gilles and Jakob, Wenzel and Ghosh, Abhijeet and Weyrich, Tim},
  booktitle={Computer Graphics Forum},
  volume={38},
  number={2},
  pages={235--244},
  year={2019},
  organization={Wiley Online Library}
}

DeepBRDF: A Deep Representation for Manipulating Measured BRDF

【CGF2020】

1
2
3
4
5
6
7
8
9
10
@inproceedings{hu2020deepbrdf,
  title={DeepBRDF: A deep representation for manipulating measured BRDF},
  author={Hu, Bingyang and Guo, Jie and Chen, Yanjun and Li, Mengtian and Guo, Yanwen},
  booktitle={Computer Graphics Forum},
  volume={39},
  number={2},
  pages={157--166},
  year={2020},
  organization={Wiley Online Library}
}

Neural BRDF Representation and Importance Sampling

【CGF2021】【Github】

1
2
3
4
5
6
7
8
9
10
@inproceedings{sztrajman2021neural,
  title={Neural BRDF representation and importance sampling},
  author={Sztrajman, Alejandro and Rainer, Gilles and Ritschel, Tobias and Weyrich, Tim},
  booktitle={Computer Graphics Forum},
  volume={40},
  number={6},
  pages={332--346},
  year={2021},
  organization={Wiley Online Library}
}

Invertible Neural BRDF for Object Inverse Rendering

【PAMI2021】【Github】

1
2
3
4
5
6
7
8
9
10
@article{chen2021invertible,
  title={Invertible neural BRDF for object inverse rendering},
  author={Chen, Zhe and Nobuhara, Shohei and Nishino, Ko},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume={44},
  number={12},
  pages={9380--9395},
  year={2021},
  publisher={IEEE}
}

A Compact Representation of Measured BRDFs Using Neural Processes

【TOG2021】【Github】

1
2
3
4
5
6
7
8
9
10
@article{zheng2021compact,
  title={A compact representation of measured brdfs using neural processes},
  author={Zheng, Chuankun and Zheng, Ruzhang and Wang, Rui and Zhao, Shuang and Bao, Hujun},
  journal={ACM Transactions on Graphics (TOG)},
  volume={41},
  number={2},
  pages={1--15},
  year={2021},
  publisher={ACM New York, NY}
}

Neural Layered BRDFs

【SA2022】

1
2
3
4
5
6
7
@inproceedings{fan2022neural,
  title={Neural Layered BRDFs},
  author={Fan, Jiahui and Wang, Beibei and Hasan, Milos and Yang, Jian and Yan, Ling-Qi},
  booktitle={ACM SIGGRAPH 2022 Conference Proceedings},
  pages={1--8},
  year={2022}
}

A Sparse Non-parametric BRDF Model

【TOG2022】

1
2
3
4
5
6
7
8
9
10
@article{tongbuasirilai2022sparse,
  title={A Sparse Non-parametric BRDF Model},
  author={Tongbuasirilai, Tanaboon and Unger, Jonas and Guillemot, Christine and Miandji, Ehsan},
  journal={ACM Transactions on Graphics},
  volume={41},
  number={5},
  pages={1--18},
  year={2022},
  publisher={ACM New York, NY}
}

Material & Shape

同时估计材质和形状。

Learning to Reconstruct Shape and Spatially-Varying Reflectance from a Single Image

【TOG2018】

1
2
3
4
5
6
7
8
9
10
@article{li2018learning,
  title={Learning to reconstruct shape and spatially-varying reflectance from a single image},
  author={Li, Zhengqin and Xu, Zexiang and Ramamoorthi, Ravi and Sunkavalli, Kalyan and Chandraker, Manmohan},
  journal={ACM Transactions on Graphics (TOG)},
  volume={37},
  number={6},
  pages={1--11},
  year={2018},
  publisher={ACM New York, NY, USA}
}

Learning Efficient Illumination Multiplexing for Joint Capture of Reflectance and Shape

【TOG2019】

1
2
3
4
5
6
7
8
9
@article{kang2019learning,
  title={Learning efficient illumination multiplexing for joint capture of reflectance and shape.},
  author={Kang, Kaizhang and Xie, Cihui and He, Chengan and Yi, Mingqi and Gu, Minyi and Chen, Zimin and Zhou, Kun and Wu, Hongzhi},
  journal={ACM Trans. Graph.},
  volume={38},
  number={6},
  pages={165--1},
  year={2019}
}

Two-shot Spatially-varying BRDF and Shape Estimation

【CVPR2020】【Github】

1
2
3
4
5
6
7
@inproceedings{boss2020two,
  title={Two-shot spatially-varying brdf and shape estimation},
  author={Boss, Mark and Jampani, Varun and Kim, Kihwan and Lensch, Hendrik and Kautz, Jan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={3982--3991},
  year={2020}
}

Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image

【CVPR2020】

1
2
3
4
5
6
7
@inproceedings{li2020inverse,
  title={Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image},
  author={Li, Zhengqin and Shafiei, Mohammad and Ramamoorthi, Ravi and Sunkavalli, Kalyan and Chandraker, Manmohan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={2475--2484},
  year={2020}
}

A Unified Spatial-Angular Structured Light for Single-View Acquisition of Shape and Reflectance

【CVPR2023】

1
2
3
4
5
6
7
@inproceedings{xu2023unified,
  title={A unified spatial-angular structured light for single-view acquisition of shape and reflectance},
  author={Xu, Xianmin and Lin, Yuxin and Zhou, Haoyang and Zeng, Chong and Yu, Yaxin and Zhou, Kun and Wu, Hongzhi},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={206--215},
  year={2023}
}

Material Cont.

其它材质相关。

NeuMIP: Multi-Resolution Neural Materials

【TOG2021】

1
2
3
4
5
6
7
8
@article{kuznetsov2021neumip,
  title={NeuMIP: Multi-resolution neural materials},
  author={Kuznetsov, Alexandr},
  journal={ACM Transactions on Graphics (TOG)},
  volume={40},
  number={4},
  year={2021}
}

Woven Fabric Capture from a Single Photo

【SGA2022】

1
2
3
4
5
6
7
@inproceedings{jin2022woven,
  title={Woven fabric capture from a single photo},
  author={Jin, Wenhua and Wang, Beibei and Hasan, Milos and Guo, Yu and Marschner, Steve and Yan, Ling-Qi},
  booktitle={SIGGRAPH Asia 2022 Conference Papers},
  pages={1--8},
  year={2022}
}

A Semi-procedural Convolutional Material Prior

【CGF2023】【Github】

1
2
3
4
5
6
7
@inproceedings{zhou2023semi,
  title={A Semi-Procedural Convolutional Material Prior},
  author={Zhou, Xilong and Ha{\v{s}}an, Milo{\v{s}} and Deschaintre, Valentin and Guerrero, Paul and Sunkavalli, Kalyan and Kalantari, Nima Khademi},
  booktitle={Computer Graphics Forum},
  year={2023},
  organization={Wiley Online Library}
}

工具

软件

Photogrammetry software for 3D capture - Adobe Substance 3D

  • 支持从照片生成材质。
  • 算法来自论文：MaterIA: Single Image High‐Resolution Material Capture in the Wild。

Bounding Box Software - Materialize

  • 支持从照片生成材质，编辑自由度很高。

ArmorLab | PBR Texture Creation

  • 基于节点，支持从照片生成材质、超分、修复、平铺转化、文生图、文字辅助变种生成。最高16k。

在线

Generate PBR Material with AI (toggle3d.com)

  • 支持从照片生成材质、文生材质、材质属性调整。

AI Texture Generator for Blender, Unreal, Unity | Polycam

  • 支持文生材质，可以附加参考图片。

Online AI PBR Material Generator - Digimans.ai

  • 支持从照片生成材质。

文章

AI生成游戏中基于物理的渲染（PBR）贴图探索 - 知乎 (zhihu.com)

Research
#CG
基于AI的材质研究
https://reddish.fun/posts/Research/AI-based-Material-Research/
作者
bit704
发布于
2023年8月11日
许可协议