StarGAN2 の履歴(No.5)

• 履歴一覧
• 差分 を表示
• 現在との差分 を表示
• ソース を表示
• StarGAN2 へ行く。
  • 1 (2024-07-04 (木) 01:36:34)
  • 2 (2024-07-04 (木) 01:40:05)
  • 3 (2024-07-04 (木) 18:16:25)
  • 4 (2024-07-06 (土) 23:36:48)
  • 5 (2024-07-08 (月) 14:51:03)
  • 6 (2024-08-01 (木) 00:21:32)
  • 7 (2024-08-01 (木) 18:35:48)
  • 8 (2024-08-01 (木) 22:44:55)
  • 9 (2024-08-03 (土) 17:42:43)
  • 10 (2024-08-06 (火) 17:10:03)
  • 11 (2024-08-06 (火) 23:23:00)
  • 12 (2024-08-08 (木) 23:24:18)
  • 13 (2024-08-18 (日) 01:43:56)
  • 14 (2024-09-02 (月) 23:51:17)
  • 15 (2024-09-03 (火) 14:44:14)
  • 16 (2024-09-05 (木) 01:03:15)
  • 17 (2024-09-05 (木) 13:17:28)
  • 18 (2024-09-21 (土) 17:25:44)
  • 19 (2024-10-12 (土) 19:05:22)
  • 20 (2024-11-05 (火) 00:06:28)
  • 21 (2024-11-16 (土) 14:05:28)

---

私的AI研究会 > StarGAN2

顔の合成：StarGAN-v2（その２）== 編集中 ==†

　「StarGAN-V2」で特徴操作・顔の合成をおこない顔を変化させる

「StarGAN-V2」†

概要†

• StarGAN v2 は韓国の Naver 社 Clova AI Rsearch チームによって提案されたスタイル変換モデル
  
• ベースモデルには GANs の一種である StarGAN と呼ばれるモデルが使用されている
  
  
  モデル概要図（下記論文所収）
  
  
• 論文「StarGAN v2: Diverse Image Synthesis for Multiple Domains」
  <paper>
  ・https://arxiv.org/pdf/1912.01865
  <framework>
  ・https://github.com/clovaai/stargan-v2
  

• 前回の検証
  ・顔の合成：StarGAN-v2
  

実行環境の構築†

対処した問題点とエラー詳細†

FileNotFoundError: [WinError 3] 指定されたパスが見つかりません。: '{:'†

• エラー内容
  
  ▼　FileNotFoundError: ログ
  ▲　FileNotFoundError: ログ

  (py38_gan) python main.py --mode sample --num_domains 2 --resume_iter 100000 --w_hpf 1 --checkpoint_dir expr/checkpoints/celeba_hq --result_dir expr/results/celeba_hq --src_dir assets/representative/celeba_hq/src_0 --ref_dir assets/representative/celeba_hq/ref_0
  Namespace(batch_size=8, beta1=0.0, beta2=0.99, checkpoint_dir='expr/checkpoints/celeba_hq', ds_iter=100000, eval_dir='expr/eval', eval_every=50000, f_lr=1e-06, hidden_dim=512, img_size=256, inp_dir='assets/representative/custom/female', lambda_cyc=1, lambda_ds=1, lambda_reg=1, lambda_sty=1, latent_dim=16, lm_path='expr/checkpoints/celeba_lm_mean.npz', lr=0.0001, mode='sample', num_domains=2, num_outs_per_domain=10, num_workers=4, out_dir='assets/representative/celeba_hq/src/female', print_every=10, randcrop_prob=0.5, ref_dir='assets/representative/celeba_hq/ref_0', result_dir='expr/results/celeba_hq', resume_iter=100000, sample_dir='expr/samples', sample_every=5000, save_every=10000, seed=777, src_dir='assets/representative/celeba_hq/src_0', style_dim=64, total_iters=100000, train_img_dir='data/celeba_hq/train', val_batch_size=32, val_img_dir='data/celeba_hq/val', w_hpf=1.0, weight_decay=0.0001, wing_path='expr/checkpoints/wing.ckpt')
  Number of parameters of generator: 43467395
  Number of parameters of mapping_network: 2438272
  Number of parameters of style_encoder: 20916928
  Number of parameters of discriminator: 20852290
  Number of parameters of fan: 6333603
  Traceback (most recent call last):
    File "main.py", line 182, in <module>
      main(args)
    File "main.py", line 37, in main
      solver = Solver(args)
    File "N:\anaconda_win\work\stargan-v2\core\solver.py", line 58, in __init__
      self.ckptios = [CheckpointIO(ospj(args.checkpoint_dir, '{:06d}_nets_ema.ckpt'), data_parallel=True, **self.nets_ema)]
    File "N:\anaconda_win\work\stargan-v2\core\checkpoint.py", line 17, in __init__
      os.makedirs(os.path.dirname(fname_template), exist_ok=True)
    File "C:\Users\izuts\anaconda3\envs\py38_gan\lib\os.py", line 223, in makedirs
      mkdir(name, mode)
  FileNotFoundError: [WinError 3] 指定されたパスが見つかりません。: '{:'

• 参考サイト
  Path error upon running the example you provided to transform a custom image #3
  

• 対処法：「stargan-v2/core/solver.py」→ 58行目 変更
  

          else:
  #            self.ckptios = [CheckpointIO(ospj(args.checkpoint_dir, '{:06d}_nets_ema.ckpt'), data_parallel=True, **self.nets_ema)]
              self.ckptios = [CheckpointIO(ospj(args.checkpoint_dir, '100000_nets_ema.ckpt'), data_parallel=True, **self.nets_ema)]

RuntimeError: Error(s) in loading state_dict for Generator: Missing key(s) in state_dict: "hpf.filter"†

• エラー内容
  
  ▼　RuntimeError: ログ
  ▲　RuntimeError: ログ

  (py38_gan) python main.py --mode sample --num_domains 2 --resume_iter 100000 --w_hpf 1 --checkpoint_dir expr/checkpoints/celeba_hq --result_dir expr/results/celeba_hq --src_dir assets/representative/celeba_hq/src_0 --ref_dir assets/representative/celeba_hq/ref_0
  Namespace(batch_size=8, beta1=0.0, beta2=0.99, checkpoint_dir='expr/checkpoints/celeba_hq', ds_iter=100000, eval_dir='expr/eval', eval_every=50000, f_lr=1e-06, hidden_dim=512, img_size=256, inp_dir='assets/representative/custom/female', lambda_cyc=1, lambda_ds=1, lambda_reg=1, lambda_sty=1, latent_dim=16, lm_path='expr/checkpoints/celeba_lm_mean.npz', lr=0.0001, mode='sample', num_domains=2, num_outs_per_domain=10, num_workers=4, out_dir='assets/representative/celeba_hq/src/female', print_every=10, randcrop_prob=0.5, ref_dir='assets/representative/celeba_hq/ref_0', result_dir='expr/results/celeba_hq', resume_iter=100000, sample_dir='expr/samples', sample_every=5000, save_every=10000, seed=777, src_dir='assets/representative/celeba_hq/src_0', style_dim=64, total_iters=100000, train_img_dir='data/celeba_hq/train', val_batch_size=32, val_img_dir='data/celeba_hq/val', w_hpf=1.0, weight_decay=0.0001, wing_path='expr/checkpoints/wing.ckpt')
  Number of parameters of generator: 43467395
  Number of parameters of mapping_network: 2438272
  Number of parameters of style_encoder: 20916928
  Number of parameters of discriminator: 20852290
  Number of parameters of fan: 6333603
  Initializing generator...
  Initializing mapping_network...
  Initializing style_encoder...
  Initializing discriminator...
  Preparing DataLoader for the generation phase...
  Preparing DataLoader for the generation phase...
  Loading checkpoint from expr/checkpoints/celeba_hq\100000_nets_ema.ckpt...
  Traceback (most recent call last):
    File "main.py", line 182, in <module>
      main(args)
    File "main.py", line 73, in main
      solver.sample(loaders)
    File "C:\Users\izuts\anaconda3\envs\py38_gan\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
      return func(*args, **kwargs)
    File "N:\anaconda_win\work\stargan-v2\core\solver.py", line 179, in sample
      self._load_checkpoint(args.resume_iter)
    File "N:\anaconda_win\work\stargan-v2\core\solver.py", line 74, in _load_checkpoint
      ckptio.load(step)
    File "N:\anaconda_win\work\stargan-v2\core\checkpoint.py", line 48, in load
      module.module.load_state_dict(module_dict[name])
    File "C:\Users\izuts\anaconda3\envs\py38_gan\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
      raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
  RuntimeError: Error(s) in loading state_dict for Generator:
          Missing key(s) in state_dict: "hpf.filter".

• 参考サイト
  Error when loading state_dict for Generator #103
  

• 対処法：「stargan-v2/core/checkpoint.py」→ 48行目 変更
  

              if self.data_parallel:
  #                module.module.load_state_dict(module_dict[name])
                  module.module.load_state_dict(module_dict[name], False)

UnboundLocalError: local variable 'frames' referenced before assignment†

• エラー内容
  
  ▼　UnboundLocalError: ログ
  ▲　UnboundLocalError: ログ

  (py38_gan) python main.py --mode sample --num_domains 2 --resume_iter 100000 --w_hpf 1 --checkpoint_dir expr/checkpoints/celeba_hq --result_dir expr/results/celeba_hq --src_dir assets/representative/celeba_hq/src_0 --ref_dir assets/representative/celeba_hq/ref_0
  Namespace(batch_size=8, beta1=0.0, beta2=0.99, checkpoint_dir='expr/checkpoints/celeba_hq', ds_iter=100000, eval_dir='expr/eval', eval_every=50000, f_lr=1e-06, hidden_dim=512, img_size=256, inp_dir='assets/representative/custom/female', lambda_cyc=1, lambda_ds=1, lambda_reg=1, lambda_sty=1, latent_dim=16, lm_path='expr/checkpoints/celeba_lm_mean.npz', lr=0.0001, mode='sample', num_domains=2, num_outs_per_domain=10, num_workers=4, out_dir='assets/representative/celeba_hq/src/female', print_every=10, randcrop_prob=0.5, ref_dir='assets/representative/celeba_hq/ref_0', result_dir='expr/results/celeba_hq', resume_iter=100000, sample_dir='expr/samples', sample_every=5000, save_every=10000, seed=777, src_dir='assets/representative/celeba_hq/src_0', style_dim=64, total_iters=100000, train_img_dir='data/celeba_hq/train', val_batch_size=32, val_img_dir='data/celeba_hq/val', w_hpf=1.0, weight_decay=0.0001, wing_path='expr/checkpoints/wing.ckpt')
  Number of parameters of generator: 43467395
  Number of parameters of mapping_network: 2438272
  Number of parameters of style_encoder: 20916928
  Number of parameters of discriminator: 20852290
  Number of parameters of fan: 6333603
  Initializing generator...
  Initializing mapping_network...
  Initializing style_encoder...
  Initializing discriminator...
  Preparing DataLoader for the generation phase...
  Preparing DataLoader for the generation phase...
  Loading checkpoint from expr/checkpoints/celeba_hq\100000_nets_ema.ckpt...
  Working on expr/results/celeba_hq\reference.jpg...
  Working on expr/results/celeba_hq\video_ref.mp4...
  video_ref: 100%|█████████████████████████████████████████| 2/2 [00:00<?, ?it/s]
  Traceback (most recent call last):
    File "main.py", line 182, in <module>
      main(args)
    File "main.py", line 73, in main
      solver.sample(loaders)
    File "C:\Users\izuts\anaconda3\envs\py38_gan\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
      return func(*args, **kwargs)
    File "N:\anaconda_win\work\stargan-v2\core\solver.py", line 190, in sample
      utils.video_ref(nets_ema, args, src.x, ref.x, ref.y, fname)
    File "C:\Users\izuts\anaconda3\envs\py38_gan\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
      return func(*args, **kwargs)
    File "N:\anaconda_win\work\stargan-v2\core\utils.py", line 222, in video_ref
      video.append(frames[-1:])
  UnboundLocalError: local variable 'frames' referenced before assignment

• 対処法1：「stargan-v2/core/utils.py」→ 202行目 追加
  

  def video_ref(nets, args, x_src, x_ref, y_ref, fname):
      video = []
      frames = []

• 対処法2：「stargan-v2/core/solver.py」→ 18行目から 変更
  生成画像を保存した後、保管ビデオの作成時にエラーとなる（作成には 3つ以上の画像が必要）
  

          if len(ref.x) > 2:
              fname = ospj(args.result_dir, 'video_ref.mp4')
              print('Working on {}...'.format(fname))
              utils.video_ref(nets_ema, args, src.x, ref.x, ref.y, fname)

TypeError: gaussian() got an unexpected keyword argument 'multichannel'†

• エラー内容
  
  ▼　TypeError: ログ
  ▲　TypeError: ログ

  (py38_gan) python main.py --mode align --inp_dir assets/representative/custom_1/male --out_dir assets/representative/celeba_hq/ref_test/male
  Namespace(batch_size=8, beta1=0.0, beta2=0.99, checkpoint_dir='expr/checkpoints', ds_iter=100000, eval_dir='expr/eval', eval_every=50000, f_lr=1e-06, hidden_dim=512, img_size=256, inp_dir='assets/representative/custom_1/male', lambda_cyc=1, lambda_ds=1, lambda_reg=1, lambda_sty=1, latent_dim=16, lm_path='expr/checkpoints/celeba_lm_mean.npz', lr=0.0001, mode='align', num_domains=2, num_outs_per_domain=10, num_workers=4, out_dir='assets/representative/celeba_hq/ref_test/male', print_every=10, randcrop_prob=0.5, ref_dir='assets/representative/celeba_hq/ref', result_dir='expr/results', resume_iter=0, sample_dir='expr/samples', sample_every=5000, save_every=10000, seed=777, src_dir='assets/representative/celeba_hq/src', style_dim=64, total_iters=100000, train_img_dir='data/celeba_hq/train', val_batch_size=32, val_img_dir='data/celeba_hq/val', w_hpf=1, weight_decay=0.0001, wing_path='expr/checkpoints/wing.ckpt')
  Number of parameters of generator: 43467395
  Number of parameters of mapping_network: 2438272
  Number of parameters of style_encoder: 20916928
  Number of parameters of discriminator: 20852290
  Number of parameters of fan: 6333603
  Initializing generator...
  Initializing mapping_network...
  Initializing style_encoder...
  Initializing discriminator...
  Traceback (most recent call last):
    File "main.py", line 182, in <module>
      main(args)
    File "main.py", line 78, in main
      align_faces(args, args.inp_dir, args.out_dir)
    File "N:\anaconda_win\work\stargan-v2\core\wing.py", line 427, in align_faces
      x_aligned = aligner.align(x)
    File "N:\anaconda_win\work\stargan-v2\core\wing.py", line 305, in align
      img_np, landmarks = pad_mirror(img_np, landmarks)
    File "N:\anaconda_win\work\stargan-v2\core\wing.py", line 392, in pad_mirror
      small_blurred = gaussian(cv2.resize(img, (W, H)), H//100, multichannel=True)
  TypeError: gaussian() got an unexpected keyword argument 'multichannel'

• 参考サイト
  TypeError: rescale() got an unexpected keyword argument 'multichannel' #6
  scikit-image API reference
  

• 対処法：「stargan-v2/core/wing.py」→ 392行目 変更
  

  #    small_blurred = gaussian(cv2.resize(img, (W, H)), H//100, multichannel=True)
      small_blurred = gaussian(cv2.resize(img, (W, H)), H//100, channel_axis=1)

更新履歴†

• 2024/07/03 初版
  
   

参考資料†

• StarGAN
  
  • StarGAN v2 - Official PyTorch Implementation
    
  • StarGAN-v2 論文 (PDF)
    
  • StarGAN-v2で顔の特徴操作・顔の合成を実現するレシピ
    
  • StarGAN-v2で顔変換アプリを作ってみた
    
  • PyTorch StarGANでセレブの顔を変化させてみる
    
  • GANの潜在空間に新垣結衣は住んでいるのか？
    

• Python
  
  • [Python] assert文で超お手軽テスト
    
  • Pythonで例外を発生させる：raise
    

• Error
  

• Others
  
  • 画像変換の最新動向【2018-2020年】