Dataset generation

Script to generate the dataset for the project.

source

generate_dataset

 generate_dataset (cfg:omegaconf.dictconfig.DictConfig)

source

generate_run

 generate_run (rng:numpy.random._generator.Generator, solver, generator:ph
               ysmodjax.solver.generator.Generator=<physmodjax.solver.gene
               rator.Gaussian object at 0x107f165c0>, **ic_params)

Test

from hydra import initialize, compose

with initialize(version_base=None, config_path="../../conf"):
    cfg = compose(
        config_name="generate_data",
        return_hydra_config=True,
        overrides=[
            "+dataset=test2d",
            "solver=tension_modulated_2d",
            "solver.use_nonlinear=True",
            "ic_generator=noise2d",
            "number_ics=3",
            "solver.final_time=0.001",
            "+ic_params.ic_max_amplitude=0.01",
        ],
    )
    hydra.core.hydra_config.HydraConfig.instance().set_config(cfg)
    generate_dataset(cfg)
hydra:
  run:
    dir: ${output_dir}
  sweep:
    dir: multirun/${now:%Y-%m-%d}/${now:%H-%M-%S}
    subdir: ${hydra.job.num}
  launcher:
    _target_: hydra._internal.core_plugins.basic_launcher.BasicLauncher
  sweeper:
    _target_: hydra._internal.core_plugins.basic_sweeper.BasicSweeper
    max_batch_size: null
    params: null
  help:
    app_name: ${hydra.job.name}
    header: '${hydra.help.app_name} is powered by Hydra.

      '
    footer: 'Powered by Hydra (https://hydra.cc)

      Use --hydra-help to view Hydra specific help

      '
    template: '${hydra.help.header}

      == Configuration groups ==

      Compose your configuration from those groups (group=option)


      $APP_CONFIG_GROUPS


      == Config ==

      Override anything in the config (foo.bar=value)


      $CONFIG


      ${hydra.help.footer}

      '
  hydra_help:
    template: 'Hydra (${hydra.runtime.version})

      See https://hydra.cc for more info.


      == Flags ==

      $FLAGS_HELP


      == Configuration groups ==

      Compose your configuration from those groups (For example, append hydra/job_logging=disabled
      to command line)


      $HYDRA_CONFIG_GROUPS


      Use ''--cfg hydra'' to Show the Hydra config.

      '
    hydra_help: ???
  hydra_logging:
    version: 1
    formatters:
      simple:
        format: '[%(asctime)s][HYDRA] %(message)s'
    handlers:
      console:
        class: logging.StreamHandler
        formatter: simple
        stream: ext://sys.stdout
    root:
      level: INFO
      handlers:
      - console
    loggers:
      logging_example:
        level: DEBUG
    disable_existing_loggers: false
  job_logging:
    version: 1
    formatters:
      simple:
        format: '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'
    handlers:
      console:
        class: logging.StreamHandler
        formatter: simple
        stream: ext://sys.stdout
      file:
        class: logging.FileHandler
        formatter: simple
        filename: ${hydra.runtime.output_dir}/${hydra.job.name}.log
    root:
      level: INFO
      handlers:
      - console
      - file
    disable_existing_loggers: false
  env: {}
  mode: null
  searchpath: []
  callbacks: {}
  output_subdir: .hydra
  overrides:
    hydra: []
    task:
    - +dataset=test2d
    - solver=tension_modulated_2d
    - solver.use_nonlinear=True
    - ic_generator=noise2d
    - number_ics=3
    - solver.final_time=0.001
    - +ic_params.ic_max_amplitude=0.01
  job:
    name: notebook
    chdir: true
    override_dirname: +dataset=test2d,+ic_params.ic_max_amplitude=0.01,ic_generator=noise2d,number_ics=3,solver.final_time=0.001,solver.use_nonlinear=True,solver=tension_modulated_2d
    id: ???
    num: ???
    config_name: generate_data
    env_set: {}
    env_copy: []
    config:
      override_dirname:
        kv_sep: '='
        item_sep: ','
        exclude_keys: []
  runtime:
    version: 1.3.2
    version_base: '1.3'
    cwd: /home/carlos/projects/physmodjax/nbs/scripts
    config_sources:
    - path: hydra.conf
      schema: pkg
      provider: hydra
    - path: /home/carlos/projects/physmodjax/conf
      schema: file
      provider: main
    - path: ''
      schema: structured
      provider: schema
    output_dir: ???
    choices:
      dataset: test2d
      ic_generator: noise2d
      solver: tension_modulated_2d
      hydra/env: default
      hydra/callbacks: null
      hydra/job_logging: default
      hydra/hydra_logging: default
      hydra/hydra_help: default
      hydra/help: default
      hydra/sweeper: basic
      hydra/launcher: basic
      hydra/output: default
  verbose: false
solver:
  _target_: physmodjax.solver.wave2d_tenmod.Wave2dSolverTensionModulated
  sampling_rate: 16000
  final_time: 0.001
  n_gridpoints_x: 41
  length_x: 0.4
  aspect_ratio: 0.9
  rho: 1380
  h: 0.00019
  E: 3500000000.0
  nu: 0.3
  d1: 8.0e-05
  d3: 1.4e-05
  Ts0: 2620
  n_max_modes: 225
  use_nonlinear: true
ic_generator:
  _target_: physmodjax.solver.generator.Noise2d
  num_points_x: ${solver.n_gridpoints_x}
  aspect_ratio: ${solver.aspect_ratio}
name: test2d
output_dir: data/${name}
number_ics: 3
seed: 3407
ic_params:
  ic_max_amplitude: 0.01
  0%|          | 0/3 [00:00<?, ?it/s] 33%|███▎      | 1/3 [00:00<00:01,  1.07it/s, Saved file=ic_00001.npy]
bar_u shape: (225, 16)
bar_v shape: (225, 16)
 67%|██████▋   | 2/3 [00:01<00:00,  1.09it/s, Saved file=ic_00002.npy]
bar_u shape: (225, 16)
bar_v shape: (225, 16)
100%|██████████| 3/3 [00:02<00:00,  1.06it/s, Saved file=ic_00003.npy]
bar_u shape: (225, 16)
bar_v shape: (225, 16)

Convert to single file


source

convert_to_single_file

 convert_to_single_file (data_dir:str, output_file:str,
                         target_dtype:str='np.float32')
Type Default Details
data_dir str the directory where the files are
output_file str the output file
target_dtype str np.float32 the dtype of the output file