migman 0.9.8

Migman

Migman is a CLI tool for migrating and verifying MariaDB/MySQL tables using dump/load workflows and checksums.

Install (global tool)

dotnet tool install --global migman

Update

dotnet tool update --global migman

Usage

migman --help
migman run --dumper
migman run --loader
migman run --checksum
migman run --sync
migman init --docker
migman init --sample
migman init --docker --sample --force

Init options

• --docker: create docker-compose.yml for MariaDB/MySQL.
• --sample: create sample SQL scripts and connection helpers.

Configuration

migman.yml example:

source:
  type: mariadb
  host: localhost
  port: 3307
  user: root
  password: secret
  database: source_db
target:
  type: mysql
  host: localhost
  port: 3308
  user: root
  password: secret
  database: target_db
migration_mode: dump
dump_path: ./dumps
log_path: ./migman.log
log_level: info
command_timeout: 300
threads: 4
tables:
  - name: sample_pk_split
    target_name: sample_pk_split_copy
    compress: false
    drop_table: true
    chunking:
      type: pk_split
      key: id
      size: 20
  - name: sample_pk_offset
    target_name: sample_pk_offset_copy
    compress: false
    drop_table: true
    chunking:
      type: pk_offset
      keys:
        - order_id
        - line_id
      order_by:
        - column: order_id
          direction: asc
        - column: line_id
          direction: asc
      size: 20

migration_mode values: dump or sync (sync compares checksums and inserts/updates target rows and deletes target rows missing in source when mismatched). Use migman run --sync when migration_mode: sync. If the target table is missing, sync creates it from the source schema. tables[].target_name overrides the target table name (defaults to tables[].name). drop_table: true is only applied in loader mode; sync mode rejects it. command_timeout sets the MySQL command timeout in seconds (default: 300).

Configuration options

Top-level options:

• source: source database connection settings.
• target: target database connection settings.
• migration_mode: dump or sync. In sync, use migman run --sync.
• dump_path: directory for dump files.
• log_path: log file path (optional).
• log_level: error, warning, info, debug, trace (default: info).
• command_timeout: MySQL command timeout in seconds (default: 300).
• threads: number of worker threads (default: 4).
• tables: list of table configs.

source / target fields:

• type: mariadb or mysql.
• host: hostname or IP.
• port: integer port.
• user: username.
• password: password.
• database: database name.

tables[] fields:

• name: source table name (required).
• target_name: target table override (optional).
• compress: true to write .tsv.gz, false for .tsv.
• drop_table: true to drop target before load; ignored in sync (error if set).
• chunking: chunking config (required).

chunking for pk_split:

• type: pk_split (required).
• key: numeric PK column (required).
• size: rows per chunk (required).
• where: optional filter applied to range/chunk queries.
• order_by: optional override for sort direction (default: asc).

chunking for pk_offset:

• type: pk_offset (required).
• keys: list of PK columns (required; at least 2).
• size: rows per chunk (required).
• where: optional filter applied to chunk queries.
• order_by: optional list of key columns with direction (asc or desc).
  • Must include the same columns as keys in the same order.
  • Mixed directions are supported (e.g., sid desc, dt_create desc).

Chunking

Migman supports two chunking strategies for splitting large tables.

pk_split

Splits a table by a single numeric primary key range.

Required fields:

• chunking.type: pk_split
• chunking.key: primary key column
• chunking.size: rows per chunk Optional fields:
• chunking.where: filter condition applied to min/max range and chunk queries

Example:

chunking:
  type: pk_split
  key: id
  where: status = 'active'
  size: 20000

pk_offset

Splits a table by an ordered composite key (or single key) using keyset pagination.

Required fields:

• chunking.type: pk_offset
• chunking.keys: ordered list of key columns
• chunking.size: rows per chunk

Optional fields:

• chunking.order_by: must match chunking.keys order and only asc is supported

Example:

chunking:
  type: pk_offset
  keys:
    - order_id
    - line_id
  order_by:
    - column: order_id
      direction: asc
    - column: line_id
      direction: asc
  size: 20000

Logging

log_level supports: error, warning, info, debug, trace. command_timeout sets the MySQL command timeout in seconds (default: 300).