Overview

Upgrade handlers enable coordinated on-chain upgrades across all validators at specific block heights via governance proposals. They provide a mechanism for chains to perform data migrations, parameter updates, and module upgrades in a deterministic way.

When to Use Upgrade Handlers

Upgrade handlers are required when:
  • Breaking state changes: Modifying storage formats or data structures
  • Module migrations: Updating module versions or parameters
  • Protocol upgrades: Implementing new features that require state transitions
  • Data migrations: Moving data between different storage locations

Basic Structure

Registering Upgrade Handlers

Upgrade handlers are registered in your app’s RegisterUpgradeHandlers() method:
app/upgrades.go
package app

import (
    "context"

    upgradetypes "cosmossdk.io/x/upgrade/types"
    sdk "github.com/cosmos/cosmos-sdk/types"
    "github.com/cosmos/cosmos-sdk/types/module"
)

func (app *App) RegisterUpgradeHandlers() {
    app.UpgradeKeeper.SetUpgradeHandler(
        "v1.0.0", // upgrade name (must match governance proposal)
        func(ctx context.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {
            sdkCtx := sdk.UnwrapSDKContext(ctx)
            sdkCtx.Logger().Info("Starting upgrade", "name", plan.Name)

            // Run module migrations
            migrations, err := app.ModuleManager.RunMigrations(ctx, app.configurator, fromVM)
            if err != nil {
                return nil, err
            }

            // Add custom migration logic here

            sdkCtx.Logger().Info("Upgrade complete", "name", plan.Name)
            return migrations, nil
        },
    )
}

Module Version Management

The upgrade handler receives and returns a module.VersionMap that tracks module versions:
// fromVM contains the module versions before the upgrade
// The returned VersionMap contains the new versions after migration
migrations, err := app.ModuleManager.RunMigrations(ctx, app.configurator, fromVM)

Organizing Upgrade Code

For better maintainability, organize upgrades in separate packages:

    Common Migration Patterns

    Parameter Migrations

    Migrating module parameters to new formats:
    func migrateParams(ctx sdk.Context, keeper paramskeeper.Keeper) error {
        // Get old params
        var oldParams v1.Params
        keeper.GetParamSet(ctx, &oldParams)
    
        // Convert to new format
        newParams := v2.Params{
            Field1: oldParams.Field1,
            Field2: convertField(oldParams.Field2),
            // New field with default value
            Field3: "default",
        }
    
        // Set new params
        keeper.SetParams(ctx, newParams)
        return nil
    }
    

    State Migrations

    Moving data between different storage locations:
    func migrateState(ctx sdk.Context, storeKey storetypes.StoreKey) error {
        store := ctx.KVStore(storeKey)
    
        // Iterate over old storage
        iterator := storetypes.KVStorePrefixIterator(store, oldPrefix)
        defer iterator.Close()
    
        for ; iterator.Valid(); iterator.Next() {
            oldKey := iterator.Key()
            value := iterator.Value()
    
            // Transform key/value if needed
            newKey := transformKey(oldKey)
            newValue := transformValue(value)
    
            // Write to new location
            store.Set(newKey, newValue)
    
            // Delete old entry
            store.Delete(oldKey)
        }
    
        return nil
    }
    

    Module Addition/Removal

    Adding or removing modules during upgrade:
    // In constants.go
    var Upgrade = upgrades.Upgrade{
        UpgradeName: "v2.0.0",
        CreateUpgradeHandler: CreateUpgradeHandler,
        StoreUpgrades: storetypes.StoreUpgrades{
            Added:   []string{"newmodule"},
            Deleted: []string{"oldmodule"},
        },
    }
    
    // In handler.go
    func CreateUpgradeHandler(...) upgradetypes.UpgradeHandler {
        return func(c context.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) {
            // Delete old module version
            delete(vm, "oldmodule")
    
            // Initialize new module
            if err := newModuleKeeper.InitGenesis(ctx, defaultGenesis); err != nil {
                return nil, err
            }
    
            // Run migrations
            return mm.RunMigrations(c, configurator, vm)
        }
    }
    

    Best Practices

    1. Idempotency

    Make migrations idempotent when possible:
    func migrateSomething(ctx sdk.Context, store sdk.KVStore) error {
        // Check if migration already done
        if store.Has(migrationCompleteKey) {
            ctx.Logger().Info("Migration already completed, skipping")
            return nil
        }
    
        // Perform migration
        // ...
    
        // Mark as complete
        store.Set(migrationCompleteKey, []byte{1})
        return nil
    }
    

    2. Error Handling

    Use comprehensive error handling and logging:
    func migrate(ctx sdk.Context, keeper Keeper) error {
        ctx.Logger().Info("Starting migration", "module", "mymodule")
    
        count := 0
        iterator := keeper.IterateAllRecords(ctx)
        defer iterator.Close()
    
        for ; iterator.Valid(); iterator.Next() {
            if err := processRecord(iterator.Key(), iterator.Value()); err != nil {
                ctx.Logger().Error("Failed to migrate record",
                    "key", iterator.Key(),
                    "error", err,
                )
                return fmt.Errorf("migration failed at record %d: %w", count, err)
            }
            count++
    
            // Log progress for long migrations
            if count%1000 == 0 {
                ctx.Logger().Info("Migration progress", "processed", count)
            }
        }
    
        ctx.Logger().Info("Migration complete", "total_migrated", count)
        return nil
    }
    

    3. Testing

    Create comprehensive tests for upgrade handlers:
    func TestUpgradeHandler(t *testing.T) {
        app := setupApp(t)
        ctx := app.NewContext(false, tmproto.Header{Height: 1})
    
        // Setup pre-upgrade state
        setupOldState(t, ctx, app)
    
        // Run upgrade handler
        _, err := v1_0_0.CreateUpgradeHandler(
            app.ModuleManager,
            app.configurator,
            &upgrades.UpgradeKeepers{
                // ... keepers
            },
            app.keys,
        )(ctx, upgradetypes.Plan{Name: "v1.0.0"}, app.ModuleManager.GetVersionMap())
    
        require.NoError(t, err)
    
        // Verify post-upgrade state
        verifyNewState(t, ctx, app)
    }
    

    Upgrade Process

    1

    Create Upgrade Proposal
    2

    Submit a governance proposal with the upgrade details:
    3

    mantrachaind tx gov submit-proposal software-upgrade v1.0.0 \
        --title "Upgrade to v1.0.0" \
        --description "Upgrade description" \
        --upgrade-height 1000000 \
        --from validator \
        --deposit 10000000stake
    
    4

    Vote on Proposal
    5

    Validators and delegators vote on the upgrade:
    6

    mantrachaind tx gov vote 1 yes --from validator
    
    7

    Prepare Binary
    8

    Build and distribute the new binary with the upgrade handler:
    9

    # Build new binary
    make build
    
    # Test upgrade on local network
    ./scripts/test-upgrade.sh
    
    # Distribute to validators
    # Use Cosmovisor for automated upgrades
    
    10

    Monitor Upgrade
    11

    Watch logs during the upgrade height:
    12

    # Monitor upgrade logs
    tail -f ~/.mantrachaind/logs/upgrade.log
    
    # Verify upgrade success
    mantrachaind query upgrade applied v1.0.0
    

    Cosmos EVM Specific Migrations

    For Cosmos EVM chains, common migrations include:
    • ERC20 Precompiles Migration: Critical for v0.3.x to v0.4.0
    • Fee Market Parameters: Updating EIP-1559 parameters
    • Custom Precompiles: Registering new precompiled contracts
    • EVM State: Migrating account balances or contract storage

    Troubleshooting

      References