feat: add high-level DispatchFunc() interface for multi-type and mixed dispatch

[Ziminli]

TL;DR: Add a high-level DispatchFunc() interface for multi-type and mixed dispatch, and use DispatchFunc() to dispatch DataType and block size at the same time in various implementations.

Key Changes

• High-Level Dispatcher Interface:

  • Add a high-level DispatchFunc() interface for multi-type and mixed dispatch;
  • Any int64_t-convertible types can use this interface to simplify the dispatch usage.

• Simplify Block Size Dispatching

  • Update the code in various CUDA implementations to use DispatchFunc() to dispatch DataType and block size at the same time.

• Remove Unnecessary Macros

  • Remove the CUDA_BLOCK_SIZE_XXX macros/aliases and simply use numbers instead.

• Style Correction

  • Fix a naming issue.

Known Issues & Future Work:

• Spot some issues that does not comply with the design and these have been marked with TODO

[Ziminli]

NVIDIA

MetaX

Processing /root/InfiniOps
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pytest in /opt/conda/lib/python3.10/site-packages (from InfiniOps==0.1.0) (8.4.1)
Requirement already satisfied: pytest-cov in /opt/conda/lib/python3.10/site-packages (from InfiniOps==0.1.0) (7.0.0)
Requirement already satisfied: pytest-xdist in /opt/conda/lib/python3.10/site-packages (from InfiniOps==0.1.0) (3.8.0)
Requirement already satisfied: ruff in /opt/conda/lib/python3.10/site-packages (from InfiniOps==0.1.0) (0.15.7)
Requirement already satisfied: torch in /opt/conda/lib/python3.10/site-packages (from InfiniOps==0.1.0) (2.4.0+metax3.2.1.3)
Requirement already satisfied: exceptiongroup>=1 in /opt/conda/lib/python3.10/site-packages (from pytest->InfiniOps==0.1.0) (1.3.0)
Requirement already satisfied: iniconfig>=1 in /opt/conda/lib/python3.10/site-packages (from pytest->InfiniOps==0.1.0) (2.1.0)
Requirement already satisfied: packaging>=20 in /opt/conda/lib/python3.10/site-packages (from pytest->InfiniOps==0.1.0) (25.0)
Requirement already satisfied: pluggy<2,>=1.5 in /opt/conda/lib/python3.10/site-packages (from pytest->InfiniOps==0.1.0) (1.6.0)
Requirement already satisfied: pygments>=2.7.2 in /opt/conda/lib/python3.10/site-packages (from pytest->InfiniOps==0.1.0) (2.19.2)
Requirement already satisfied: tomli>=1 in /opt/conda/lib/python3.10/site-packages (from pytest->InfiniOps==0.1.0) (2.3.0)
Requirement already satisfied: coverage>=7.10.6 in /opt/conda/lib/python3.10/site-packages (from coverage[toml]>=7.10.6->pytest-cov->InfiniOps==0.1.0) (7.11.0)
Requirement already satisfied: execnet>=2.1 in /opt/conda/lib/python3.10/site-packages (from pytest-xdist->InfiniOps==0.1.0) (2.1.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch->InfiniOps==0.1.0) (3.20.0)
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/lib/python3.10/site-packages (from torch->InfiniOps==0.1.0) (4.15.0)
Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch->InfiniOps==0.1.0) (1.14.0)
Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch->InfiniOps==0.1.0) (3.4.2)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch->InfiniOps==0.1.0) (3.1.6)
Requirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from torch->InfiniOps==0.1.0) (2025.5.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch->InfiniOps==0.1.0) (3.0.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/lib/python3.10/site-packages (from sympy->torch->InfiniOps==0.1.0) (1.3.0)
Building wheels for collected packages: InfiniOps
  Building wheel for InfiniOps (pyproject.toml) ... done
  Created wheel for InfiniOps: filename=infiniops-0.1.0-cp310-cp310-linux_x86_64.whl size=582095 sha256=55811192803802b57b0a76e9cf5b50628c8535111482d007b18943ff1598a427
  Stored in directory: /tmp/pip-ephem-wheel-cache-ys36zi23/wheels/84/73/f5/9b7b2aa420f23ecb29296accd9546f83a60d50799cf5250af8
Successfully built InfiniOps
Installing collected packages: InfiniOps
  Attempting uninstall: InfiniOps
    Found existing installation: InfiniOps 0.1.0
    Uninstalling InfiniOps-0.1.0:
      Successfully uninstalled InfiniOps-0.1.0
Successfully installed InfiniOps-0.1.0

Iluvatar

Moore

Cambricon