Autosplit

LeanXcale is not only able to partition data based on the primary key, but also on other dimensions such as time. Typically, historical data is the one that can grow indefinitely, and historical data always has timestamps (or perhaps date plus time columns or an auto-increment column) associated with each row. So by setting auto-split based on the timestamp column, it becomes possible to keep each table fragment small enough to ingest data very efficiently. In this way, the speed of the ingest remains constant regardless of the size of the table, instead of getting slower and slower.

  public void createTable() throws SQLException {

    String SQL_CREATE_TABLE_HASH_AUTOSPLIT =
            "CREATE TABLE " + TABLE_NAME + " (" +
                    COLS +
                    "PRIMARY KEY (client_id,loan_id,ts)" +	//bidimensional field required as part of PK
                    ")" +
                    " PARTITION BY HASH (client_id)" +
                    " PARTITION BY DIMENSION ts EVERY 30000000 KEEP 300000000" +
                    " DISTRIBUTE BY HASH";

    utils.createTable(SQL_CREATE_TABLE_HASH_AUTOSPLIT);
  }