MySQL orden por cadena con números

Puede usar SUBSTR y CAST AS UNSIGNED/SIGNED dentro de ORDER BY:

SELECT * FROM table_name ORDER BY
    SUBSTR(col_name FROM 1 FOR 1),
    CAST(SUBSTR(col_name FROM 2) AS UNSIGNED)