Orden de MySQL por duplicados arriba primero

consulta,

SELECT  a.*
FROM    TableName a
        INNER JOIN 
        (
            SELECT  Color, COUNT(*) totalCount
            FROM    TableName
            GROUP   BY Color
        ) b ON  a.Color = b.Color
ORDER   BY b.TotalCount DESC, a.ID ASC

• Demostración de SQLFiddle

SALIDA

╔════╦════════╗
║ ID ║ COLOR  ║
╠════╬════════╣
║  2 ║ green  ║
║  4 ║ green  ║
║  5 ║ green  ║
║  1 ║ red    ║
║  6 ║ red    ║
║  3 ║ yellow ║
╚════╩════════╝