Seguro que puedes hacer eso:
DELETE FROM `table1`, `table2` WHERE `orderId` = 500
consulte http://dev.mysql.com/doc/refman/ 5.0/en/delete.html
[EDITAR:]
Este es todo el truco:
DELETE FROM `orders`, `orders_items`
USING `orders`
INNER JOIN `orders_items` ON `orders`.`orderId` = `orders_items`.`orderId`
WHERE `orders`.`orderId`= 500
Si orderId es un varchar, cambie la instrucción a = '500'
.