Dear Florihana,
If you want to automatically subtract an amount (10% of the order amount in your case), you can use the ExtraShippingCharge function:
ExtraShippingCharge = -0.1*Amount
This will subtract 10% of the total product prices from all subsequent rules (of this method only; shipping costs from other shipping methods are not affected!).
However, this will be added (or with negative sign subtracted) in all cases, even if the total shipping costs will turn negative. So implementing the floor with 5 is not possible.
To include the $5 floor on shipping costs, you'll have to manually include it in ALL rules:
Variable=ShipDiscount; Value = 0.1*Amount
Weight<10; Shipping=max(5, 12-ShipDiscount)
Weight<20; Shipping=max(5, 123-ShipDiscount)
Weight<30; Shipping=max(5, 1234-ShipDicscount)
Best regards,
Reinholdd