Just as you should never fast charge the units, you also should never over discharge them. To be on the safe side, wait until they show the low battery signal. Don’t wait until the charge is completely gone as it will destroy the battery.
You should be cautious of how you use the batteries. As rule of thumb, you should never mix different batteries. The best way of going about it is using batteries of the same capacity, brand, and chemistry. You also should never mix rechargeable and non-rechargeable batteries as they have different voltages. This results in them damaging each other.