I ran into something similar recently while helping a friend set up a KC868v3, with one of the outputs connected to an LED and dropper resistor and the output set to off there was around 7.5V on the output and the output LED was lit even though it was set to off.
Looking at the circuit diagram, the p-channel MOSFET switches the voltage from LO0/LO1 onto the output Y1, Y2, etc, and also onto the output LED via a dropper resistor. If the output already has a voltage on it, in this case via another LED and a dropper resistor, it'll light the output LED even with the output off.
I don't know exactly what the setup was with the external LEDs so I'm reverse-engineering things based on the observed behaviour.
Looking at the circuit diagram, the p-channel MOSFET switches the voltage from LO0/LO1 onto the output Y1, Y2, etc, and also onto the output LED via a dropper resistor. If the output already has a voltage on it, in this case via another LED and a dropper resistor, it'll light the output LED even with the output off.
I don't know exactly what the setup was with the external LEDs so I'm reverse-engineering things based on the observed behaviour.