The WisBlock voltage regulator is designed for a minimum input voltage of 3.7V.
Using a battery with a lower voltage can work (down to 3.3V), but when I tried it (not a real deep test), I got an excessive current consumption.
You can try to connect the battery to the solar connector, which leaves your USB port available for programming and debugging. But still at voltage below 3.7V you might experience higher consumption.