Large Scale Inference with Theoretical Guarantees under Various Distributed Settings