Having a big Hall where speakers are up to than 33m from each other the time difference is 100ms for them. But normally if you think about an orchestra which has 10m distribution of musicians the time between the musicians is about 33ms.
So I think having a latency of 30ms should do.