The Montreal-based International Civil Aviation Organization (ICAO) has no regulatory power, and the EU has no authority over flights taking off and landing in Belarus or flying over its air space, apart from direct flights that originate or land in Europe.
Based on where Belavia (Belarus' national air carrier) flies, most of its destinations are EU members. The flights that pass over EU airspace without landing or originating in the EU are just its flights to Albania, Egypt, Montenegro, Serbia, Switzerland, and the UK. Also maybe Israel, and Lebanon. But the UK is just as outraged as the EU is about this, so it might follow the EU's lead if any ban happens. Switzerland might too. I'd also expect many of the countries currently applying for EU membership (Albania, Montenegro, and Serbia) might go along with a ban just because they have more to gain from kissing up to the EU than they have to lose for snubbing Belarus. Just an EU ban on takeoffs and landings from its territory could decimate Belarus' national air carrier.