Dogma isn't his best film, but I don't think it is quite that bad. Just wait until you get to Jersey Girl. He did get some critical acclaim for the screenplay. Also, $10 million is hardly a big budget, and is just a bit above middle of the road for Smith (his average budget is probably more around $5 million). It is also something of a nod to the German film Wings Of Desire (remade in the US as City of Angels), at least in the first scenes featuring Loki and Bartelby.
I did not find the film to be incoherent at all. Two fallen angels are trying to use a loophole to defy the will of God, and re-enter heaven, thus disproving God's omnipotence if they succeed. The dialogue is not quite as genuine as usual for Smith (at least for his Askewniverse films), but I think that owes to the subject matter and plot.
The angels motivations made no sense. Loki seemed to do everything he did because he was a cunt/bored with life. Didn't get Bartleby at all. Seemed to be all over the map. They're angels... I have no way of relating. And Smith didn't make them relatable IMHO. The bit about them becoming human when they lost their wings... dumb as fuck. They could have gone the existential route and tried to explore it. Nope. Just a thing that happened for no reason and that had zero impact on the story. Bartleby killing Loki.. just dumb. No tension. Zero stakes. It just happened.
Ok, so the girl was selected to prevent this because she's the great great.... great grand niece of Jesus. Ok, fine. They just failed to explain why this is relevant.
Also, Jay and Silent Bob are the profets. Ok, fine. But why and how? They failed to explain how being a profet works and what their role is in all this. They actually talk about it in the film. But no resolution. Never answered.
WTF is Rufus doing there? Why did he exist and why is he in the film. Same goes for Salma Hayek's character. Actually... same for all the characters bar Satan. But Satan's kids.. omg that was lame. How was the homeless person relevant in that film? How did the main character figure out she had to kill the homeless guy, and why did this kill her? And God being omnipotent just ruins all of the above even more. God is choosing not to fix this. Ok, why? The explanation in the film is that God goes AWOL sometimes. Ok.. but that's not how omnipotence works. I can drive a jumbo jet through that plot hole.
I thought the entire film was just a list of people doing random things resulting in random and unforseen consequences that ultimately don't matter. And in the end God anyway just shows up and fixes everything. So the entire film was a waste of time. This is the equivalent of a main character waking up in the end and says "it was only a dream". I hate that. At no point in the film did I give a shit about what would happen next. I was bored from start to finish.