I did some further testing. I added a load to the half bridge stage and saw that when no minimum load is present, the gate drivers will fail immediately. With a minimum load drawing at least 0.5 A, they don't fail anymore.
Just other things I do note. This board is just a prototype or rather a prototype of a prototype. In the past I've always used gate drive transformers or a regular gate driver IC like IR2110, this is just something new I wanted to try instead as it can simplify and shrink the circuit significantly. So yes, I agree that a 0.1uF cap is needed, but I don't think this is the primary cause for them failing. For final stage implementation - Yes, I will certainly add them, for a prototype - NO. I will explain this just now.
I tested the Vcc-GND voltage across each driver (High and Low Side) on the scope. There was no detectable ripple at all. If the omission of the 0.1uF cap or the electrolytic caps having a high ESR and inductance was the cause, then I would've seen some voltage spikes, however I saw zilch. Second reason is that the driver output stage always seemed to NOT be shorted across Vcc-to-GND but rather between Vout-to-Vcc or Vout-to-GND. This tells me there are spikes coming from the MOSFET gate causing the schmitt-trigger FETs to fail.
I measured the voltage across the 20-ohm resistor and I saw some very high spikes when I remove the load of the half bridge. This decreased when I increased the load. Peaks reached about 9-10V over the 20-ohm resistor, indicating about 0.5A peaks. The driver states that it can handle 4A peaks.
Last thing I noticed. I did a torture test on the two drivers. One is FOD3180 (2A peaks) and the other SI8261BBC (4A peaks). I found something very, very interesting. I added random caps in increasing numbers and sure enough, the FOD3180 could handle spikes of up to 2A (remember I'm talking about peak spikes of very, very short duration). Over that and it would fail. However, the SI8261BBC barely managed over 0.9A (peak) before it fails. I don't get it as to why since they're supposed to be stronger.
However something else I'm seeing is that When the low side FET is switching, it's highly detectable on the gate of Q1, causing about 200-250mA to be sink/sourced by its driver, which must not be the case. The opposite is also true. I haven't got the time yet to figure all of that out yet, I'll read through the whole PDF that Alec_t posted sometime tomorrow.
In the 1st image, this was measured on the PWM generator for Q1 (just to show when what switches and the dead time). Then the 2nd image shows the voltage over Q2. It doesn't matter whether its high or low side, they both look the same. I'm assuming this can cause me potential further problems. I'll probably look into it in the coming week.