SPI throughput slower on mainline kernel

I first noticed this on my Orange Pi Zero. The latest Armbian releases (4.13.x) have some kind of gap between bytes when transmitting over SPI. I have an SPI performance test built into my SPI_LCD code:

https://github.com/bitbank2/SPI_LCD

If you run the demo program, it measures how many frames per second it can write to an SPI-connected LCD. On older Armbian builds with kernel version 3.x, it would usually get 27-33FPS on the H2/H3 boards. With the mainline kernel builds and running the same code, I'm seeing 9-17FPS (depending on the board). The H5 doesn't seem to be affected (I'm using an Orange Pi Zero +2 H5 for testing also). Anyone else seeing this? Any simple solution?

The people at FriendlyArm are also affected by this. I've been testing the NanoPi Duo using their 4.11 build and I get 3 FPS! The NanoPi Duo nightly build of Armbian gets 9 FPS on the same setup. I'm using SPI LCD displays for gaming and 30FPS gives a good experience, but 10 doesn't.

Update: The NanoPi NEO running Armbian kernel 4.13.12 shows 28.5 FPS (which is normal/good). So, why does the OPZ and NanoPi-Duo show such slow throughput when they have the same CPU?