(largely a note to self)
https://wiki.ubuntu.com/KernelTeam/Specs/BootPerformance#Kernel%20Config%20Changes - points out that depmod+modprobe does a lot more than it needs to. It shouldn't be too hard to make a binary map to easily jump from a PCI ID to a module.
I need to look at oprofile and other kernel profiling stuff.
Also, research using qemu(+kvm) - at the very least it's a nice sandbox. In addition you can do a lot of traces pretty easily in an emulator...