In my (newly gotten!) experience, non-fiction is the best source of inspiration. I love fantasy and read it probably twice as often as I do non-fiction, but any specific ideas that I have from that source are just that, derivative of the source. In running a game based on existing fantasy, I either have to be concerned with matching the setting as accurately as I can or departing from it to create something original.
Non-fiction (history, economics, politics, anthropology), on the other hand, has limitless space in which to insert fantasy. I can change anything I want because--hey!--it's fantasy, and I can borrow whatever I like without guilt. Non-fiction provides inspiration whereas fiction limits inspiration because the work is already done for me.
No doubt this is a personal preference and reaction, but has anyone else had this same experience?