The initial set of sources come from your connected server. After that, you can get other sources from the clients you connect to via source exchange. Also, aMule will gradually poll the other servers in your server list.
So, if you limit the servers in your server list, it might have some slight impact on when you get certain sources. However, if you have source exchange enabled, you should eventually learn about all the sources.
It's almost certainly your ipfilter that is reducing the number of sources. Either it's too sweeping/aggressive, or those are sources you really didn't want anyway, because they are monitoring you or providing corrupt file data.