The main problem - distance between client and server.
My server located in Russia, Moscow. Some usa people having same issues (Luigifan2010 from Florida), because too huge distance: trough ocean. Europan people (Poland, Germany, France, Italy, Great Britain, etc.) Haven't issues with speed because they are near to me. My local speed (in Moscow) is 60 Mbps, but from Florida same connection will be 1-5 Mbps. Therefore I made mirror on Google Drive to host lab builds and config packs to solve download problem for North/South America, Australia and Asia.
I have bit slow speed with SMBX site, for example, because same distance problem.
You can try to use europan proxies/anonymisers like
http://cameleo.ru to take speed-up of loading of my pages.
Added after 1 hour 23 minutes:P.S. I started testing of CDN servise
https://www.cloudflare.com which can give some boost of site loading for USA and other people, also it have free tariff plan which is a good thing.