I am sure many of you have noticed the forums have on many occasions become remarkably non-responsive. The problem has been getting increasingly worse for some time. The cause: AI crawlers.
These are the web spiders that scour sites such as this one for every bit of content they can find to then train AI engines. Some are well-behaved and easily identified and blocked if appropriate, but many, many are not. They scan a site as rapidly as possible then restart the probing anew almost immediately. The server spends almost all its time repeating itself to these crawlers with few CPU cycles left to respond to the forum members.
South-east Asian countries (especially, China, Singapore, and Hong Kong) are homes for most of the malevolent AI crawlers. And with AI being the technology all the cool kids want to implement, the number of these malevolent crawlers is growing at an alarming rate. I did have some mechanisms in place to deal with the crawlers, but my methods were not able to handle the every growing load the crawlers.
Yesterday, I took a different approach to blocking Internet addresses and ranges of addresses for known AI crawlers. It seems to be working...for now at least...so the forum should be much ore responsive...for now at least.
Server performance issues
-
admin_jsfisher
- Site Admin
- Posts: 123
- Joined: Sun Jan 04, 2015 12:35 am
Re: Server performance issues
I think we might need Cloudflare.
-
admin_jsfisher
- Site Admin
- Posts: 123
- Joined: Sun Jan 04, 2015 12:35 am
Re: Server performance issues
In the recent past, server performancehas often been worse that abysmal.
As you may know, I have hosted this forum and Holowiki on my home internet connection for over 15 years. That has been a low-cost solution (to me, and no-cost to you) way of keeping this forum alive. But all things change with time. With the depletion of IPv4 addressing space and the exponential rise in AI bots scouring everything, home hosting is less and less viable.
I was nearing the point of throwing in the towel. I'd be willing to help anyone who wanted to take over the forum, but what I could to with my home cable modem connection was at its limit. However, for reasons completely independent of holography, I had a personal need to explore commercial shared hosting. Still all very basic, low cost stuff, actually stupid personal stuff, but just big enough that I can move the holography forum and the (grossly neglected wiki) to the same shared hosting environment.
Let me be very clear on this: I am not looking for, do not want, nor will I not accept anyone's financial contribution towards funding this commercial hosting or the domain name registrations. Sure, if you (hint, hint) wanted to "share" samples of any of your work, I'd accept, but I am not looking for money to run the forum In fact, in anyone wanted to take things over, I will help that person do it.
But I digress. Over the next few weeks or so, I will be moving Holowiki.org and Holographyforum.org to the shared hosting environment. The wiki migration will likely go unnoticed except for a brief period where the SSL certificate is invalid. The forums will likely suffer about the same fate, except the forum search function will be impacted. It looks like the search function will be less effective. (There is currently a background process that builds an elaborate index for full-text searching. I don't think that process will carry over into the new environment.) Searching will still work, just not as conveniently.
I will post more as things progress.
As you may know, I have hosted this forum and Holowiki on my home internet connection for over 15 years. That has been a low-cost solution (to me, and no-cost to you) way of keeping this forum alive. But all things change with time. With the depletion of IPv4 addressing space and the exponential rise in AI bots scouring everything, home hosting is less and less viable.
I was nearing the point of throwing in the towel. I'd be willing to help anyone who wanted to take over the forum, but what I could to with my home cable modem connection was at its limit. However, for reasons completely independent of holography, I had a personal need to explore commercial shared hosting. Still all very basic, low cost stuff, actually stupid personal stuff, but just big enough that I can move the holography forum and the (grossly neglected wiki) to the same shared hosting environment.
Let me be very clear on this: I am not looking for, do not want, nor will I not accept anyone's financial contribution towards funding this commercial hosting or the domain name registrations. Sure, if you (hint, hint) wanted to "share" samples of any of your work, I'd accept, but I am not looking for money to run the forum In fact, in anyone wanted to take things over, I will help that person do it.
But I digress. Over the next few weeks or so, I will be moving Holowiki.org and Holographyforum.org to the shared hosting environment. The wiki migration will likely go unnoticed except for a brief period where the SSL certificate is invalid. The forums will likely suffer about the same fate, except the forum search function will be impacted. It looks like the search function will be less effective. (There is currently a background process that builds an elaborate index for full-text searching. I don't think that process will carry over into the new environment.) Searching will still work, just not as conveniently.
I will post more as things progress.
-
Joe Farina
- Posts: 908
- Joined: Wed Jan 07, 2015 2:10 pm
Re: Server performance issues
Thank you for maintaining the forum for the last 15 years, and thanks for the update. If it turns out that a brave soul wants to take over the forum, it would be a great service to those of us still practicing holography. It would be a shame to lose such a wealth of information.
-
admin_jsfisher
- Site Admin
- Posts: 123
- Joined: Sun Jan 04, 2015 12:35 am
Re: Server performance issues
You are most welcome.Joe Farina wrote: ↑Sat Oct 24, 2026 12:47 pm Thank you for maintaining the forum for the last 15 years, and thanks for the update. If it turns out that a brave soul wants to take over the forum, it would be a great service to those of us still practicing holography. It would be a shame to lose such a wealth of information.
And just to be clear, I was not threatening to discontinue either the forum or the wiki. It was just that things had cascaded to the point where the service I was providing was complete garbage. But I now have an affordable solution that isn't garbage.
At some point I should probably post periodic backups to archive.org (aka "The Way-Back Machine"). That way, even if I don't find a successor before I leave this mortal coil, the content will still be somewhere.
Re: Server performance issues
Thank you for all your work!
The idea of seeing the monopolist (= FB) guarding all things holographic sends shivers down my spine. So how can we help? What about doing some fundraising?
Sounds like an excellent idea. If memory serves, in the early days of the holography forum (Colin's forum that is) one could download the forum posts in its entirety...
The idea of seeing the monopolist (= FB) guarding all things holographic sends shivers down my spine. So how can we help? What about doing some fundraising?
At some point I should probably post periodic backups to archive.org (aka "The Way-Back Machine"). That way, even if I don't find a successor before I leave this mortal coil, the content will still be somewhere.
Sounds like an excellent idea. If memory serves, in the early days of the holography forum (Colin's forum that is) one could download the forum posts in its entirety...