Time  Nick             Message
05:05 thd              dcook: What does one need to do for bugzilla to ban an account?
05:17 dcook            Have too many URLs in a comment I think
05:17 dcook            Possibly also too many comments too quickly
05:17 dcook            But the URL thing seems to be the key factor
05:58 reiveune         hello
06:11 magnuse          \o/
06:53 cait             good morning #koha!
07:07 matts            hi koha!
07:11 matterbridge     [mattermost] <paulderscheid> morning #koha
07:11 cait             paulderscheid[m]: did I miss the instructions on how to get on mattermost?
07:11 cait             bonjour matts :)
07:12 cait             Roles page is up: https://wiki.koha-community.org/wiki/Roles_for_24.11
07:21 paulderscheid[m] seems so cait
07:21 paulderscheid[m] :D
07:21 matterbridge     [mattermost] <aulda> We're just in Beta at the moment...
07:22 ashimema         it's close.. very close πŸ˜‰
07:22 ashimema         we're just awaiting dns stuff to migrate through... ssl etc
07:22 paulderscheid[m] I love it
07:22 matterbridge     [mattermost] <aulda> Should have the correct URL sorted today so that we can launch more formally
07:24 matterbridge     [mattermost] <aulda> Also, I saw the call for the community newsletter so I'll draft a piece for that, and for the mailing lists giving people heads up about Mattermost.
07:24 matterbridge     [mattermost] <paulderscheid> That's a very good idea!
07:25 cait             aulda: not sure if you can see that, but sounds good, just let me know when we are supposed to sign up
07:25 cait             I think the URL shoudl be sorted from what I saw (email between rangi and tcohen)
07:25 matterbridge     [mattermost] <paulderscheid> Yep, the bridge is bi-directional :)
07:25 matterbridge     [mattermost] <aulda> I see you @cait
07:25 matterbridge     [mattermost] <aulda> πŸ˜‰
07:41 cait             aulda: matterbridge on the agenda for Wednesday? :) I just added some other topics as well
07:41 cait             https://wiki.koha-community.org/wiki/Development_IRC_meeting_17_April_2024
07:45 paulderscheid[m] Beta is over :D
08:06 ashimema         Yeay.. address is now live proper by the looks
08:06 ashimema         chat.koha-community.org
08:17 ashimema         interesting
08:19 ashimema         did the bridge stop already
08:28 magnuse          oh noes
08:34 magnuse          signing up for mattermost was pretty smooth, at least
08:44 matterbridge     [mattermost] <PerplexedTheta> The bridge went down because I didn't update the config with the new URL - it should be back up now
08:44 matterbridge     [mattermost] <PerplexedTheta> chat.koha-community.org is live!
08:52 Annelisterman[m] What is this mattermost? Are we supposed to join it instead of this IRC?
08:53 matterbridge     [mattermost] <aulda> At Hackfest we discussed a new way of enabling community communication and chat, and settled on Mattermost as the best option for us.
08:54 matterbridge     [mattermost] <aulda> We have been setting up a Mattermost server (hosted by PTFS Europe), and testing the functionality in the last few days.
08:54 thd              Annelisterman[m]: I believe that there is intended to be a bridge, there it is.
08:54 matterbridge     [mattermost] <aulda> I am going to pop an announcement in the Koha Community newsletter and the maillists in the coming days.
08:55 matterbridge     [mattermost] <aulda> Yes, the bridge between Mattermost and IRC enables those on Mattermost to see the IRC chat and vice-versa.
08:56 Annelisterman[m] Ok. Is this IRC going to stay active though? Or maybe you are explaining it all in the newsletter. :)
08:58 thd              Annelisterman [m]: IRC will be active as long as people want it to be by participating there and to whatever extent the chat/IRC bridge works.
08:58 matterbridge     [mattermost] <aulda> I will explain the discussion that we had at Hackfest in the newsletter, yes. The intention, for now, is that IRC stays active. It will then depend on how active it is in the long term I would think, and how inclusive Mattermost proves to be.
09:00 Annelisterman[m] Ok. We'll have to look into it. We are now using IRC with Matrix/Element at the moment.
09:00 Annelisterman[m] s/now//
09:03 matterbridge     [mattermost] <PerplexedTheta> Now enabled IRC Bridge as an auto-joined channel, so nobody misses out on the fun
09:24 cait             Annelisterman[m]: ultimately the idea will be to use a newer tool, that also allows channels etc. and gather the community there
09:25 cait             but we are testing for now, IRC is not going away, but it will not be able to offer the same functionality, for example people posting pictures, different channels, settng your own status as a new solution will provide
10:11 tcohen           cait wb to work :-D
10:11 tcohen           bug 36593 for you
10:11 huginn`          04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36593 enhancement, P5 - low, ---, tomascohen, Needs Signoff , Add support for the `time` column type on TestBuilder
10:23 matterbridge     [mattermost] <cait> tcohen++ thank you! Will check a little later today, still working on the backlog of mails
10:23 matterbridge     [mattermost] <cait> but RM time is planned for too :)
10:23 thd              tcohen: I thought that you may have been in France.
10:25 cait             we have been :)
10:26 thd              cait: I saw tcohen on IRC at an earlier than usual hour.
10:31 tcohen           hi thd
10:31 tcohen           jetlag
10:31 tcohen           I woke at 5
10:34 cait             ouch
10:40 thd              I have been up but my sleep schedule was somewhat rearranged as struggled to find my typo.  I left a letter out of my username and never noticed while rechecking my password.
10:42 thd              tcohen: When would you have have time later today or tomorrow to pre-setup docker-mailserver?
10:44 thd              tcohen: Fixing  that: Would you have time later today or tomorrow to pre-setup docker-mailserver?
10:56 oleonard         Hi #koha
10:56 oleonard         Sorry to any who are back at work already with a Hackfest Hangover
10:56 thd              tcohen: I am writing rangi about updating DNS now.  He will probably be asleep.
10:59 tcohen           thd I don't think I'll have time today or tomorrow, catching up with jetlag and a pile of emails
10:59 thd              tcohen: Were you also in France?
11:00 tcohen           can you drop me an email about what you need to do, and I try to think about it in between things?
11:00 tcohen           thd yeah
11:01 thd              Something like what we did to add the wiki container to portainer.  In this case, there is no database to worry about.
11:01 tcohen           so you want to set a postfix server on a container?
11:02 thd              It is a little more than Postfix but Postfix is essentially all we would be using.
11:03 thd              I have been testing the configs and I have a very good configuration after breaking things a few times.
11:04 thd              Dovecot configuration can be improved but we only presently use Dovecot for authentication and transfering bounces somewhere else as needed.
11:07 thd              tcohen: The big accomplishment was a clever trick a manged to get working where some addresses could send mail directly to any server without being blocked by the necessary configuration of relay functions.
11:08 thd              Everything is sender dependent so that you can have different functions for different senders without one function clobbering everything.
11:09 thd              ... and it delivers to Gmail, etc. mailboxes via secure-mailgate.com passing all tests.
11:21 tcohen           thd I like where you're headed to, but in Marseille there was a comment
11:21 tcohen           about finding a better SMTP provider that does DKIM
11:22 tcohen           the one HKS3 had found was not good enough, and they are committed to finding another one
11:23 thd              Yes, I know that option is being explored, but that will add more proprietary services which a FOSS project should not depend upon especially when these services are prone to suspend and then cancel accounts over temporary rises in bounced mail.
11:24 thd              Is HKS3 themselves not satisfied with secure-mailgate.com?
11:24 tcohen           got it
11:24 tcohen           they only do DKIM if you handle your domain with them I think
11:25 tcohen           that's what I understood
11:28 thd              Discourse officially recommends these proprietary SMTP services as the easy way to do email but that is because many adopters find setting up a mailserver too challenging.  However, every one of these services has unhappy users who briefly had too many spammers trying to register Discourse accounts with someone else's email address etc. which then produces bounces as one way of getting too many bounces.
11:31 thd              Mail relays are easy.  The tricky thing is configuring so that everything can work and some senders on the system can bypass the relay configuration.
11:32 thd              Or if service has been suspended because some problem is unavoidable a minor change in configuration and restarting the container has delivery working again without a new proprietary service.
11:32 tcohen           thd, conceptually I understand all you're talking about.
11:33 tcohen           Please let me know on an email (so it doesn't get lost for me) how I can help you
11:33 tcohen           and I'll start thinking about it in background and come back to you once my brain is back to a normal-ish place
11:33 tcohen           timezona
11:35 thd              The first email you will have is where you and paulderscheid[m] will be on the CC in the message to rangi about DNS changes for anything to work including any new proprietary system.
11:38 thd              Also, if you have enough brain power at the moment, I am not certain where to ask rangi to direct DNS for an additional testing instance of docker-mailserver.  An additional instance we can break without disrupting things where port 25 is not on the same IP address.
11:43 thd              tcohen: Aside from services such http and https I think Traefik has trouble managing things other than DNS certificates.  Email protocols are sometimes a problem for Traefik if Traefik is put in the middle for multiple mail services on the same port.
12:58 tcohen           thd I never used traefik like that. I know it does SNI and HostSNI can be used to match things, but I never used it really
13:17 thd              tcohen: Assuming that Traefik may be a problem when acting as a proxy for email protocols but can be bypassed for mail server ports.  Is there an additional IP address on that machine or anther system where we can run a test instance of docker-mailserver such that a production instance and a test instance do not conflict for running the same services with the same email standards specified ports?
15:32 matterbridge     [mattermost] <aulda> **Mattermost - the new place to be for all things Koha**
15:32 matterbridge     [mattermost] <aulda> [chat.koha-community.org](http://chat.koha-community.org)
15:32 matterbridge     [mattermost] <aulda> Many of us have been discussing for ages how we would like to improve communication across the Koha Community and these discussions really came to a head at Hackfest last week in Marseille.
15:32 matterbridge     [mattermost] <aulda> We all decided that we needed a new tool that is as inclusive as possible, open source, modern, easy to use and familiar. The tool selected is [Mattermost](https://mattermost.com/) and you can start to use it straight away to ask questions, discuss bugs, raise issues or find out news about the Koha project.
15:32 matterbridge     [mattermost] <aulda> It will take a few weeks to settle into, and to implement all of the connections, integrations and bots that will make it as near perfect as a tool can be, but if you are happy to just dive in, then go over to [chat.koha-community.org](http://chat.koha-community.org) and log in.
15:32 matterbridge     [mattermost] <aulda> There is currently a bridge to IRC so that anything posted in IRC appears in a channel in Mattermost and anything posted in that channel (in Mattermost) appears in IRC. There are no plans to leave IRC completely - it will take time for everyone to get used to Mattermost and for the community to make sure it isn’t cutting out any voices if it does decide to stop using IRC.
15:32 matterbridge     [mattermost] <aulda> The current Mailing lists are also unaffected by this change.
15:32 matterbridge     [mattermost] <aulda>
15:32 matterbridge     [mattermost] <aulda> PTFS Europe is hosting the Koha Community Mattermost server and there are a number of admin users across the community.
15:43 matterbridge     [mattermost] <matts> Hi from mattermost :)
15:47 domm[m]          I get a blank page on firefox, and "Limit exceeded" via google-chrome - I assume some finetuning is still needed :-)
15:48 bag              so do I
15:48 bag              hmm
15:48 bag              domm[m]: try this https://chat.koha-community.org/landing#/
15:49 matterbridge     [mattermost] <jpahd> oh no
15:50 matterbridge     [mattermost] <PerplexedTheta> I am not surprised. The server currently rate-limits at 10 queries per second, thats probably not enough now we have 35 people!
15:50 matterbridge     [mattermost] <jpahd> Ah
15:50 matterbridge     [mattermost] <jpahd> Ok
15:50 matterbridge     [mattermost] <PerplexedTheta> Let me up this and quickly restart the server
15:50 domm[m]          JS not loading
15:50 domm[m]          EvalError: call to eval() blocked by CSP
15:51 matterbridge     [mattermost] <jpahd> As long as we don't need a k8s cluster to run our chat :D
15:52 matterbridge     [mattermost] <PerplexedTheta> ```bash
15:52 matterbridge     [mattermost] <PerplexedTheta> ptfs@KOHA-MATTERMOST:~$ free -h
15:52 matterbridge     [mattermost] <PerplexedTheta>                total        used        free      shared  buff/cache   available
15:52 matterbridge     [mattermost] <PerplexedTheta> Mem:           1.9Gi       1.1Gi        73Mi        41Mi       1.0Gi       850Mi
15:52 matterbridge     [mattermost] <PerplexedTheta> Swap:          2.0Gi        81Mi       1.9Gi
15:52 matterbridge     [mattermost] <PerplexedTheta> ptfs@KOHA-MATTERMOST:~$ uptime
15:52 matterbridge     [mattermost] <PerplexedTheta>  15:52:10 up 4 days,  1:31,  1 user,  load average: 0.36, 0.28, 0.22
15:52 matterbridge     [mattermost] <PerplexedTheta> ptfs@KOHA-MATTERMOST:~$ df -h
15:52 matterbridge     [mattermost] <PerplexedTheta> Filesystem      Size  Used Avail Use% Mounted on
15:52 matterbridge     [mattermost] <PerplexedTheta> udev            965M     0  965M   0% /dev
15:52 matterbridge     [mattermost] <PerplexedTheta> tmpfs           197M  848K  196M   1% /run
15:52 matterbridge     [mattermost] <PerplexedTheta> /dev/vda1        59G   17G   41G  30% /
15:52 matterbridge     [mattermost] <PerplexedTheta> tmpfs           984M  1.1M  983M   1% /dev/shm
15:52 matterbridge     [mattermost] <PerplexedTheta> tmpfs           5.0M     0  5.0M   0% /run/lock
15:52 matterbridge     [mattermost] <PerplexedTheta> /dev/vda15      124M   12M  113M  10% /boot/efi
15:52 matterbridge     [mattermost] <PerplexedTheta> /dev/sda         20G  312M   19G   2% /data/compose/55/volumes/app/mattermost/data
15:52 matterbridge     [mattermost] <PerplexedTheta> tmpfs           197M     0  197M   0% /run/user/1000
15:52 matterbridge     [mattermost] <PerplexedTheta> ```
15:52 matterbridge     [mattermost] <PerplexedTheta> We have lots of room to grow πŸ˜‰
16:08 dpk              I have a question for the experts here..  We are seeing loads of these errors:
16:08 dpk              [2024/04/15 17:07:46] [WARN] No method count found for Koha::Items DBIx::Class::SQLMaker::ClassicExtensions::puke(): Fatal: NULL-within-IN not implemented: The upcoming SQL::Abstract::Classic 2.0 will emit the logically correct SQL instead of raising this exception. at /usr/share/koha/lib/Koha/Objects.pm line 583
16:08 dpk              at /usr/share/koha/lib/C4/Search.pm line 2049.
16:09 dpk              And similar error at a lesser rate for some other queries like /usr/share/koha/opac/cgi-bin/opac/opac-ISBDdetail.pl line 188.
16:11 dpk              This has been exacerbated today because the Amazon bots descended on us.  I found that they were using a URL of /opac/opac-search.pl instead of /cgi-bin/opac/opac-search.pl so the recommended robots.txt file was not effective.
16:11 dpk              I have updated robots.txt, but it will take hours before they re-read it and stop.
16:11 matterbridge     [mattermost] <ashimema> @perplexedtheta isn't this what we saw today?
16:11 dpk              Still interested in this NULL-within-IN error though.
16:13 matterbridge     [mattermost] <PerplexedTheta> Yeah it is
16:13 matterbridge     [mattermost] <mblenk> [Bug 29087](https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29087)
16:13 huginn`          04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29087 major, P5 - low, ---, koha-bugs, NEW , Holds to pull list can crash with a SQL::Abstract puke
16:13 huginn`          04Bug 29087: major, P5 - low, ---, koha-bugs, NEW , Holds to pull list can crash with a SQL::Abstract puke
16:14 matterbridge     [mattermost] <mblenk> Although it says 'Holds to pull' it uses the same method
16:14 matterbridge     [mattermost] <PerplexedTheta> Ha damn, thats amazing, it triggers the bot in IRC too
16:15 matterbridge     [mattermost] <PerplexedTheta> Did we end up deploying this on the customer @mblenk ? If not, something to look at
16:15 matterbridge     [mattermost] <PerplexedTheta> Did we end up deploying this on the customer @mblenk ? If not, something to look at  / try
16:15 matterbridge     [mattermost] <mblenk> I think the best way around it for now until we have a patch is to create a dummy itemtype with the `notforloan` column set to 1
16:15 matterbridge     [mattermost] <mblenk> That should prevent the error appearing, just make sure none of your items are linked to that dummy itemtype
16:16 matterbridge     [mattermost] <mblenk> Then when we have a suitable patch you can delete that itemtype
16:16 matterbridge     [mattermost] <mblenk> Does that make sense dpk?
16:18 dpk              No - I'll need a little more explanation.
16:20 dpk              Our library is a research and archive library so we have lots of items that are not for loan, so this could be relevant.
16:21 dpk              Need to leave the terminal for about 25 minutes.  will check back here when I return.
16:21 matterbridge     [mattermost] <mblenk> Do you have any itemtypes in your system that are marked as 'Not for loan'?
16:25 matterbridge     [mattermost] <mblenk> I'm heading out shortly dpk, I'll be back online tomorrow if you still need help. Have a look at the bug I mentioned above and see if that sounds like the issue you're having
16:30 matterbridge     [mattermost] <rtrhimanshu> Hello
16:31 matterbridge     [mattermost] <rtrhimanshu> Is anyone live here
16:33 matterbridge     [mattermost] <caroline> hello @rtrhimanshu
16:34 matterbridge     [mattermost] <rtrhimanshu> Hello
16:34 matterbridge     [mattermost] <rtrhimanshu> I was wondering if anyone has tried to bring sort1 and sort2 visible in self registration of patrons
16:35 matterbridge     [mattermost] <caroline> I think I remember a bug about that... let me check
16:35 matterbridge     [mattermost] <rtrhimanshu> Also is it possible to get automatic data from 264 to 260
16:35 matterbridge     [mattermost] <caroline> [Bug 13901](https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13901)
16:35 huginn`          04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13901 normal, P5 - low, ---, dudzikmichal, Patch doesn't apply , sort1 and sort2 not on self registration form
16:35 huginn`          04Bug 13901: normal, P5 - low, ---, dudzikmichal, Patch doesn't apply , sort1 and sort2 not on self registration form
16:36 matterbridge     [mattermost] <caroline> 264 -> 260 you can use MARC modification templates https://koha-community.org/manual/latest/en/html/cataloging.html#marc-modification-templates
16:36 matterbridge     [mattermost] <caroline> It's not automatic, but it can be used in batch modification, on single records, or when importing MARC files
16:37 matterbridge     [mattermost] <rtrhimanshu> that bug is still pending no one has worked on it
16:37 matterbridge     [mattermost] <caroline> yep, it happens sometimes... you can comment on it to maybe try to revive it
16:38 matterbridge     [mattermost] <rtrhimanshu> I normally use z39.50 to import data so while importing the data gets imported in 264 so i wondered if there is any way to get in 260
16:39 matterbridge     [mattermost] <caroline> hm, I don't think you can use MARC modifiation templates during z39.50 imports, no... someone correct me if I'm wrong.
16:40 matterbridge     [mattermost] <caroline> you can save it and then do Edit > Edit with template
16:42 matterbridge     [mattermost] <rtrhimanshu> ok
16:46 tadzik           is it necessary for the bridge to prefix every IRC message with "[mattermost] "? It's quite obvious that this is where the messages are coming from, and it just adds noise and makes things harder to read
16:48 dpk              mblenk: Yes, most of our itemtypes (except books) are marked not-for-loan.
16:50 dpk              tadzik: I agree - not sure what the bridge is, but its configuration probably needs tweaking.
16:52 dpk              Definitely looks like there is config that would help here: https://github.com/42wim/matterbridge/blob/master/README.md#settings
16:53 dpk              Whoever is running the bridge to this channel will need to look into this.
16:53 matterbridge     [mattermost] <PerplexedTheta> I can remove the protocol line
16:55 matterbridge     <PerplexedTheta> Is that better?
16:57 ashimema         Matrixx text
16:57 perplexedtheta   Bad timing!
16:57 perplexedtheta   Now try
16:57 mattermost       [PerplexedTheta] I think that is probably the most concise
16:58 dpk              Shrink the mattermost nick to mm?
16:59 dpk              We can always see what mm is by doing a right click on mm in the user list.
17:00 mattermost       [PerplexedTheta] Seems fair
17:01 mm               [PerplexedTheta] Done
17:01 * mm             [ashimema] considers removing his element client now.
17:01 dpk              What service are you using matterbridge to gateway?
17:02 Guest1180        [PerplexedTheta] The mattermost server, which is hosted in DigitalOcean
17:02 Guest1180        [ashimema] chat.koha-community.org
17:03 Guest1180        [PerplexedTheta] I think we will need to put the nick back to mattermost
17:03 Guest1180        [PerplexedTheta] mm is being rejected
17:03 dpk              I preferred mm, but as long as its 9 characters or less, you don't mess up the formatting in the IRC client.
17:04 dpk              mattermost is too long.
17:04 Guest1180        [PerplexedTheta] It was working okay in WeeChat
17:04 Guest1180        [PerplexedTheta] What client do you use?
17:04 dpk              XChatAzure on the Mac
17:05 Guest1180        [PerplexedTheta] I see - I can try mmost
17:05 reiveune         bye
17:06 mmost            [PerplexedTheta] Has that worked?
17:08 ashimema         mm worked for me too
17:08 mmost            [PerplexedTheta] It was rejected - probably reserved? Name kept getting changed to Guest
17:09 mmost            [PerplexedTheta] `mmost` should be concise enough hopefully?
17:09 dpk              mmost is fine.
17:10 dpk              mmost: mblenk - are you still around?
17:10 mmost            [ashimema] he'll have finished for the day now dpk
17:11 mmost            [wizzyrea] did someone say that amazon cover images were being a problem today?
17:11 dpk              Bummer.
17:11 dpk              wizzyrea: I was commenting earlier that Amazon bots were DDOSing our catalog.
17:12 mmost            [wizzyrea] hm ok
17:12 dpk              The robots.txt file lacked the URL they were usings (they omitted the /cgi-bin/ prefix)
17:13 mmost            [wizzyrea] we've been getting a lot of alibaba bots recently, just scraping
17:20 dpk              The searches they are doing are just overwhelming our server's capacity.
17:21 dpk              They are not supposed to be able to do that.
17:21 dpk              but the robots.txt file was lacking the URLs they were using (E.g. /opac/opac-search.pl vs /cgi-bin/opac/opac-search.pl)
17:22 dpk              I now have both paths in the file.
17:22 dpk              But I have to wait for them to re-read the file before it takes effect.
17:38 mmost            [rtrhimanshu] Is there a way to import Descriptions of books from amazon
17:41 mmost            [wizzyrea] lots of scrapers don't honour robots.txt. If it's unidentifiable as "helpful" we often block aws IP's
17:50 dpk              Amazon explicitly says they will respect robots.txt.  I will take that on faith for now.  They only read that file
17:50 dpk              once a day and it was clearly missing the URLs they were using.
17:52 dpk              I thing this may be a bug in the default robots.txt file and related docs for Koha.  I suspect the actual URLs in use may have altered over the years, dropping the /cgi-bin prefix, or perhaps it just works with both (have not tested).
18:09 mmost            [PerplexedTheta] @wizzyrea we've been getting alibaba stuff to
18:09 mmost            [PerplexedTheta] @wizzyrea we've been getting alibaba stuff too
18:09 mmost            [PerplexedTheta] I don't think its bots specifically commissioned by alibaba
18:09 mmost            [PerplexedTheta] I think its people using cloud.alibaba.com for botting
18:09 mmost            [PerplexedTheta] Probably nation state stuff. Which is worse
18:10 mmost            [PerplexedTheta] I think its people using alibabacloud.com for botting
18:13 mmost            [wizzyrea] it's Quite Strange
18:13 mmost            [wizzyrea] we block it - are almost to the point of blocking their subnets
18:14 mmost            [wizzyrea] i'm both relieved and unnerved that it's not just us
18:20 mmost            [ashimema] I suggested the subnet approach.. we haven't quite got to that point yet.. but I think we're close too
19:04 mmost            [PerplexedTheta] We've blocked a few of their subnets
19:04 mmost            [PerplexedTheta] But haven't gone for entire ASNs yet
19:04 wajasu           i was getting slammed every by bots every few months.  i updated my robots.txt and it took 4 days before they stopped coming. my CPU dropped to make the opac usable again.  that was last November
19:08 dpk              This is the robots.txt file I currently have: https://dpaste.com/F49PD8WQ3
19:15 wajasu           whoa. I'm getting bogged down now as well.  I'm investigating.
19:16 wajasu           whoa. I'm getting bogged down now as well.  I'm investigating.
19:39 dpk              Hey, GOOD NEWS.  Amazonbot has disappeared from my sight.  Looks like they reread the robots.txt file or my email to them was acted on.
19:41 bag              nice
19:42 mmost            [caroline] good!
19:48 wajasu           i reconciled you robots to mine.  mine was about the same, but i didn't have the Semrush line and Cache-delay.  Mine had the opac-reserve.pl though.  i'm about to reload my webserver.
19:56 dpk              We don't have any reserve traffic - not really that kind of library.  Probably reasonable to add it to the list.
19:57 dpk              Now I just want to get to the bottom of this error:
19:57 dpk              [2024/04/15 20:37:47] [WARN] No method count found for Koha::Items DBIx::Class::SQLMaker::ClassicExtensions::puke(): Fatal: NULL-within-IN not implemented: The upcoming SQL::Abstract::Classic 2.0 will emit the logically correct SQL instead of raising this exception. at /usr/share/koha/lib/Koha/Objects.pm line 583
19:57 dpk              at /usr/share/koha/lib/C4/Search.pm line 2049.
19:58 dpk              Found in plack-opac-error.log
20:03 wajasu           dpk: maybe we have the situation for Koha::Items using  IN ... NULL and this warning is rewriting accordingly.
20:03 wajasu           https://stackoverflow.com/questions/129077/null-values-inside-not-in-clause
20:04 wajasu           i'm not in the position to look at any code right now.
20:07 dpk              This bug was called out earlier as possibly related: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29087
20:07 huginn`          04Bug 29087: major, P5 - low, ---, koha-bugs, NEW , Holds to pull list can crash with a SQL::Abstract puke
22:09 mmost            [PerplexedTheta] perplexedtheta updated the channel display name from: IRC Bridge to: IRC bridge
22:09 mmost            [PerplexedTheta] perplexedtheta updated the channel purpose to: A bridge between the IRC and Mattermost servers
22:10 wajasu           i have this one IP that I suspect is from a crawler, as its mentioned in the 1st colum of plack.log.
22:10 wajasu           i put a ufw deny rule for that IP, but i still have GET requests coming in the plack.log
22:11 wajasu           are requests somehow queued to rabbitmq or some other plack mechanism?  is that why i still see request soming in.
22:12 wajasu           its keeping my VPS server CPU at 40% so my opac take 60secs to return a page.
22:13 mmost            [PerplexedTheta] Do you use cloudflare or something similar?
22:13 mmost            [PerplexedTheta] I got fooled by X-Forwarded-For a while back
22:15 wajasu           oh. maybe i need to get the ufw rule before the https access :)
22:33 wajasu           well, since i was using ufw, i added:  ufw insert 1 deny from <Crawler_IP> to any
22:34 wajasu           did a ufw reload,   and tail -f /var/log/koha/<kohainstance>/plack.log and the crawler isn't getting in for now.
22:49 pastebot         "wajasu" at 127.0.0.1 pasted "example of IP block with ufw firewall for a crawler. note you must have the DENY before the ALLOW rule" (28 lines) at https://paste.koha-community.org/42128