Time Nick Message
22:21 caroline_crazycatlady good night!
22:03 ashimema Looking forward to seeing many of you next week in Marseille.. and will miss all those who can't attend.
22:03 davidnind[m] thanks!
22:03 ashimema thanks everyone who attended :)
22:03 huginn Log: http://meetings.koha-community.org/2019/development_irc_meeting_25_september_2019.2019-09-25-20.59.log.html
22:03 huginn Minutes (text): http://meetings.koha-community.org/2019/development_irc_meeting_25_september_2019.2019-09-25-20.59.txt
22:03 huginn Minutes: http://meetings.koha-community.org/2019/development_irc_meeting_25_september_2019.2019-09-25-20.59.html
22:03 huginn Meeting ended Wed Sep 25 22:03:14 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
22:03 ashimema #endmeeting
22:02 ashimema #info Next meeting: 09 October 2019, 14 UTC
22:02 thd +1
22:01 ashimema any objections/requests?
22:01 ashimema 14UTC for the earlier slow
22:00 ashimema 9th Oct would follow the usual pattern
21:58 ashimema #topic Set time of next meeting
21:58 ashimema #info Nothing to report
21:58 ashimema #topic Review of coding guidelines
21:58 ashimema so.. moving on
21:57 ashimema I'll try and sort that and put it in the next agenda..
21:57 thd The link to the former proposal is missing, the only link is to the HAL documentation.
21:56 ashimema and without tcohen here to promote it I think we should postpone discussion untill the next meeting.
21:55 ashimema Not sure about anyone else, but I've not had a chance to get myself up to speed on this one..
21:55 ashimema #topic API Proposal for related resources (HAL)
21:55 ashimema brill.. next topic
21:54 ashimema thanks :)
21:54 ashimema #info nkuitse is going to subit a new bug with the alternative of making the graphing availabilty syspref controlled.
21:54 nkuitse bye all -- i've got to eat dinner -- thanks for helping with this!
21:54 rangi no worries
21:54 nkuitse thanks again rangi
21:54 nkuitse i'm happy with this -- still going to patch something but i like the long-term solution a lot better
21:53 ashimema thanks nkuitse
21:53 nkuitse ok, yeah, i'll do that ashimema
21:53 ashimema that way you can see where the thinking came from
21:53 nkuitse hah! :-)
21:53 ashimema yeah, create a new bug and perhaps add the original into the 'see also'
21:53 caroline_crazycatlady [off] rangi says nothing the whole meeting and swoops in with simple yet effective solution from nowhere :)
21:52 nkuitse i don't want to muddy the waters, so to speak
21:52 ashimema that would be great
21:52 nkuitse do you think it would make sense to start a new ticket for it?
21:52 nkuitse i can take a look at the code and come up with a proposed patch
21:52 nkuitse :-)
21:51 nkuitse genius!
21:51 nkuitse charting should be on by default so as not to surprise people
21:51 rangi then those who can, can leave it on
21:51 nkuitse very clean, very easy
21:51 ashimema I like that appraoch rangi
21:51 rangi then for your use case, when you can't trust your users, you can disable charts
21:51 nkuitse ooh, i like that
21:51 nkuitse 's ok
21:51 rangi is to add a syspref, disable charts
21:51 cait sorr, got to leave
21:51 rangi what would make a lot more sense
21:50 nkuitse since there are other dedicated tools for visualizing data -- "do one thing well"!
21:50 nkuitse what i really wanted to do -- no offense to anyone who worked on the charting implementation -- was to remove charting altogether
21:50 nkuitse and Koha has no idea which is which
21:49 nkuitse but you can still try to chart the ones that don't make sense
21:49 ashimema totally
21:49 nkuitse just pointing out that charting is showing us that some reports make sense to chart and others don't
21:48 ashimema again.. I may be missing something.. it's been a while since I worked in this particular area
21:48 nkuitse i'm not suggesting munging in a GROUP BY
21:48 nkuitse but not knowing what you're doing can bring down the whole server
21:48 ashimema so how would we programatically be able to add in a GROUP BY ?
21:48 nkuitse yep
21:48 ashimema you can write pretty much any arbitrary read only sql in our reports system..
21:48 nkuitse exactly -- i agree
21:47 ashimema I'm not sure how that would work nkuitse..
21:47 nkuitse i don't know of a memnice kind of thing -- ionice might help, i suppose
21:47 davidnind[m] sorry, thinking out loud here :)
21:47 nkuitse rather than reading in a gazillion rows of raw data and then compiling that into the pretty pie chart the user's after
21:47 nkuitse the problem is, imho, that the graphing should be done on reports with a GROUP BY
21:47 davidnind[m] is there a way in Perl to limit what resources a script uses? something like nice that you can use on the command line (speaking from a point of ignorance about system admin + Perl)
21:46 ashimema at the same time I understand your worries
21:46 ashimema I don't see how a graph that didn't contain all rows would be useful to the end user
21:46 nkuitse and our users like reports in Koha, when we haven't told them to stop running them that is ;-)
21:46 cait we have had some libraries bring down their instance with reports pre-graphs
21:46 cait the sql reports are mighty, but they bare a bit of risk that's true
21:46 ashimema I think I'd prefer an appraoch that simply didn't return the all array untill requested
21:46 nkuitse i just head straight for mysql, i.e., from command line
21:45 nkuitse i'm woefully ignorant of fancy reporting tools
21:45 davidnind[m] could using something like metabase or Urungi in the meantime take the load of your server for reports (I haven't tried either of these, and not familiar with them except what I saw at KohaCon)
21:45 ashimema personally.. we just train all customers to take care when running reports
21:44 nkuitse right, DOS only -- though we've had some data loss (recovered from recent backup)
21:44 ashimema so not before 22 October in reality
21:44 nkuitse 888,000 bibs
21:44 ashimema it's not an imediate security issue so it won't get a special release.
21:44 cait yes, that's a lot
21:44 nkuitse we've got ~2.2 million items
21:44 nkuitse i don't know how many people have the requisite permisions to create reports, but i'm dreading a bad query that's missing a JOIN
21:43 nkuitse it's a pain, because bywater is managing our instance (we host it), but i don't want to wait much longer
21:42 nkuitse if it's that long then we'll patch it on our server and reverse it before we upgrade
21:42 cait next month
21:42 cait so it will be a little while
21:42 cait nkuitse: it was me saying on the bug that maintenance releases just happened
21:41 nkuitse as long as we don't paint ourselves into a corner in the process
21:41 nkuitse i'm fine with wording tweaks; i'm lobbying for a quick fix now and then improve on it later
21:40 cait hm yes - wonder if we shuld mention the pref, but too much text is not good either
21:40 nkuitse so it's pretty clear
21:39 nkuitse the wording is "Include first n rows of full report (ignore pagination)" -- that's the label for the checkbox
21:39 huginn Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23624 normal, P5 - low, ---, paul, Passed QA , Count rows in report without (potentially) consuming all memory
21:39 nkuitse https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92994
21:39 ashimema oh good.. Tomas already did the followup I would have proposed in bug 23624
21:39 cait cool
21:39 nkuitse that's exactly what the latest rev of the patch does:
21:38 nkuitse yeah
21:38 cait i just want to avoid people not being aware of limits
21:38 cait a hint on the graph dialog maybe
21:37 nkuitse because that uses LIMIT and OFFSET
21:37 nkuitse but keep in mind that the limit would not affect browsing through report results
21:37 nkuitse yes, some recent tweaking in the work on the bug does just that
21:36 nkuitse oh, sorry, i misunderstood
21:36 nkuitse you mean a maximum that the user can set when running the report?
21:36 cait please note... only a maximum of x rows will be processed or something like that
21:36 cait as a hint?
21:36 cait could we include the pref setting int the gui?
21:36 nkuitse like $nrows * $ncols * 100 or something
21:36 nkuitse not that i know of, but we could certainly use some heuristics
21:35 thd Is there any efficient way to test quickly for the overall size of the returned data before actually returning it?
21:35 nkuitse but how do they know if a report will generate large results? -- they're not DB folks!
21:35 nkuitse for us, the problem is the outages -- our staff need to run reports but we've had to tell them to hold off on anything that might generate large results
21:34 ashimema 10,000 rows for a report that returns 50 fields is obviously not
21:34 nkuitse there's a very nice fix for 23624 -- very clean
21:34 ashimema 10,000 rows for a report that only returns two fields would be low memory..
21:33 huginn Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23624 normal, P5 - low, ---, paul, Passed QA , Count rows in report without (potentially) consuming all memory
21:33 nkuitse the query for the result set size was bug 23624
21:33 ashimema but it would be incredibly report dependant
21:33 thd Would it not help to introduce a slight delay to first query the size of the result set and if it would be over some large number of rows then prompt with an altered query form with some restriction set to avoid thrashing the system along with a warning.
21:33 nkuitse if you never have memory issues then you leave it unset, i.e., unlimited
21:33 nkuitse so, for example, if you feel pretty good about allowing 10,000 rows then you set it at 10,000
21:32 nkuitse a new sys pref that you can set to limit the number of rows
21:32 cait i tihnk it adds a pref
21:32 ashimema I've not had a chance to look at your patch yet so I may be misunderstanding your proposed solution
21:32 nkuitse the patches in the big report default to unlimited
21:32 ashimema to me a requiest to build a graph should be handled distinctly and with a warning of possible memory issues..
21:32 cait i tihnk we need some message/tooltip in the gui
21:31 ashimema My gut feeling is that limiting the return by default is incorrect as it leads to meaningless output
21:31 nkuitse i'm not sure -- i haven't checked, or at least i don't think i have
21:31 davidnind[m] speaking from a point of ignorance about all things Perl :)
21:30 nkuitse the big one that first brought this to our attention was a report of all items at a library
21:30 davidnind[m] does that mean there is someting in the other report-related Perl scripts that stops them taking over the system resources?
21:30 nkuitse anything at all
21:30 ashimema sorry.. I didn't mean version.. I meant what reports allow graphs.. whether it's anything you build of just limited subset
21:30 cait nkuitse: there is the guided reports step by step thing - and sql - but when you run them it's the same yes
21:29 cait yes
21:29 cait 18.11 have them too
21:29 nkuitse no, we're running 18.11
21:29 cait um sorry no
21:29 cait 19.05
21:29 * ashimema can't remember off the top of his head where/when graphs become available
21:29 nkuitse aren't they the same thing?
21:29 cait i think they share templates
21:29 cait saved sql too?
21:28 nkuitse yes
21:28 ashimema so it's only for guided reports?
21:28 nkuitse that's the long and short of it
21:28 nkuitse (and only if they check the box that says "use all rows" or whatever when creating the chart!)
21:28 nkuitse but guided_reports.pl has no way of knowing whether the user will do that
21:27 nkuitse the full results aren't needed unless the user ends up making a chart
21:27 nkuitse so that the charting javascript can (if the user ends up charting) use all results
21:26 nkuitse then it passes \@allrows to the TT2 template
21:26 nkuitse }
21:26 nkuitse push @allrows, { cells => \@cells };
21:26 nkuitse my @cells = map { +{ cell => $_ } } @$row;
21:26 nkuitse while (my $row = $sth2->fetchrow_arrayref()) {
21:26 nkuitse the problem is in guided_reports.pl
21:25 nkuitse one report recently led to (I think) 5+ GB above and beyond normal memory consumption
21:25 ashimema can you summarise the problem nkuitse
21:25 nkuitse the charting feature in guided_reports.pl consumes unbounded memory -- all data is loaded into memory, converted to JSON (so then 2 copies in mem) and only then sent back to browser
21:25 ashimema This one was only braught to my attention about an hour ago.. not had a great deal of time to look into it..
21:24 nkuitse we've had several outages and service degradations recently due to this
21:24 nkuitse that's my bug
21:24 huginn Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23626 major, P2, ---, paul, Signed Off , Add a system preference to limit the number of rows of data used in a chart when viewing report results
21:24 ashimema #info Bug 23626
21:24 thd I am trying to keep additional undefined values out of wiki migration where I can. Undefined values might not be avoidable for some columns.
21:23 ashimema #topic What's the best way to limit the amount of memory consumed when running a guided report?
21:23 ashimema lets jump on
21:23 ashimema nope.. not yet covered davidnind[m]
21:22 ashimema well.. it sounds like there's interest in such guidance so I'll try to come up with some words and we can continue discussion next meeting.
21:22 davidnind[m] is this covered in the devlopment guide? looks like it should be if it isn't already
21:21 ashimema thd++
21:20 thd Undefined values provoke bugs where the code is not expecting undefined. I personally find seeing undefined to be ugly, difficult to read in some context, and seldom necessary.
21:20 ashimema yup...
21:20 cait empty could still be counted?
21:20 ashimema I think when the return is expected to be a set rather than a singular then we should always return a set (even if empty).. but for the cases where a singular is expected then undef checks are more sane
21:20 cait ashimema: if we have checks for 'number of objects returned' could that be a problem?
21:19 cait I tihnk in this case there is not really a difference between empty and undef
21:18 thd Undefined may have a logical status which may be distinct from an empty set value, however, it seems to be too often used without any logical distinction between undefined and empty set.
21:18 cait d
21:18 cait bugs are ba
21:18 ashimema it's mostly a matter of style.. but I've certinly seen a number of cases lately where the defined check was missed and as such we ended up with bugs
21:18 cait it sounds like the nicer way, but not sure i have enough knowledge to be helpful here :)
21:18 ashimema I mostly wanted to get some opinions on it.. before really suggesting a coding guideline
21:17 ashimema or... we could always return an object, object set and as such do away with the need for the defined check
21:16 ashimema $thing = defined($result->link_method) ? $result->link_method->related_method : undef;
21:15 cait if you get an empty object - no checks required?
21:15 cait hm
21:15 ashimema we could either fall to QAers to make sure any calls to such methods are always of a ternary form
21:14 huginn Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23272 enhancement, P5 - low, ---, tomascohen, RESOLVED FIXED, Koha::AuthorisedValue should use Koha::Object::Limit::Library
21:14 ashimema So.. whilst QAing bug 23272 I noticed that we're starting to develop a pattern bad practice.
21:14 cait is there a best practice there we could follow?
21:13 ashimema #topic Should we allow Koha::Object methods that normally return Koha::Object or Koha::Objects type object to ever return 'undef'?
21:12 ashimema #topic General development discussion (trends, ideas, ...)
21:12 cait yep
21:12 ashimema moving on?..
21:12 ashimema my bad
21:12 ashimema oops
21:11 cait not 26 in the qa queue... but i would like the to be right now
21:11 ashimema #info QA queue: 26 Major/Critical, 77 Total
21:10 cait i hope to have a little session next week with the attending parts of QA team to discuss a few of them together
21:10 cait 77 in the QA queue
21:10 * ashimema has a headache coming on so is rushing through a bit ;)
21:10 cait most are new/nso, so work for everyone there
21:10 ashimema anything else cait
21:10 cait 26 majors/criticals that need attention, in different stages
21:10 ashimema #info cait and ashimema will work on some triage next week at Marseille hackfest.
21:09 ashimema #info It's been a quiet summer, but some bigger bugs have recently hit the QA queue and need attention.
21:09 cait let me take a quick look
21:09 cait it's been a bit of a quiet summer
21:09 ashimema #topic Updates from the QA team
21:08 ashimema #info The latest releases have all gone out, well done RMaints and Team.
21:08 cait maybe we should do some triage work next week
21:08 cait I think some bigger things just hit the QA queue, but I am bit behind
21:08 ashimema #topic Updates from the Release Maintainers
21:08 ashimema #info As above, we're now moving into the polishing phase of the cycle.. there's still time for a few more minor enhancements but generally I would like the team to concentrate on polishing what already pushed and bugfixes.
21:06 ashimema #topic Update from the Release manager (19.11)
21:06 ashimema rmaints
21:06 ashimema tcohen around?
21:06 ashimema yeay :)
21:06 nkuitse #info Paul Hoffman, Fenway Library Organization, USA
21:05 cait #info Katrin Fischer, BSZ, Germany
21:05 ashimema anyone have anything else?
21:05 ashimema #info Marseille Hackfest is upon us... 30 Sept - 4th Oct
21:05 ashimema #info The 'adventurous pushing' window is now closed.. though I will contemplate a few more during Marseile hackfest. Lets move into the polishing phase.
21:03 ashimema #topic Announcements
21:01 thd #info Thomas Dukleth, Agogme, New York City - no more jury duty for a few years again
21:00 davidnind[m] #info David Nind, NZ
21:00 * ashimema is aiming for a short one tonight
21:00 ashimema #info Martin Renvoize, PTFS Europe
21:00 ashimema #topic Introductions
20:59 davidnind[m] Will be back on my desktop soon, on mobile phone at the moment...
20:59 huginn The meeting name has been set to 'development_irc_meeting_25_september_2019'
20:59 huginn Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:59 huginn Meeting started Wed Sep 25 20:59:50 2019 UTC. The chair is ashimema. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:59 ashimema #startmeeting Development IRC meeting 25 September 2019
20:58 thd Yes, I survived jury duty ;)
20:57 ashimema anyone around for the dev meeting?
20:47 huginn caroline_crazycatlady: The operation succeeded.
20:47 caroline_crazycatlady @later tell cait nvm found the picture frame... now trying to find out what the URL is for my image...
20:40 huginn caroline_crazycatlady: The operation succeeded.
20:40 caroline_crazycatlady @later tell cait can you walk me through adding an image in a news item? I can't find the picture frame thing you were talking about
20:33 giuseppep_ Hello, after upgrade to 19.05 I'm having this issue when booking from OPAC, it return me Template process failed: undef error - The method Koha::Hold->is_cancelable is not covered by tests! . What is it ?
20:30 * ashimema is feeling rough tonight and really just wants to go to bed..
20:29 wahanui i heard Next meeting was https://wiki.koha-community.org/wiki/Next_IRC_meetings
20:29 ashimema Next meeting
20:29 wahanui Meeting is over :)
20:29 ashimema Meeting?
20:17 davidnind[m] thanks!
20:08 caroline_crazycatlady I'll let you know what I think of it
20:08 caroline_crazycatlady the book has been ordered! :)
19:53 davidnind[m] Would be interested to know if it is useful or not - it looks like it would be
19:52 davidnind[m] Having never done a migration, the table of contents seems to cover some of things people talk about on the lists or IRC
19:51 caroline_crazycatlady I think I will ask my boss to buy it for me
19:51 caroline_crazycatlady davidnind[m]: interesting!
19:20 davidnind[m] Has anyone every used it, or could recommend it either way?
19:19 davidnind[m] This looks like an interesting book - Migrating Library Data: A Practical Manual https://www.alastore.ala.org/content/migrating-library-data-practical-manual
17:26 huginn Bug 23675: enhancement, P5 - low, ---, koha-bugs, NEW , UseACQFrameworkForBiblioRecords does not work with diacritics (MARC21)
17:26 huginn caroline_catlady: The operation succeeded.
17:26 caroline_catlady @later tell cait can you test? https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23675
17:00 koha-jenkins Project Koha_18.11_U18 build #174: SUCCESS in 49 min: https://jenkins.koha-community.org/job/Koha_18.11_U18/174/
17:00 cait ok, got to go :) cya al later
16:58 cait maybe :)
16:58 * oleonard guesses German ghosts say "üüüüüüüüüüüüüööööööööööööööääääääää"
16:58 caroline_catlady ok
16:57 cait but if you file it, i might be able to later
16:57 cait can't test right now, just have a few minutes until bus time
16:57 cait üöä would be some typical German ones
16:57 caroline_catlady ok. I'm filinf a bug, but I wanted someone to confirm I wasn't hallucinating :)
16:57 cait because we want to - but updates haven't happened yet
16:56 cait caroline_catlady: not yet, not good to hear
16:55 caroline_catlady oh well, thank you tcohen :)
16:55 wahanui it has been said that tcohen is the man you want to thank
16:55 caroline_catlady tcohen?
16:54 caroline_catlady or magnuse maybe? I think norwegian has some non standard characters
16:49 caroline_catlady *some
16:48 caroline_catlady I'm having problems with diacritics and I wondered if it was the same with come german characters
16:47 caroline_catlady cait, do any of your libraries use UseACQFrameworkForBiblioRecords ?
16:11 koha-jenkins Project Koha_18.11_U18 build #173: SUCCESS in 50 min: https://jenkins.koha-community.org/job/Koha_18.11_U18/173/
15:51 koha-jenkins Project Koha_18.11_D8 build #180: SUCCESS in 26 min: https://jenkins.koha-community.org/job/Koha_18.11_D8/180/
15:51 koha-jenkins Project Koha_18.11_D9 build #181: SUCCESS in 26 min: https://jenkins.koha-community.org/job/Koha_18.11_D9/181/
15:27 reiveune bye
15:04 koha-jenkins Project Koha_18.11_D8 build #179: SUCCESS in 38 min: https://jenkins.koha-community.org/job/Koha_18.11_D8/179/
15:01 koha-jenkins Project Koha_18.11_D9 build #180: SUCCESS in 32 min: https://jenkins.koha-community.org/job/Koha_18.11_D9/180/
14:54 koha-jenkins Project Koha_18.11_U18 build #172: SUCCESS in 27 min: https://jenkins.koha-community.org/job/Koha_18.11_U18/172/
14:29 caroline_catlady hi wizzyrea!
14:29 wahanui privet, wizzyrea
14:29 wizzyrea hi
13:51 wahanui OK, oleonard.
13:51 oleonard wahanui: everyone is headed to the hackfest in Marseille
13:50 wahanui oleonard: I forgot everyone
13:50 oleonard wahanui: forget everyone
13:50 wahanui everyone is headed to kohacon that's where everyone is
13:50 oleonard wahanui: everyone?
13:25 wahanui somebody said everyone was headed to kohacon that's where everyone is
13:25 caroline_crazycatlady hey everyone!
11:48 calire :D
11:47 oleonard Still waking maybe
11:47 oleonard Backing?
11:46 * oleonard is backing, keeping his seat warm
11:44 calire come back!
11:44 oleonard-away Oh no, I'm not away!
11:44 calire hi oleonard-away
11:38 oleonard-away Hi #koha
09:25 koha-jenkins Project Koha_Master_D8 build #424: STILL UNSTABLE in 42 min: https://jenkins.koha-community.org/job/Koha_Master_D8/424/
09:18 koha-jenkins Project Koha_Master_U18 build #362: FIXED in 37 min: https://jenkins.koha-community.org/job/Koha_Master_U18/362/
09:18 wahanui Congratulations!
09:18 koha-jenkins Yippee, build fixed!
09:10 koha-jenkins Project Koha_Master_D9 build #880: FIXED in 31 min: https://jenkins.koha-community.org/job/Koha_Master_D9/880/
09:10 wahanui Congratulations!
09:10 koha-jenkins Yippee, build fixed!
08:39 huginn News from kohagit: Bug 23517: (follow-up) AddReserve expects a priority parameter <http://git.koha-community.org/gitweb/?p=koha.git;a=commitdiff;h=06bd256019f519124676719db976d4f6e8258131>
07:01 magnuse \o/
07:00 liliputech_asu salut #koha!
06:48 alex_a Bonjour
06:34 reiveune hello
06:19 huginn magnuse: The operation succeeded.
06:19 magnuse @later tell ashimema already making plans for the next hackfest in marseille?
06:16 * magnuse waves
00:38 koha-jenkins Project Koha_18.11_D8 build #178: SUCCESS in 38 min: https://jenkins.koha-community.org/job/Koha_18.11_D8/178/
00:37 koha-jenkins Project Koha_18.11_U18 build #171: SUCCESS in 35 min: https://jenkins.koha-community.org/job/Koha_18.11_U18/171/
00:25 koha-jenkins Project Koha_18.11_D9 build #179: FIXED in 26 min: https://jenkins.koha-community.org/job/Koha_18.11_D9/179/
00:25 wahanui Congratulations!
00:25 koha-jenkins Yippee, build fixed!