Time Nick Message 00:36 jcamins_away rangi: any idea why MARC::Record uses expat? 00:38 rangi hmm I think it uses whatever the xml parser is 00:39 rangi you could try messing with parserdetails.ini 00:39 rangi try libxml::xml instead or something 00:39 jcamins_away That's what I would've thought, but it looks like we're wrong. 00:39 rangi maybe you can tell it what to use 00:40 jcamins_away The configured parser is LibXML, and it's using that, but it also uses Expat, which is incredibly slow in comparison. 00:40 rangi its marc::file::xml eh? 00:41 * rangi is on phone so can't look 00:41 jcamins_away I'll poke around. 00:41 jcamins_away I was just hoping you knew off the top of your head. :) 00:42 jcamins_away Wow. We use every single parser module. 00:43 rangi heh 00:43 rangi I suspect you can tell it what to use somehow 00:44 jcamins_away According to cfouts, expat is faster anyway. 00:44 rangi expat_xs maybe 00:45 jcamins_away Oh, right. They're different. Good point. 00:46 rangi xs means its a wrapper for compiled c binaries 03:01 druthb o/ 05:54 jcamins_away @later tell rangi In case you were wondering, ZOOM doesn't store IDs. 05:54 huginn jcamins_away: The operation succeeded. 07:08 rangi hey trea and cait 07:08 trea hey rangi 07:08 cait hi rangi :) hi trea 07:08 trea hi cait 07:08 rangi ur up late trea 07:08 trea yea, just noodlin around 07:24 * rangi has to head off in aminute to buy some groceries 07:26 cait don't forget the cookies :) 07:26 rangi :) 07:26 * rangi will bbiab 07:55 rangi back 07:56 cait wb rangi 08:03 bag hey there rangi and cait 08:03 cait heya bag 08:03 bag alright I'm off to bed 08:03 bag :D 08:06 rangi sleep well 08:41 magnuse kia ora #koha 08:42 magnuse bbl 08:42 magnuse nick magnus_afk 08:47 cait hi magnuse :) 08:48 rangi hi magnuse 09:38 magnuse hiya cait and rangi 11:57 druthb o/ 11:57 druthb \o 11:57 druthb \o/ 12:00 magnuse o/ 12:00 jcamins_away o/ 12:01 * druthb is bringing another quarter-million library patrons into the world of being served by a Koha-powered library. :) 12:01 * magnuse is wowed 12:02 magnuse lucky quarter-million! 12:02 druthb :) 12:02 jcamins Well, eww. Zebra has some major issues, doesn't it? 12:03 magnuse it has been rumoured... 12:04 druthb wahanui: druthb? 12:04 wahanui She finally snapped, like we all knew she would. 12:04 * druthb nods. 12:04 magnuse oops? 12:08 druthb hm. 12:08 druthb wahanui: magnuse? 12:08 wahanui magnuse is afraid that we added another 10000 bugs while he was eating pizza. 12:08 * druthb giggles. 12:08 druthb wahanui: Brooke? 12:08 wahanui I'M TAKING THE PISS OUT OF 'EM. or a ratbag. 12:08 magnuse hehe 12:08 magnuse @wunder boo 12:08 huginn magnuse: The current temperature in Bodo Vi, Norway is -7.0�C (1:00 PM CET on January 07, 2012). Conditions: Partly Cloudy. Humidity: 67%. Dew Point: -10.0�C. Pressure: 29.45 in 997 hPa (Falling). 12:08 druthb cait? 12:08 wahanui That's Ms. Cait to you! or in a very good mood today :) 12:08 cait heya 12:08 druthb @wunder lawrence, ks 12:08 huginn druthb: The current temperature in Channel 6 Downtown, Lawrence, Kansas is 0.9�C (6:08 AM CST on January 07, 2012). Conditions: Clear. Humidity: 63%. Dew Point: -5.0�C. Windchill: 1.0�C. Pressure: 30.12 in 1019.9 hPa (Steady). 12:09 magnuse guten tag cait 12:09 cait @wunder Konstanz 12:09 huginn cait: The current temperature in Konstanz, Germany is 2.0�C (1:00 PM CET on January 07, 2012). Conditions: Light Snow. Humidity: 77%. Dew Point: 0.0�C. Pressure: 30.24 in 1024 hPa (Falling). 12:09 druthb guten tag, cait. :) *hug* 12:09 magnuse +2 and light snow = slush ? 12:09 cait it melts when hitting the grond 12:09 cait ground 12:10 magnuse yay 12:12 jcamins Welp, looks like using mod_perl with Zebra Might Not Be A Good Idea. 12:13 cait perhaps it's not zebra 12:13 cait but how we use it? 12:13 cait and hi jared 12:13 jcamins Alas, this is Zebra. 12:14 magnuse ouch? 12:15 magnuse would it be the same for plack? i don't know either of them... 12:15 pastebot "jcamins" at 98.14.120.176 pasted "Yikes! This is NOT O(n)" (31 lines) at http://paste.koha-community.org/192 12:16 jcamins The important bits are columns 1 and 4. 12:17 jcamins For 5k routines, Net::Z3950::ZOOM::event took 32735 seconds to run 95441 times. For 1k records, 71.1s to run 19181 times. 12:19 magnuse 32735s is quite a long time... 12:19 jcamins It is. 12:19 jcamins It is *100* times as long per call. 12:19 magnuse what kind of operation are you profiling there? 12:20 jcamins I am currently telling myself "surely there's something I can do to make it happy." 12:20 jcamins link_bibs_to_authorities.pl. 12:23 magnuse ah, and you are running that with mod_perl? 12:23 jcamins Nope, running it from the command line. 12:24 magnuse ah, so where does mod_perl come into it? 12:24 * magnuse is confused 12:24 jcamins With CGI, Zebra connections are short-lived. 12:24 jcamins With mod_perl, not so much. 12:26 jcamins It's unfortunate that I can't check whether my changes help quickly. 12:29 jcamins To be clear, the problem is not Zebra, it's the Perl interface to it. 12:31 jcamins Woohoo! 12:31 jcamins Wait, maybe I didn't solve it. 12:32 jcamins Drat. 12:32 * magnuse knows the feeling ;-) 12:32 * druthb enjoys watching jcamins sometimes. 12:33 jcamins Hrm. Microseconds are smaller than milliseconds. 12:33 jcamins This is good. 12:33 druthb lol 12:33 cait heh 12:33 cait :) 12:33 magnuse yay 12:33 jcamins Wait... 12:33 * jcamins does some math. 12:33 jcamins 14.6s < 71.1s. 12:34 jcamins Ladies and germs, we have solved the problem! 12:34 druthb yes, yes it is. 12:34 magnuse woohoo! 12:34 cait woohoo 12:34 druthb "we." You and Myshkin? 12:35 jcamins Actually, Myshkin has abandoned me to cuddle with Shari. 12:35 jcamins He doesn't like Zebras. 12:35 druthb At this hour on a Saturday morning...he's trying to set an example for you. 12:35 jcamins The problem, in case you were wondering, is that you have to destroy your Zebra recordSets when you're done with them. 12:36 cait destroy! 12:36 cait sounds radical :) 12:36 * magnuse hums "anarchy in the uk" 12:36 jcamins I didn't recognize that was the issue at first because we already had the destroy command. 12:36 jcamins But it was commented out. 12:36 jcamins Whoops. 12:38 cait so zebra is not so bad? ;) 12:38 jcamins No, thank goodness. 12:39 fredericd jcamins: what are you trying to do exactly? 12:39 * druthb finishes loading 400K+ bibs and the quarter-million patrons, and runs some touch-up scripts. 12:40 jcamins fredericd: run link_bibs_to_authorities.pl on 167k records. 12:40 jcamins Without dying of boredom. 12:40 cait hehe 12:40 cait how long does it take now? 12:40 cait I think you began with an hour? 12:40 jcamins Being impatient, that last is problematic. 12:40 jcamins cait: well, 3k records just took... 308s. 12:40 jcamins cait: previously 5k records took... 12:41 jcamins 33238s. 12:41 cait oh 12:41 cait so it should be faster now :) 12:41 jcamins No typo there. 12:41 jcamins Yes. 12:41 fredericd what have you modified? 12:42 jcamins fredericd: for the performance change I just uncommented the oAResultSet->destroy(). 12:42 jcamins fredericd: more generally, I completely rewrote all the linking code. 12:43 fredericd intersting... 12:43 jcamins Bug 7284 12:43 wahanui Bug 7284 is the first step to making that happen. 12:43 huginn 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7284 enhancement, P5 - low, ---, jcamins, ASSIGNED , Authority matching algorithm improvements 12:44 jcamins wahanui: interesting things? Yes indeed. 12:44 wahanui jcamins: excuse me? 12:45 * jcamins has another idea for optimization. 12:46 cait heh 12:46 cait go jcamins go 12:47 jcamins I'll profile 10k, then try my modification. 12:52 magnuse bbl 12:57 druthb Namaskar, kmkale! 12:59 kmkale Namaste druthb :-D 12:59 kmkale LTNC 13:00 druthb Indeed. It's been a briskly busy fall for me, so I haven't been on here much. 13:00 jcamins I should make BiblioAddsAuthorities use the new working linking. 13:01 kmkale Namaste jcamins :) 13:01 jcamins Namaste, kmkale. 13:09 jcamins Woohoo! MySQL now takes more time than Zebra! 13:20 cait um yay? :) 13:20 jcamins cait: yes, that's good. 13:21 jcamins I mean, the fact that updating a record is hugely inefficient is bad, but it's progress. 13:22 fredericd jcamins: is your code somewhere? 13:23 fredericd patch for bug 6990 could help 13:23 huginn 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6990 enhancement, PATCH-Sent, ---, frederic, NEW , TransformKohaToMarc enhancement 13:23 jcamins fredericd: https://github.com/jcamins/koha/commits/bug_7284 13:23 jcamins Cool! 13:23 jcamins That's exactly what I was just thinking we needed! 13:24 jcamins fredericd++ 13:24 * jcamins may add an additional sign-off on that. 13:26 fredericd jcamins: A C4::Heading::UNIMARC class is required... 13:26 jcamins fredericd: we've never had one in the past. 13:26 jcamins fredericd: I'll add a simple one, though. 13:27 jcamins fredericd: but that means someone will have to test it. :P 13:27 jcamins fredericd++ # for volunteering to test linking with UNIMARC. 13:27 jcamins :P 13:27 fredericd I don't say no :-) 13:31 jcamins What does TransformHtmlToMarc do? 13:31 jcamins It returns a MARC::Record object... 13:31 jcamins right? 13:31 jcamins Seems to. 13:34 fredericd jcamins: I got your branch. Could you rebase it to HEAD? It would help tracking your modifications. 13:37 jcamins fredericd: I'll rebase when I'm done. 13:37 jcamins fredericd: it is based on head as of two days ago. 13:38 jcamins I can't push branches after rebasing. 13:40 jcamins (I guess that's the disadvantage of frequent upstream changes... it makes sharing repositories difficult) 13:43 jcamins fredericd: what if I used a different branch? 13:43 jcamins That should work... right? 13:44 fredericd Yes. You could have a branch bug_7284_v2 for example 13:45 fredericd But I don't need it anymore 13:45 fredericd I was trying to figure out you modification 13:45 jcamins fredericd: does it make sense? 13:46 fredericd I can with git diff jcamins/master, where jcamins is the name of your remote repo 13:46 jcamins Ah. Cool! I didn't know that. 13:46 jcamins The "master" branch on there is spectacularly out of date. 13:46 jcamins GitHub won't let me push to it anymore. 13:47 jcamins I guess I should figure out why and fix that at some point. 13:47 fredericd you can do a git merge 13:48 fredericd I can't test your patch without having a C4::Heading::UNIMARC class! 13:48 jcamins I will add one. :) 13:48 fredericd and I don't like that much how C4::Heading::MARC21 hard code authorities definitions... 13:50 jcamins It's certainly not ideal, but it's been in Koha for years, so it can stay a bit longer. 13:50 jcamins I've been thinking of making it use the framework, though. 13:51 jcamins Thoughts? 13:53 fredericd The framework must contain info required to build $bib_heading_fields data structure 13:53 jcamins Yeah, and I've been thinking that I'd prefer to use that. 13:53 jcamins Since someone might not want to link 655. 13:57 fredericd I have libraries who use specific authorities types located in the 69X biblio tags. For them, C4::Heading::MARC should have to be modified by hand ($bib_heading_fields) 13:59 jcamins So, the question is, how do I cache the results of SELECT authtypecode,tagfield FROM ... 14:01 jcamins Ah, got it. 14:03 jcamins Wait, no I don't. 14:05 fredericd This way, we even could merge MARC21 and UNIMARC classes... 14:07 jcamins Yes, I like that idea. 14:07 jcamins Except for one thing. How do I cache the results of the database access? 14:08 jcamins memoize valid_bib_heading_tag, and have it take $tag and $framework? 14:09 jcamins Oh, there's the problem. Subdivisions. 14:10 jcamins fredericd: do headings currently link with authorities in UNIMARC at all? 14:10 fredericd Yes, but they could stay hardcoded. Not absolutely sure, but I think that MARC21 and UNIMARC share the same letters to identify subdivisons 14:11 fredericd was replying to your subdiv question 14:11 jcamins The form subdivision in UNIMARC is $j. 14:12 jcamins Actually, bigger problem: UNIMARC uses different indexing. 14:12 jcamins GRS-1. 14:13 fredericd we don't use Zebra as we should/could 14:13 * jcamins is not writing UNIMARC indexing. 14:14 jcamins I'm getting paid to fix linking for MARC21, and I can add a UNIMARC class easily enough. 14:14 fredericd I have a pending patch for improving Zebra indexing, and facets building: bug 3216 14:14 huginn 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=3216 normal, PATCH-Sent, ---, frederic, NEW , UNIMARC author facets 14:18 fredericd jcamins: C4::Heading::MARC21::$bib_heading_fields has something to distinguish subject/series/main_entry 14:18 fredericd it doesn't exist in auth_tag_structure and auth_subfield_structure tables... 14:18 jcamins Yes, but that doesn't serve any useful purpose. 14:19 fredericd so, code using that should be removed before 14:19 jcamins Yeah, I was just thinking of doing that. 14:22 cait 3216 looks really interesting 14:22 cait will it work for marc21 too? 14:22 fredericd yes 14:23 cait and hi fredericd :) 14:23 cait we use icu for one library with hebrew 14:23 cait is this only icu for facets or will it use it in general then? 14:24 fredericd general 14:24 fredericd and you can choose during installation 14:28 cait sounds really good 14:28 cait hope to find time to look at it... but probably not for a while :( 14:32 fredericd cait: This patch is essential for UNIMARC libraries. Without it, authors facets don't include firstname 14:32 cait oh! 14:32 cait I think a sign off from someone knowing unimarc would be good then 14:33 cait and another for marc21 14:33 cait there are some system preferences that don't work well with icu 14:33 cait I think it was stemming, fuzzy or/and field weighting 14:33 cait we never really figured it out, but there is some weirdness about them and icu 14:35 fredericd Yes. But it may have been fixed in Zebra/YAZ itself 14:37 cait worth a test :) 14:40 cait perhaps link that to the diacritics patch? 14:41 cait hm bug not patch 14:41 cait I have to run to catch my bus - bye all 15:26 jcamins AAAAAAAHHHHH!!!!!!! 15:26 jcamins We do the stupidest things! 15:34 * druthb occasionally worries about jcamins, you know. 15:34 jcamins druthb: ModZebra is part of C4::Biblio. 15:34 druthb yes....? 15:34 jcamins ModZebra. 15:34 wahanui it has been said that modzebra is part of C4::Biblio. 15:34 jcamins The function used to modify... 15:34 jcamins authorities. 15:35 druthb it's also used to trigger reindexing of a biblio. 15:35 jcamins Yeah, I know. 15:36 druthb Properly, that probably ought to be two separate functions. If you check out an item, you need to update the index---you *don't* need to tinker with its' attached authorities at that point. 15:36 jcamins Ah, no, you misunderstand me. 15:37 jcamins In order to modify an authority record, you must call ModZebra. 15:37 jcamins In C4::Biblio. 15:37 jcamins Providing a "biblionumber" that looks suspiciously like the authid. 15:37 druthb ahhh... 15:37 druthb probably oughta put that in C4::Search::Zebra or some such, then, in the long run. 15:38 jcamins Exactly. 15:38 * druthb is a little slow sometimes, but she usually gets it, eventually. 15:41 jcamins :) 15:42 jcamins Having exhausted members of his own family... 15:42 * jcamins moves on to cataloging imaginary books by members of druthb's family. 15:43 druthb lol 15:44 jcamins Excellent, this works. 15:44 druthb "Thick as mud", by D Ruth Bavousett. "Living with a Crazy Transsexual", by Lindsey Bavousett. "I'm a Teenager, Therefore My Life Sucks," by Libby Bavousett. "Meh, Whatever." by Captain Midnight Bavousett 15:44 * jcamins grumbles about circular dependencies. 15:46 jcamins druthb: lol 16:19 jcamins fredericd: I'm still having trouble figuring out how to cache the results of retrieving a list of tags linked to authorities. 16:19 jcamins fredericd: any thoughts? 16:19 jcamins It's too early for NZ still. 16:28 ktk hw to get rid of an error for patrons with message getting an error as patron has had overdue items and is blocked for 833 days We are getting this error for many patrons 16:31 jcamins ktk: I think I recall someone mentioning a bug that caused this, but I don't recall what it was. 16:31 jcamins (I mean, very recently someone mentioned it) 16:49 ktk is there any patch available to this bug then "to get rid of an error for patrons with message getting an error as patron has had overdue items and is blocked for x days" we are getting this error for several of our borrowers and we cant do any transactions 16:50 ktk does it sort out issue with any latest version of koha? or is there any way to make changes in the database directly and get this error sorted out. 18:01 fredericd jcamins: how to cache what? 18:01 jcamins fredericd: which fields are associated with authority records. 18:02 fredericd I don't understand 18:03 jcamins Wait, never mind. 18:03 jcamins As I wrote out my question in more detail I found the answer. 18:03 fredericd good for you 18:03 jcamins Oh, no, I didn't. 18:03 jcamins Un-never mind. 18:04 jcamins We discussed looking up in the framework which fields were authority-controlled, rather than hardcoding it. 18:05 jcamins So I need to figure out whether a given field is authority-controlled. 18:05 jcamins I do not want to query MySQL *every* time I am processing a heading. 18:07 fredericd jcamins: I suppose you have during initialization and populate the appropriate data structure 18:08 jcamins Yeah, my difficulty is that I really need the equivalent of the C++ "static". 18:08 jcamins Maybe I'll try memoize. 18:10 fredericd In C4::Heading::MARC21 ? 18:10 jcamins Well, just in C4::Heading. 18:10 jcamins That doesn't have to be specific to the MARC flavor. 18:11 jcamins Wait, maybe it does. 18:11 jcamins Drat. 18:11 fredericd Have you checked that: http://perldoc.perl.org/perlmod.html#BEGIN,-UNITCHECK,-CHECK,-INIT-and-END 18:12 jcamins I have not. 18:12 jcamins Cool! 18:12 jcamins Thanks. 18:29 jcamins Okay... it doesn't like variables declared with my. 18:29 jcamins our? 18:30 jcamins Yup. 18:33 rangi and make sure you clear/reset that variable whenever the db changes, or it will cause a big mess when running in a persistent environment like mod_perl or plack 18:36 jcamins rangi: how do I do that? 18:36 rangi find the routine that sets the value in the db, and make that clear the variable 18:36 jcamins Hm. 18:36 jcamins That's in a different file. 18:37 rangi right so that will now need to call a sub in the other one called something like clear_cached_heading or something 18:39 rangi for cgi it doesnt matter so much, the variable dies when the script does 18:39 rangi in mod_perl .. it will last forever 18:40 rangi i think preferences in Context does it 18:41 jcamins Hm. 18:42 jcamins Okay... what's plan B? 18:44 rangi either way, if you cache something, you have to clear it when you change the value 18:44 jcamins Right, it used to be hardcoded. 18:44 rangi yep 18:44 jcamins That has not bothered anyone in 3 years. 18:44 rangi yep, but now its not 18:44 jcamins My plan C is "put it back to hardcoded because this really isn't the issue that I'm trying to deal with." 18:44 rangi so caching it will not be the same 18:44 jcamins Well, it was up until I decided fifteen minutes ago not to hardcode it. 18:44 rangi ah 18:45 rangi id probably do the dynamic bit as a separate thing then 18:46 jcamins Right, that's the plan. 18:46 jcamins "No regressions" = good enough for me. 18:46 rangi yup 18:49 jcamins Sorry, fredericd. 18:50 rangi it can be a separate patch 18:50 rangi but i wouldnt hold up your matching work trying to do it 18:50 rangi kaizen :) 18:51 jcamins Right. 18:51 jcamins I was only doing it 'cause fredericd asked and it looked easy. 18:51 jcamins I was wrong about that last bit. 18:54 jcamins Good news! I remembered my alphabet after a few tries! 18:56 jcamins fredericd: UNIMARC pushed. 18:57 jcamins Argh! 18:57 jcamins I just pushed a typo. 18:58 jcamins fredericd: UNIMARC pushed without typo, now. 18:59 jcamins 140k records matched today. 18:59 jcamins Several orders of magnitude faster. 18:59 rangi :) 18:59 rangi awesome 19:02 jcamins rangi: incidentally, having ModZebra in C4::Biblio is horrifying. 19:02 jcamins Just thought I'd point that out. 19:04 jcamins Also, the AuthoritiesMarc<->Biblio dependency cycle is very vexing. 19:06 rangi yes 19:06 rangi thats the main idea behind Koha:: in my mind 19:07 rangi is make new modules there, more carefully and transition to them 19:07 rangi but not to try to do it all in one go 19:07 jcamins If we had Koha::Search::Engine already, I could've mostly resolved that problem. 19:07 jcamins (AuthoritiesMarc<->Biblio, not cycles in general) 19:07 rangi and i dont think we need to create the whole module 19:08 jcamins No, just the part that does searching. :P 19:08 rangi no in general 19:08 jcamins Ah. 19:08 rangi what i dont want to see 19:08 jcamins Right. 19:08 jcamins Agreed. 19:08 rangi is one huge patch that shifts a whole module over 19:08 rangi unless that module is already clean 19:08 rangi but start with Koha::Biblio for example 19:09 rangi and redo one subroutine to start 19:10 rangi the nice thing with a new namespace is you can do it carefully without busting anything already existing :) 19:10 rangi in theory 19:10 rangi :) 19:11 jcamins Right. 19:12 rangi if i were redoing biblio, i think its totally a candidate for OO 19:13 rangi using Class::Accessor as base 19:13 rangi so i can do 19:13 rangi $biblio->biblionumber(); 19:13 rangi thats as far as ive thought :) 19:14 jcamins :) 19:23 rangi oh look its wizzyrea 19:23 * wizzyrea waves 19:23 * jcamins waves back. 19:23 wizzyrea whassup 19:23 * rangi use to wave then he took an arrow to the kneww 19:23 rangi knee 19:23 rangi joke fail 19:23 * wizzyrea laughs 19:24 wizzyrea naw, not a fail - still pretty funny. 19:24 * jcamins didn't get the joke. 19:24 bag hi there wizzyrea 19:24 wizzyrea see: skyrim memes 19:24 wizzyrea heya bag 19:24 cait heya #koha 19:25 bag yo cait 19:25 wizzyrea wow! 19:25 jcamins Ah. Yeah, I won't get those. 19:25 jcamins Hi bag and cait. 19:25 rangi hey cait! 19:25 wizzyrea i am installing ancient windows games in wind 19:25 wizzyrea wine* 19:25 cait morning rangi, hi wizzyrea and jcamins :) 19:25 jcamins wizzyrea: excellent! 19:26 jcamins wizzyrea: you could play Starcraft! 19:26 cait oh starcract 19:26 cait starcraft 19:26 wizzyrea hehe, I love starcrack 19:26 wizzyrea but i've really only played SC2 19:27 rangi http://soundcloud.com/dubstep/skyrim-in-the-knee-by-rameses-b 19:27 jcamins Morrowind? 19:27 rangi skyrim is elder scrolls jcamins 19:27 wizzyrea ^^ 19:27 rangi the latest one 19:27 jcamins rangi: I know, but Morrowind is old enough it might work in wine. 19:27 rangi yup 19:28 * rangi hasnt played it, my dad has it tho 19:28 jcamins Wait... 19:28 wizzyrea I think skyrim will, but I don't have my super good video card in linux 19:28 jcamins Skyrim runs under Wine? 19:28 jcamins Wow! 19:28 wizzyrea would be better to run it in windows 19:28 jcamins That's incredible! 19:28 wizzyrea *I think* 19:28 wizzyrea I got Rift to play in Wine 19:28 wizzyrea and WoW 19:29 rangi wine is pretty sweet these days 19:29 wizzyrea it really is, quite impressive 19:29 wizzyrea of course they keep thinking of new, novel ways to break it 19:29 wizzyrea but usually 6-8 months and they'll have a tweak for it 19:29 jcamins Morrowind is the only game I've played since Myst 3. 19:30 jcamins Wow... there have been two more Mysts. 19:30 rangi heh 19:30 rangi i only play games at my parents house 19:30 * wizzyrea finds this funny 19:31 * jcamins wasn't so impressed with Myst III. 19:31 jcamins It took a week to win. :( 19:31 jcamins Myst took *years* 19:32 jcamins Okay, that might be because I was... errr... not very old... when it came out. 19:33 jcamins Apparently Myst came out almost 20 years ago. 19:33 rangi lords of midnight 19:33 cait has anyone played day of the tentacle? 19:33 * cait loved that 19:33 cait but was young too :) 19:34 jcamins Never played either of those. 19:34 rangi http://en.wikipedia.org/wiki/The_Lords_of_Midnight 19:34 rangi i was 11 when we got that 19:34 rangi i played that for years 19:34 rangi drew maps on graph paper 19:34 cait heroes of might and magic - I drew maps for that 19:34 rangi course it took about 17 minutes to load each time 19:34 cait hm not heroes 19:34 cait only might and magic 19:35 rangi on my zx spectrum 19:35 * jcamins didn't have one of those. 19:35 rangi so actually i spent years listening to a tape squeal 19:35 cait http://en.wikipedia.org/wiki/Might_%26_Magic 19:35 rangi ah yeah i played that one 19:38 cait i got soo scared in those dungeions 19:38 cait and when I looked at it years later... it seemed quite silly 19:38 rangi hehe 19:38 rangi anyone play ultima ? 19:38 jcamins Ooh... almost done with the 167k linking. 19:38 jcamins Not me. 19:39 jcamins I know about it, though. A bit of a cult classic, I think. 19:39 rangi http://en.wikipedia.org/wiki/Ultima_(series) 19:39 rangi yeah 19:39 rangi it was fun 19:39 rangi i played 1 - 5 19:41 cait :) 19:41 rangi my nephew (who is 4) was running around yesterday yelling 19:42 rangi "where's my sword, brigands are attacking" 19:42 jcamins lol 19:42 rangi i suspect he has been watching his dad play games 19:42 rangi it was at the market, was quite funny 19:43 jcamins Bib 167198. 19:43 jcamins 167313 19:44 jcamins ^^ do you see that? In just a few seconds it did over a hundred! 19:44 rangi nice 19:44 rangi and because it doesnt need to rebuild everything in one go thats totally good enough 19:45 jcamins Yup. 19:45 jcamins Heh. Producing the linking report takes a long time. 19:45 jcamins Wow. It's a 21MB file. 19:46 rangi whoa 19:46 jcamins It includes a list of every single bib it processed. 19:46 rangi ahhh 19:47 jcamins Right... guess I need to write a script that will remove that list. 19:47 rangi that'd do it, well too much info beats not enough 19:47 jcamins Exactly. 19:47 jcamins First time running it on the entire database. 19:48 cait jcamins++ :) 19:49 jcamins 12.5MB for the report only. 19:54 jcamins 340k unlinked; 495k linked; 5877 fuzzily linked. 19:54 rangi oh thats pretty darn good 19:55 jcamins Now, how to improve it? 19:56 rangi well i think thats a good improvement from the current one 19:56 jcamins Yeah, certainly. 19:56 cait what does fuzzy link mean? 19:57 rangi and worth getting others to test 19:57 rangi like cait 19:57 rangi hehe 19:57 jcamins :) 19:57 * cait hides 19:57 cait I gave him data! 19:57 * jcamins will be trying with cait's data. :) 19:57 cait how long did it take for those now? 19:57 jcamins I didn't try your data yet. 19:57 cait and you know that our data could be perfectly linked... if the script understood the $0 19:57 jcamins Right. 19:57 cait :P 19:58 rangi hmm 19:58 rangi i wonder if that paul guy would give you data 19:58 jcamins cait: you can write your own linker for that, you know. Very easy with my new architecture. 19:58 jcamins I'm sure he would. 19:59 * jcamins would probably prefer to beg MDAH for data, though. 19:59 rangi yep 19:59 rangi i just thought might be a nice PR 19:59 jcamins Not that it's likely they'd refuse to give it to me. 19:59 jcamins True. 20:00 * jcamins will consider. 20:00 cait jcamins: I should have seen that coming... 20:00 rangi :) 20:00 jcamins [off] This is why you shouldn't be a jerk. 20:01 jcamins [off] Especially not when I'm grouchy. 20:01 rangi [off] yeah it makes it harder for people to help you 20:01 jcamins Ohh. 20:02 jcamins Figured out why these aren't linking. 20:02 jcamins No auth record. 20:02 jcamins The cataloger is going to be *pissed*. 20:03 jcamins (but, when you don't have an authority record, you're kind of SOL on the whole "automatic linking" thing. 20:04 jcamins cait: remind me where to find your data? 20:04 cait ah 20:04 jcamins Wait, found it. 20:04 cait thought you had downloaded it 20:05 jcamins I did. 20:05 jcamins But I want to test it on a server, and I have a lousy home connection. 20:05 cait want me to find the link? 20:05 jcamins No, I have it. 20:05 jcamins What's the difference between _1 and _2? 20:05 jcamins Just different parts of the data? 20:05 cait I think so 20:06 cait sliced up a bit 20:06 cait yep 20:06 cait https://wiki.bsz-bw.de/doku.php?id=v-team:daten:openaccess:maub 20:06 cait says 2 parts 20:08 jcamins cait: you'll be glad to know that writing your linker module will be super-easy. 20:08 jcamins :) 20:10 wizzyrea bbiab 20:11 cait jcamins: will be even happier when you help me doing it :) 20:14 jcamins Can I softlink to a file which is under a softlinked directory? 20:16 jcamins Answer: yes. 20:16 * jcamins refuses to ask the follow up question: "Should I?" 20:17 cait don't look at me 20:17 cait i don't know 20:17 * cait offers dried cherries 20:24 jcamins Wow! 20:24 jcamins cait: can I please use your data center? 20:24 jcamins Pretty please? :) 20:24 cait my what? 20:25 jcamins BSZ-BW's data center. 20:25 jcamins I've never gotten 5MB/sec download before. 20:25 cait oh 20:25 jcamins :) 20:25 cait perhaps has something to do with belwue? 20:26 jcamins It's nice and fast, whoever hosts it. :) 20:26 cait and you are probably downloading at a good time 20:26 cait our servers are at the data center of university of konstanz 20:26 jcamins Oh, maybe they have Internet2. 20:27 cait http://www.belwue.de/ - perhaps that's the reason? 20:28 jcamins Ah. Maybe. 20:30 jcamins cait: aut = authorities, tit = biblios? 20:30 jcamins cait: oh, in answer to your question earlier, "fuzzy" is whatever the linker defines it as 20:30 cait yep 20:31 cait sounds right 20:31 jcamins In this case, it means "more than one authority matched." 20:33 jcamins Importing Mannheim's data. 20:34 jcamins 1.4 million titles? 20:34 jcamins Eek. 20:34 * jcamins will try just importing _1. 20:34 jcamins Hopefully that will be enough. 20:34 cait are there 2 auth files? 20:34 jcamins Yup. 20:35 jcamins 20k imported... 20:39 jcamins 50k imported... 20:41 jcamins cait: you have this in a Koha instance? 20:41 cait nope 20:41 jcamins It seems to me that this must be one of the larger installations. 20:41 jcamins Oh. 20:41 jcamins Right. 20:41 jcamins I guess it's not, then. 20:41 cait it's data from the union catalog 20:41 cait oh 20:41 cait and it's not converted... with our koha converter 20:42 cait so you will have wrong normalization perhaps 20:42 jcamins What does your Koha converter do? 20:42 jcamins Ah. 20:42 jcamins Hm. 20:42 cait unicode normalization 20:42 cait and some other evil things 20:42 jcamins Well, I'll bug you if it doesn't work. :P 20:42 cait like moving the subject chains 20:42 jcamins Moving the subject chains? 20:42 cait breaking them up into single subjects and move them to the proper marc21 fields 20:42 cait 689 20:42 cait look at that 20:42 jcamins Uh-oh. So there aren't subjects in 650s? 20:43 cait hm 20:43 cait there might be some from loc 20:43 cait oh right 20:43 jcamins That won't help me test your subject headings. :P 20:43 cait I should have told you that... 20:43 jcamins Yeah... 20:44 cait sorry 20:44 cait didn't think about it 20:44 jcamins That's okay. 20:44 cait about that part... the other things should not make a big difference 20:44 jcamins Well, I have 90k authorities from BSZ-BW imported now. 20:44 cait there will be 20:44 cait corporate authors 20:45 cait personal names 20:45 cait no meeting 20:45 cait topical, geographic 20:45 jcamins Eh. It's running on a server. We'll see what happens. 20:45 jcamins :D 20:45 jcamins Better increase the size limits on the Zebra directories... 20:46 jcamins 40GB each better be enough, 'cause I don't have infinite disk space on this server. 20:46 cait the structure of the 689 is ... not so nice I think 20:47 jcamins That's very diplomatic. 20:47 jcamins Perhaps Uli will be interested in testing the linking, to see if it can simplify things for you at all. 20:48 jcamins :) 20:49 cait at the moment we don't link then 20:49 cait them 20:49 cait we import them linked 20:49 cait but it would be nice to not rely so much on our authority number 20:49 cait but use koha internal numbers for linking 20:49 jcamins Yes, but maybe you can find a use for it. :) 20:49 cait and it would be cool to figure that out until may 20:49 cait because they change the fomrat 20:49 cait for the authorities 20:49 jcamins In May? 20:49 jcamins Why? 20:50 cait it's a real big project 20:50 jcamins Uh-oh. Big projects are always bad news. ;) 20:50 jcamins 140k imported. 20:50 jcamins This is blazing fast. 20:50 jcamins That worries me. 20:53 cait nice data? :P 20:53 jcamins I suppose that's possible. 20:53 cait it's not bad news, but it's really big 20:53 cait all union catalogs closing, converting data to new format 20:53 cait training catalogers 20:53 cait ... 20:54 jcamins Exciting times. :) 20:54 cait yep 20:54 cait and we will reload all koha libraries with union catalog data 20:54 cait authorities especially 20:54 cait but also the bibliographic records 20:55 jcamins Well... 20:55 jcamins This is weird. 20:55 cait the advantages of a central cataloging database :) 20:55 cait so it's a good time to make changes to the converter and improve things 20:55 jcamins I have 171k authority records in my database, but none show up in the staff client. 20:56 jcamins Actually, I'm just having issues figuring out how to access authority records. 20:56 cait hmm 20:57 cait so now they show? 20:57 jcamins Oh, I just don't know how importing authorities works, apparently. 20:57 jcamins Ignore me. 20:58 cait that wouldn't be very nice... to ignore yo 20:58 jcamins Authorities are working fine. 20:58 jcamins Strangely it's starting at 16million. 20:58 cait hm 20:59 cait what does it look at? 20:59 cait the number in 001? 20:59 jcamins 001. 20:59 jcamins Yeah. 20:59 cait ah 20:59 cait that' scorrect 20:59 jcamins Something else you should've warned me about? :P 20:59 cait always 9 digits 20:59 cait hehe 20:59 cait this is a test 20:59 cait if I did tell you everything... would not be a lot of fun 20:59 jcamins What are you testing? 20:59 jcamins lol 21:03 * magnuse waves 21:03 cait hi magnuse :) 21:04 magnuse guten abend cait 21:04 cait :) 21:06 cait jcamins: not you if you are worried about that :) 21:06 jcamins lol 21:20 jcamins Oh dear. 21:20 jcamins So... 21:20 jcamins Don't load Mannheim into a Koha database. 21:21 cait oh 21:21 cait what happened? 21:21 jcamins The tablespace was filled. 21:21 cait ? 21:21 jcamins DBD::mysql::st execute failed: The table 'auth_header' is full 21:22 cait oh 21:22 cait full?? 21:22 * jcamins shrugs. 21:22 cait what doest hat mean full? 21:23 jcamins I have no idea. 21:23 cait that's kinda.. scary 21:23 jcamins Agreed. 21:28 jcamins I think I'll just drop that database and pretend it didn't happen. 21:29 cait oh :( 21:29 * magnuse shares some equinoxe from valrhona with the weekend/late night crew 21:29 cait hi druthb 21:29 druthb hi cait 21:29 magnuse "If a table-full error occurs, it may be that the disk is full or that the table has reached its maximum size." http://dev.mysql.com/doc/refman/5.0/en/full-table.html 21:29 * jcamins will beg you for a smaller dataset that you use for testing. 21:29 magnuse kia ora druthb! 21:30 druthb kia ora, magnuse. :) 21:30 cait jcamins: I was interested in the results of that! 21:30 cait :) 21:30 cait but remind me to dig for some data on monday 21:30 jcamins cait: me too, but it'd take days to reindex, too. 21:31 cait hm k 21:32 cait I have created a small dataset for testing 21:32 cait quite small... but might be interesting for you, lots of links and things 21:33 jcamins Perfect! 21:33 cait waiting for the 'converted' version 21:33 cait so then the subjects should be correct, but it's quite small 21:33 cait wanted to it for testing links in the opac 21:36 jcamins Okay, how to I rebase a whole bunch of patches onto a new branch? 21:37 jcamins *do I 21:38 magnuse jcamins: are they patch files, or commits on a branch? 21:38 jcamins Commits on a branch. 21:38 jcamins Usually I'd use git format-patch and apply that, but I'd prefer to do it the right way. 21:38 magnuse "just" rebase that branch onto the branch where you want the commits, then, i would guess? 21:38 jcamins But won't that eliminate the original branch? 21:39 * magnuse was doing it the wrong way a minute ago 21:39 magnuse the branch that you want to get the commits from? 21:39 magnuse hm, i'm no expert, but i think it will 21:40 magnuse but would you want to have the commits in two places? 21:40 jcamins Yeah, that's what I'm trying not to do. 21:40 magnuse maybe cherry-picking? 21:40 jcamins Yes, because on the second branch I'm going to squash commits, and I want to have a working branch if I screw stuff up. 21:40 jcamins I thought that might be the answer. 21:42 jcamins Wait, I have an idea. 21:43 * magnuse is waiting 21:43 jcamins I can checkout a copy of my branch, then rebase the copy onto kc/master. 21:44 magnuse yeah, a copy sounds like what you want 21:46 jcamins That worked. 21:46 magnuse yay! 22:00 * magnuse has tried to rebase catalyst's l10n work on the 3.6.x branch and is now trying to build packages with translations off the result... 22:00 jcamins Woohoo! 22:00 jcamins magnuse++ 22:00 magnuse well, i did say *try to*! 22:03 cait magnuse++ 22:03 cait we believe in you :) 22:04 magnuse thanks 22:04 magnuse "debian/list-deps: unknown package for CGI::Session::Driver::memcached" does not look too promising, perhaps... 22:04 magnuse but i think i might know the solution, at least in theory 22:09 rangi yep 22:09 rangi that module is at debian.kc 22:10 rangi ull just need that in ur sources list 22:10 magnuse yeah, i think eythian told me to do: 22:10 magnuse sudo pbuilder create --othermirror 'deb http://debian.koha-community.org/koha [^] squeeze main' 22:10 magnuse oops, without the "[^]" of course 22:10 rangi should do it 22:10 jcamins That sounds right. 22:11 jcamins Wow. I really didn't expect this to involve quite so many lines. 22:11 jcamins +1709, -578, after squashing a bunch of small commits together. 22:14 jcamins Any idea how to sum up *all* the values in a hash? 22:17 rangi foreach loop? 22:18 * rangi attempts to burn down the house 22:18 rangi making popcorn 22:18 jcamins rangi: I was hoping there was a nicer way. 22:18 cait lol 22:19 rangi jcamins: there might be, but can't think of it 22:20 magnuse ooh, it's generating nb-NO templates from the translations... 22:20 jcamins Hooray! 22:21 magnuse still no guarantee of ultimate success... 22:21 cait crossing fingers 22:22 magnuse no luck... 22:22 magnuse dh_install: koha-common missing files (debian/tmp_docbook/*.8), aborting 22:22 magnuse make: *** [binary] Error 2 22:22 magnuse dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2 22:22 magnuse E: Failed autobuilding of package 22:23 rangi looks like man page stuff 22:23 magnuse yeah, weird 22:25 rangi hmmm 22:25 magnuse well, that's a headache for another day, i'm off to watch the last half hour of "Lewis", season 5 episode 3! 22:25 rangi :) 22:25 magnuse see ya tomorrow, folks! 22:25 rangi make sure u write it up :) 22:26 cait bye magnus_away 22:26 magnus_away rangi: will do 22:26 magnus_away c ya cait! 22:34 jcamins Is there a way to get git-bz to attach multiple patches with one command? 22:35 rangi not without hackery I think 22:35 jcamins That's what I figured. 22:35 cait jcamins: i do something like HEAD^^^ 22:35 cait and only -e for the first 22:35 cait and then just delete one ^ after the other 22:36 cait but not really elegant... 22:36 jcamins cait: that's what I do, too. 22:37 rangi read up on git, how to specify a range 22:37 rangi and try that 22:37 rangi it might work :) 22:37 jcamins Hm. 22:37 jcamins If I remember I'll try next time. 22:38 jcamins Already posted the patches. 22:38 jcamins Bug 7284 22:38 wahanui Bug 7284 is probably the first step to making that happen. 22:38 huginn 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7284 enhancement, P2, ---, jcamins, ASSIGNED , Authority matching algorithm improvements 22:38 rangi cool! 22:38 cait very cool :) 22:40 jcamins So, if anyone was looking for something *incredibly absurdly totally and utterly* awesome, today is your lucky day. 22:40 rangi Hehe 22:40 rangi I do need to do authority linking 22:40 jcamins Well, give it a whirl. 22:40 rangi but not for a week or so 22:40 jcamins Grr. 22:40 jcamins :) 22:40 rangi first I have to make the authorities 22:40 rangi from a csv file 22:41 jcamins Hmm... 22:41 jcamins Yeah, might be a good idea. 22:41 rangi on leave next week 22:41 rangi taking the family to rotorua 22:41 jcamins Cool! 22:42 rangi to see geysers and boiling mud etc 22:42 jcamins Mmm. Boiling mud. 22:43 rangi cait and magnuse have been there 22:43 rangi and paul, hdl and slef too 22:43 cait :) 22:43 cait and geysir 22:43 huginn New commit(s) needsignoff: [Bug 7284] Authority matching algorithm improvements <http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7284> 22:44 cait I have heard they have 'mud for men' at the shop there :P 22:44 cait maybe you should get some 22:44 rangi heh 22:44 rangi ill pass 22:45 cait oh 22:45 cait and they have kiwis 22:45 rangi yep 22:45 rangi we have some here in the zoo 22:45 rangi u can see them at zoodoo 22:45 jcamins Kiwis? I guess this is something other than the fruit? 22:46 cait :) 22:46 rangi the fruit is kiwifruit 22:46 rangi kiwi is the bird it takes it name from 22:47 rangi small brown and hairy looking 22:47 jcamins Ah. 22:47 jcamins I didn't know that. 22:47 rangi I think the us drops the fruit bit off the name 22:48 rangi maybe germany too cait? 22:48 cait yep 22:48 cait only kiwi here 22:58 cait rangi: going to take a hot bath too?... forgot how they are called 22:59 rangi hot pool 22:59 cait yes hat 23:00 cait that 23:01 rangi yep I think we will 23:01 cait :) 23:19 * jcamins has been evicted. 23:19 rangi oh? 23:19 jcamins A higher life form has preempted my chair. 23:19 rangi ahh 23:20 jcamins He is currently *loudly* washing his paws. 23:20 * cait waves at myshkin 23:20 jcamins When I put my face near his, he licks my nose. It's very cute. 23:21 jcamins I'm not sure exactly why. 23:24 jcamins I guess it doesn't meet his standards of cleanliness. 23:24 rangi heh 23:24 cait hehe :) 23:25 magnus_away rotorua was fun! 23:26 rangi heh you woke up to say that? 23:26 * magnus_away saw a wild kiwi on kapiti 23:26 magnus_away sure! 23:27 rangi ohh a wild one 23:27 rangi you a literally one in 100 million or so 23:27 rangi to have done that 23:28 magnus_away yay! 23:36 rangi @wunder nzwn 23:36 huginn rangi: The current temperature in Wellington, New Zealand is 14.0�C (12:00 PM NZDT on January 08, 2012). Conditions: Rain. Humidity: 100%. Dew Point: 14.0�C. Pressure: 29.86 in 1011 hPa (Steady). 23:36 rangi hope that stops before tomorrow 23:39 * magnus_away goes to sleep 23:40 cait good night magnuse :) 23:40 rangi sleep well 23:40 cait @wunder Konstanz 23:40 huginn cait: The current temperature in Taegerwilen, Taegerwilen, Germany is 3.8�C (12:35 AM CET on January 08, 2012). Conditions: Light Rain. Humidity: 91%. Dew Point: 2.0�C. Windchill: 4.0�C. Pressure: 30.22 in 1023.2 hPa (Steady). 23:41 jcamins Good night, cait. 23:41 cait oh 23:41 cait that was for magnuse :) 23:41 jcamins Yes, and that was for you. :) 23:41 cait but actually... not a bad idea 23:41 cait ohok