Time Nick Message
11:45 martinmorris hello, can anybody help me with marc importing in koha 3.0?
12:17 hdl martinmorris: ?
12:18 hdl It seems quite simple. But can you detail your problems ?
12:20 martinmorris i was getting an error when importing records with the "Check for embedded item record data" set to "Yes"
12:20 martinmorris just recreating it now
12:20 martinmorris Software error:
12:20 martinmorris Wide character in null operation at /usr/local/share/perl/5.8.8/MARC/Charset/Table.pm line 96, <fh00001katalogo-20071122.mrc> line 6039.
12:20 martinmorris CGI::Carp
12:21 martinmorris there aer 6050 records if that's at all useful
12:21 martinmorris when i set that option to no the records import fine
12:21 martinmorris that was all
12:21 martinmorris incidentally i think there's an extra perl module needed which isn't requested at install stage
12:21 martinmorris Algorithm::CheckDigits
12:21 martinmorris (for barcodes)
12:22 hdl Oh. This is optionnal perl module then.
12:22 martinmorris if you want to do barcodes, yes
12:24 hdl wide character in null operation can come out of the fact that you are in MARC21 but and your MARC records are not encoded in MARC8 when leader encoding is not "a"
12:24 martinmorris ah, they're not, they're UTF-8
12:25 hdl they are UTF8 but is leader position 9 or 8 an "a" ?
12:25 martinmorris i'd have to check and will do, but have to go to work now
12:30 martinmorris thanks hdl, bye
12:46 CGI478 hi
12:49 hdl hi
14:37 hdl kados ?
14:49 paul_ owen ?
16:41 kados morning hdl
16:41 kados hdl: it's a US holiday today, so I slept in :-)
16:42 hdl Good. I hope you slept well and nicely.
16:42 kados hdl, paul: I had to make some changes to Search.pm to fix a few bugs
16:42 hdl We had a nightmare today
16:42 kados nightmare?
16:42 hdl Search was totally broken.
16:42 hdl No answers
16:42 kados hmmm
16:43 kados seems to work OK for me
16:43 kados do you have a use case?
16:43 hdl paul found a trick but it doesnot work for me.
16:43 hdl on i14.hdlaurent.paulpoulain.com
16:44 hdl keyword particules
16:44 kados logging in
16:44 hdl or anything.
16:44 hdl It vanishes.
16:45 kados hmmm
16:45 kados can I check your sysprefs?
16:45 hdl Please do.
16:46 kados first, why do you have four: # Circulation
16:46 kados # Patrons
16:46 kados # Catalogue
16:46 kados # catalogage
16:46 kados ?
16:46 hdl I also have this error : HTML::Template::Pro:in /TMPL_IF at pos 1864:ERROR:
16:46 hdl I added catalogage in Intranetnav.
16:47 kados ahh
16:47 hdl This donot impact since paul has the same error.
16:47 kados first you're missing many sysprefs
16:47 kados QueryFuzzy, QueryStemming, QueryWeightFields
16:47 kados perhaps there is a bug if QueryWeightFields is off, the search doesn't work as expected
16:48 hdl yes but those were never added in updatedatabase.
16:48 kados hmmm
16:48 kados it's not that
16:48 kados they weren't?
16:48 kados sorry if not
16:49 hdl And i thought they were not necessary for koha to work.
16:49 kados shouldn't be
16:49 hdl mere plugins for more serach performance.
16:49 kados as default values are set to 'off'
16:49 kados in the code
16:49 kados I've just tested, and it works fine for me
16:49 kados :/
16:50 hdl I saw them desabled by default.
16:50 paul_ kados : it's very easy to explain why it can't work (at least for me) :
16:51 paul_ at the end of buildQuery :
16:51 paul_ return ( undef, $query,$simple_query,$query_cgi,$query_search_desc,$limit,$limit_cgi,$limit_desc );
16:51 paul_ so 9 parameters returned
16:51 paul_ opac-search.pl looks like :
16:51 paul_ ( $error, $search_desc, $koha_query, $federated_query, $query_type ) =
16:51 paul_ buildQuery( \@operators, \@operands, \@indexes, \@limits );
16:51 paul_ so get only 5, and they are differents.
16:51 paul_ the $query_type does not contain a query_type.
16:52 kados I plan to fix the opac as soon as the staff side is working bug free
16:52 kados hdl: is this zebra or no zebra?
16:52 kados zebra by the syspref
16:52 kados hmmm
16:53 hdl I would not dare to ask you to test a NZ installation
16:53 kados hdl: I plan to fully test NZ
16:54 hdl kados : Sure you will, but now you and we are working on "usual" zebra installation
16:55 kados yep
16:55 kados by Saturday, I expect to have zebra and nozebra working on both staff and opac side with (hopefully) no bugs
16:56 kados hdl: can you run:
16:56 kados git whatchanged
16:56 kados hdl: and tell me the last change you have on your repo?
16:58 kados one fixes boolean AND searching in the adv search, and the other is just an interface modif
16:58 kados hdl: please verify you're running off the latest rebase
16:59 hdl switching from searchdesc to query_desc and limit_desc,
16:59 hdl searchdesc is now just a flag
16:59 hdl
16:59 hdl preserve sort_by choice in facets
16:59 hdl I ran git fetch and rebase.
17:00 kados ok, you're up to date
17:00 hdl No other commit
17:00 paul_ very poor dsl connexion today...
17:00 kados what I find strange is that:
17:00 kados http://i14.hdlaurent.paulpoulain.com/cgi-bin/koha/catalogue/search.pl?&q=particules
17:01 kados is not reported in the query_desc
17:01 kados it should have:
17:01 kados kw=particules
17:01 kados or just 'particules'
17:01 hdl I know, I have
17:01 hdl QUERY CGI:q=particules at /home/hdl/Koha/Code/koha_dev//C4/Search.pm line 871.
17:01 hdl QUERY DESC:particules at
17:02 hdl in logs
17:02 kados does zebra do the search?
17:02 hdl But shows nothing in getrecords.
17:02 kados can you confirm you have no local modifs to Search.pm?
17:03 hdl Latest search is at 15h.
17:03 kados even:
17:03 kados http://i14.hdlaurent.paulpoulain.com/cgi-bin/koha/catalogue/search.pl?&q=(particules)
17:03 kados doesn't work
17:03 kados and that skips all the buildQuery code and calls getRecords directly
17:04 kados can you do:
17:04 kados git checkout -f C4/Search.pm
17:04 kados just to make sure there are no local modifs?
17:05 kados ditto with search.pl
17:05 kados and results.tmpl
17:05 kados (although, it's clearly not the template)
17:06 hdl updating paths is incompatible with switching branches/forcing
17:06 hdl Did you intend to checkout 'C4/Search.pm' which can not be resolved as commit?
17:06 kados even scan doesn't work
17:06 kados updating patches?
17:07 kados paths I mean?
17:07 kados I suspect your code is not the latest from git
17:07 kados are you running off a git repo?
17:07 kados or from an installation?
17:08 kados your search creates many many &limit=&limit=&limit=&limit=&limit=&limit=&limit=&limit=&limit=&limit=&limit=&limit=
17:09 kados while mine doesn't
17:09 paul_ kados: mine does the same thing.
17:09 kados any ideas why?
17:09 paul_ (lot of limits=&)
17:09 hdl running from git repo.
17:10 hdl And it has been doing this for some time now.
17:10 paul_ (same for me, I noticed that I don't remember when but some time ago)
17:10 hdl I updated all the required files from git and now :
17:10 hdl Can't call method "size" on an undefined value at /home/hdl/Koha/Code/koha_dev//C4/Search.pm line 391.
17:11 kados I can't reproduce either the many limits nor the search prob :/
17:11 kados there must be some difference in your code
17:11 paul_ maybe some diff in zebra config files for the search problem.
17:12 kados ahh, for the limits one
17:12 kados I bet it's the apache conf
17:12 kados you don't have the rewrite rules set up
17:13 paul_ right.
17:13 kados search still works
17:13 kados but I do have many limits now :-)
17:14 kados I think there must be some code differences
17:14 kados please make sure you have the latest git for all relevant files
17:14 hdl I did.
17:14 hdl I did for all the files you told me.
17:15 hdl And now, I get a big zebra error.
17:15 kados hmmm
17:15 kados what is the zebra error from zebra logs?
17:15 kados and as reported in apache logs?
17:18 hdl search.pl: WARNING: query problem with kw= testand (mc=OUVR or mc=DVD) ZOOM error 10014 "CCL parsing error" (addinfo: "Operator expected") from diag-set 'ZOOM' at /home/hdl/Koha/Code/koha_dev//C4/Search.pm line 339., referer: http://i14.hdlaurent.paulpoulain.com/cgi-bin/koha/catalogue/search.pl
17:19 kados yep
17:19 kados you can see the prob
17:19 kados missing a space between test and and
17:20 hdl Yes but even without any limit it is broken
17:21 kados what is the error?
17:22 hdl It works now.
17:23 kados just so you know, before I do a push, I have a list of about 25 tests i run
17:23 kados manually
17:23 kados to make sure it won't break anything
17:24 kados so I was very surprised it was broken for you
17:24 paul_ does those tests include a unimarc zebra config database ?
17:24 kados they could :-)
17:25 kados if you give me some samples to test
17:25 paul_ we do them for you, don't worry.
17:25 kados hdl: all seems to be working fine
17:25 kados hdl: perhaps it was just a file wasn't up to date?
17:26 kados if you find bugs with the current search, please let me know
17:26 kados (for zebra on staff client)
17:26 kados (no-zebra I'm testing now)
17:26 paul_ there is one change that we have to report to unimarc :
17:26 paul_ homebranch => branch
17:26 kados yes
17:26 kados sorry for the flip flop
17:26 paul_ is there one i could have missed ?
17:26 paul_ s/one/anotherone/
17:27 kados check the history of record.abs
17:27 kados but I don't think there was another one
17:27 paul_ s/wa/was/
17:28 kados mainly because users were confused
17:28 kados they don't know what a homebranch is :-)
17:28 kados (strange, I know)
17:29 kados paul/hdl: can I take a moment to explain the API changes and the reasons for them?
17:29 kados perhaps it could help in future troubleshooting
17:29 paul_ yes you can. just know that it's almost 6:30pm here...
17:29 kados ok, I'll be quick
17:30 paul_ I would also be interested by learning some things about weighting/stemming/fuzzing ...
17:30 paul_ as I don't know what to do to activate them
17:30 kados all of those can be activiated with sysprefs
17:30 paul_ yes I know that.
17:30 kados that hdl has told me I've neglected to add to updatedatabase
17:30 kados I will do that today
17:31 kados with defaults off
17:31 paul_ but when I do it, I get bugs ;-) (zebra errors mainly)
17:31 kados so for api
17:31 kados we have basically 5 types of queries:
17:31 paul_ ok, for api
17:31 kados 1. the query the user types in the search box
17:31 kados 2. the description of that query that's human readable
17:31 kados 3. the query expressed as a CGI
17:32 kados 4. the query expressed as a list of <input> parameters
17:32 kados 5. the internal query sent to zebra
17:32 kados in 2.2 we only recognized two of those
17:32 kados in dev_week we recognized three of them
17:32 kados and now in 3.0 we recognize all of them :-)
17:33 kados it's important to have all of them because depending on the operation performed by the user (such as clicking on a facet, or re-sorting, or going to next page)
17:33 kados the query must be created differently
17:33 kados otherwise, if we just use searchdesc, we may miss a sort_by param, or a offset= param
17:33 kados or a limit= param
17:33 kados make sense?
17:34 paul_ ok, make sense for me.
17:35 kados for stemming, weighting, etc.
17:35 kados I suspect you're missing some defs for zebra
17:35 paul_ I have rk
17:35 kados one thing I'd like to test with you is that we're all now using the same zebra conf files except for .chr and .abs
17:35 paul_ and rank:rank-1 in my cfg
17:36 paul_ maybe I will be able to test that tomorrow with the IPT DB
17:36 kados I'll be around early
17:36 kados and I'd be happy to help if I can
17:37 kados hdl: sorry for your nightmare earlier, I hope it wasn't my fault :/
17:40 kados paul_: with nozebra, are the indexes updated in real time? or batch?
17:40 paul_ real time
17:41 kados k, thx
17:41 paul_ (instead of filling zebraqueue table)
17:41 kados (got it)
17:41 paul_ time to leave for dinner (and some english lesson for Simon :-D )
17:41 kados hehe
17:41 kados have fun
17:50 kados hdl: did you write NZAnalyze?
18:21 hdl No paul did
18:26 hdl kados : but i wrote some bug fixes.
18:29 hdl In advsearch : you changed mc can you tell me why ?
18:30 kados it had been re-written since dev_week and didn't work properly
18:30 kados it didn't properly create nested queries
18:30 kados and (mc1 OR mc2 OR mc3)
18:30 kados etc
18:31 kados which is not the same as
18:31 kados and mc1 OR mc2 OR mc3
18:31 hdl Do you want help ?
18:32 kados yes please :-)
18:32 hdl (Some rewriting of Search.pm causes problems : when you remove beginning spaces of limits)
18:32 kados http://staff-jmf.dev.kohalibrary.com/cgi-bin/koha/catalogue/search.pl?q=Long&idx=au
18:32 hdl tell me what your problem is.
18:32 kados creates:
18:32 kados EXECUTE : biblioserver, author, Long
18:33 kados which is correct I think
18:33 kados but it doesn't return results
18:33 hdl surely, it is the fact that you donot have query_type set.
18:34 kados hmmm
18:34 kados where is query_type used?
18:34 kados (there's a query_type for nozebra?)
18:35 kados Search.pm line 1228
18:35 hdl no. I was wrong
18:35 kados my $sth = $dbh->prepare("SELECT biblionumbers,value FROM nozebra WHERE server=? AND indexname=? AND value $operator ?");
18:36 hdl Have you set up your indexes ?
18:36 hdl Do you have data in nozebra ?
18:37 hdl Do you have values in nozebraindexes system preferences ?
18:37 kados yep, I have data
18:37 kados nothing in nozebraindexes
18:37 kados I thought rebuild_nozebra.pl filled it?
18:38 kados select * from nozebra where value like "Long%";
18:38 kados biblioserver | __RAW__ | longest | 4,thebiggest-1;
18:38 kados biblioserver | __RAW__ | long | 2,howibecame-2;239,ayeardowny-1;362,thelongway-1;428,whatmaryjo-1;
18:39 kados all have __RAW__
18:39 hdl but _RAW_ is not OK for author search.
18:39 kados I guess that points to my nozebraindexes?
18:39 chris yep
18:39 hdl It will look for AU in your index.
18:39 kados right
18:39 hdl or au
18:40 hdl I have Nozebraindexes value but that suits UNIMARC.
18:40 kados yep
18:40 kados I'll just take values from the script
18:40 chris hmm i thought rebuild_nozebra.pl did it .. it did when i tried
18:41 hdl all you have to do is to take the index values from advsearch.pl and build a hash on your record.abs using ccl.properties equivalents.
18:41 kados *nod*
18:41 hdl then rebuild nozebra data.
18:42 kados but rebuild_nozebra.pl says it's supposed to do it
18:43 kados chris: yea, it's not doing it for me
18:43 kados must be a bug
18:43 hdl (Some of your rewriting of Search.pm causes problems : because you remove beginning spaces of limits)
18:43 chris 's' => \$sysprefs, # rebuild 'NoZebraIndexes' syspref
18:43 chris did you pass in the s ?
18:43 kados ahh, no
18:43 hdl what is proposed is a very simple indexation
18:44 chris give that a whirl and see if that makes it go
18:44 kados yea, that worked
18:44 chris in theory it should do it if its empty or if you pass in -s
18:44 kados hdl: can you give me an example?
18:50 kados hdl: doesn't look like nozebra pays attention to the limit, does it?
18:54 kados hdl: in the nozebra code there is some terminology confusion
18:55 kados hdl: operator/operand aren't identified correctly
18:55 kados index too
18:58 kados strange, it seems 'kw' doesn't search everything
18:59 kados and facets don't seem to work with nozebra
19:00 paul_away (just passing around : I confirm that facets don't work with NZ)
19:00 kados paul_away: is that by design?
19:00 kados paul_away: or a bug?
19:03 kados paul_away: also, can you please fix the variable names in the nozebra code? operator/operand/index aren't used correctly
19:03 kados for instance, operator = 'and/or/not'
19:04 kados operand = 'particules' || 'particules or melinda'
19:04 kados index = kw
19:04 kados I'm afraid anyone reading that code will be very confused
19:45 [K] *** part FreeNode!#koha: ru55el n=russel@203-118-134-114.netspace.net.nz
19:50 [K] *** join #koha@FreeNode: ru55el n=russel@203-118-134-114.netspace.net.nz
21:00 masonj .
03:32 fbcit-1 hello #koha
03:33 fbcit-1 chris: you around?
03:34 fbcit-1 kados?
03:35 fbcit-1 paul_?
08:58 paul chris around ?