| Time |
S |
Nick |
Message |
| 08:30 |
|
paul |
kados around ? |
| 08:43 |
|
kados |
paul: yep |
| 08:43 |
|
paul |
hello kados |
| 08:43 |
|
kados |
hi :-) |
| 08:44 |
|
paul |
do you have a minut for a quick question ? |
| 08:44 |
|
kados |
yep |
| 08:44 |
|
paul |
I have indexed a large (50 000 biblios) with zebra |
| 08:44 |
|
paul |
(& a new rebuild_zebra script i'll commit soon) |
| 08:44 |
|
paul |
but... |
| 08:44 |
|
kados |
update_items already rebuilds the zebra index |
| 08:45 |
|
kados |
if you pass it the -all parameter |
| 08:45 |
|
paul |
when I search I get results only if I enter something that is in the title. |
| 08:45 |
|
paul |
sample : |
| 08:45 |
|
paul |
keyword chaos => returns 10 biblios |
| 08:45 |
|
paul |
author chaos => returns the same 10 biblios |
| 08:46 |
|
paul |
keyword : <an author> => returns nothing |
| 08:46 |
|
paul |
author <an author> => nothing |
| 08:46 |
|
kados |
there are so many possible reasons for that :-) |
| 08:46 |
|
paul |
stemming=0, field_weightiing=0 too |
| 08:46 |
|
kados |
it could be the zebra conf files |
| 08:46 |
|
paul |
yes, but I have a more precise question |
| 08:46 |
|
kados |
ok |
| 08:47 |
|
paul |
I uncommented buildQuery last lines : |
| 08:47 |
|
paul |
warn "QUERY:".$koha_query; |
| 08:47 |
|
paul |
warn "SEARCHDESC:".$human_search_desc; |
| 08:47 |
|
paul |
warn "FEDERATED QUERY:".$federated_query; |
| 08:47 |
|
kados |
it would be useful to see what the query is |
| 08:47 |
|
paul |
and get : |
| 08:47 |
|
paul |
[Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] QUERY:chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 3118., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl |
| 08:47 |
|
paul |
[Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] SEARCHDESC:au: chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 3119., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl |
| 08:47 |
|
paul |
[Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] FEDERATED QUERY:au= chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 3120., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl |
| 08:47 |
|
paul |
[Mon Oct 23 14:40:25 2006] [error] [client 192.168.0.2] QUERYING : chaos at /home/paul/koha.dev/rel_3_0/C4/Search.pm line 2776., referer: http://i18.bureau.paulpoulain.[…]talogue/search.pl |
| 08:47 |
|
paul |
QUERY:chaos seems wrong to me. |
| 08:47 |
|
kados |
why? |
| 08:48 |
|
kados |
it will use the default |
| 08:48 |
|
paul |
should be related to author somewhere no ? |
| 08:48 |
|
kados |
ahh, yes |
| 08:48 |
|
paul |
(because is was "author : chaos" |
| 08:48 |
|
kados |
right |
| 08:48 |
|
paul |
what should I see here ? |
| 08:48 |
|
kados |
on dev_week with MARC21 |
| 08:48 |
|
kados |
au:chaos |
| 08:49 |
|
paul |
same as SEARCHDESC so ? |
| 08:49 |
|
kados |
(but do you have an author with 'chaos'? |
| 08:49 |
|
kados |
yes |
| 08:49 |
|
owen |
Hi everybody |
| 08:49 |
|
kados |
hi owen |
| 08:49 |
|
paul |
nope. but I get answers anyway. |
| 08:49 |
|
kados |
it should be au:chaos |
| 08:52 |
|
kados |
paul: are you sure toins has not modified buildQuery? |
| 08:52 |
|
paul |
unless he made something wrong no. |
| 08:53 |
|
paul |
(could stemming=0 & field_weight=0 change sometihng ?) |
| 08:53 |
|
kados |
I do see one possible problem |
| 08:54 |
|
kados |
see this clause: |
| 08:54 |
|
kados |
else { |
| 08:54 |
|
kados |
$query.=" $operand"; |
| 08:54 |
|
kados |
if (!$index) { |
| 08:54 |
|
kados |
around like 3265 |
| 08:54 |
|
kados |
line even |
| 08:55 |
|
kados |
maybe $query.=" $operand"; should be: |
| 08:55 |
|
kados |
$query.="$index: $operand"; |
| 08:55 |
|
kados |
but I'm not 100% sure, I'd have to test it |
| 08:55 |
|
paul |
i'll test & let you know |
| 08:56 |
|
kados |
(in fact, if it works, it should be modified slightly to test for the existance of $index first |
| 08:56 |
|
kados |
$query.=" $index: $operand"; would be safer (adds a space) |
| 09:05 |
|
paul |
kadose : about my script that rebuild zebra : it does many things : |
| 09:05 |
|
paul |
s/kadose/kados/ |
| 09:05 |
|
paul |
it : |
| 09:05 |
|
paul |
- create the directory structure |
| 09:05 |
|
paul |
- copies the needed files from zebraplugin into the directory structure |
| 09:06 |
|
paul |
- creates the zebra-biblios.cfg with the dir structure too |
| 09:06 |
|
paul |
- dump the records |
| 09:06 |
|
paul |
- zebraidx |
| 09:06 |
|
paul |
so, once you have a koha.xml defined, just run it, it does what is needed where it is needed |
| 09:06 |
|
paul |
(& takes care of record.abs depending on your marc flavour) |
| 09:07 |
|
kados |
cool |
| 09:07 |
|
kados |
so you only run it once, on installation? |
| 09:08 |
|
kados |
or is it meant to be run every time you re-index? |
| 09:09 |
|
paul |
you can do whatever you want, it creates what is needed. |
| 09:10 |
|
paul |
there is even a parameter not to dump your file & preserve your export, when you want to test things |
| 09:47 |
|
paul |
kados : something is wrong with my setup : i can't do any search on something except title/subtitle : "find caroll" in yaz-client don't work, while "find chaos" works fine. |
| 09:48 |
|
paul |
this is not koha related, this is probably record.abs |
| 09:50 |
|
kados |
yep, sounds like it |
| 09:50 |
|
kados |
there are several places to check: |
| 09:50 |
|
paul |
in yaz-client find "ti=chaos" should return records with title=chaos, right ? |
| 09:51 |
|
kados |
record.abs, bib1.att, ccl.properties |
| 09:51 |
|
dewey |
record.abs, bib1.att, ccl.properties are the most important files |
| 09:51 |
|
paul |
(because it don't work either. only "find chaos" returns something) |
| 09:51 |
|
kados |
paul: ti=chaos will work if you specified you're using ccl |
| 09:51 |
|
paul |
yep : yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties @:9900/biblios is enough right ? |
| 09:51 |
|
kados |
yep |
| 09:52 |
|
paul |
ccl.properties contains : ti u=4 s=pw |
| 09:52 |
|
kados |
(and by the way, I've been starting zebra on a unix socket these days, much faster even) |
| 09:52 |
|
paul |
bib1.att : att 4 Title |
| 09:52 |
|
kados |
ok, and record.abs for Title? |
| 09:52 |
|
paul |
melm 200$atitle:w,title:p,title:s |
| 09:52 |
|
paul |
melm 200$ctitle:w,title:p,title:s |
| 09:52 |
|
paul |
melm 200$dtitle:w,title:p,title:s |
| 09:52 |
|
paul |
# subtitle |
| 09:53 |
|
paul |
melm 200$etitle:w,title:p,title:s |
| 09:53 |
|
paul |
# authorities title |
| 09:53 |
|
paul |
melm 500title |
| 09:53 |
|
paul |
melm 501title |
| 09:53 |
|
paul |
melm 503title |
| 09:53 |
|
paul |
melm 510title |
| 09:53 |
|
paul |
melm 512title |
| 09:53 |
|
paul |
melm 513title |
| 09:53 |
|
paul |
melm 514title |
| 09:53 |
|
paul |
melm 515title |
| 09:53 |
|
paul |
melm 516title |
| 09:53 |
|
paul |
melm 517title |
| 09:53 |
|
paul |
melm 518title |
| 09:53 |
|
paul |
melm 519title |
| 09:53 |
|
paul |
melm 520title |
| 09:54 |
|
paul |
melm 530title |
| 09:54 |
|
paul |
melm 531title |
| 09:54 |
|
paul |
melm 532title |
| 09:54 |
|
paul |
melm 540title |
| 09:54 |
|
paul |
melm 541title |
| 09:54 |
|
paul |
melm 545title |
| 09:54 |
|
kados |
looks fine |
| 09:54 |
|
paul |
oups. not kicked... |
| 09:54 |
|
kados |
hehe |
| 09:54 |
|
paul |
and for authors : |
| 09:54 |
|
paul |
record.abs : melm 200$fauthor,author:p,author:s |
| 09:54 |
|
paul |
ccl.properties : au u=1003 s=pw |
| 09:54 |
|
paul |
bib1.att : att 1003 Author |
| 09:54 |
|
paul |
that's my feeling... |
| 09:54 |
|
paul |
hehe what ? |
| 09:54 |
|
kados |
one thing |
| 09:54 |
|
kados |
author vs Author |
| 09:54 |
|
kados |
not sure it matters |
| 09:55 |
|
paul |
your record.abs (the marc21 one on CVS) is case-variant. |
| 09:55 |
|
paul |
so I thought it did not matter |
| 09:55 |
|
kados |
ok, then it doesn't :-) |
| 09:55 |
|
kados |
hmmm |
| 09:55 |
|
paul |
maybe i'm using a file i'm not looking at... |
| 09:56 |
|
kados |
can you confirm that zebra is indexing using the right config files? |
| 09:56 |
|
kados |
check zebra-biblios.cfg tab file |
| 09:56 |
|
paul |
profilePath:${srcdir:-.}:/home/zebradbs/rel_3_0/biblios/tab/:/usr/local/share/idzebra/tab/:${srcdir:-.}/tab/ |
| 09:57 |
|
kados |
I would limit it to just the one dir |
| 09:57 |
|
kados |
I ahve: |
| 09:57 |
|
kados |
profilePath:/koha/zebradb/biblios/tab/ |
| 10:03 |
|
paul |
kados : things are a little better : |
| 10:03 |
|
paul |
find <title> works |
| 10:03 |
|
paul |
find <author> works |
| 10:03 |
|
paul |
but find "ti=<title>" does not work |
| 10:05 |
|
kados |
do you have a line in ccl.properties: |
| 10:05 |
|
kados |
ti title |
| 10:05 |
|
kados |
(ie, an alias ) |
| 10:05 |
|
paul |
nop, just ti u=4 s=pw |
| 10:06 |
|
kados |
ahh |
| 10:06 |
|
kados |
that'll work too |
| 10:07 |
|
paul |
mmm... my ccl.properties is really different from the dev_week one (beeping toins) |
| 10:10 |
|
paul |
something seems wrong with CVS ... |
| 10:11 |
|
kados |
what's happening? |
| 10:12 |
|
paul |
my ccl.properties from dev_week is 1020 lines long, & toins one is less than 100. |
| 10:12 |
|
paul |
both are said "uptodate" |
| 10:13 |
|
kados |
paul: in the same repo? |
| 10:13 |
|
kados |
cm: did I ever respond to your email last week? |
| 10:14 |
|
cm |
hi kados. no, you didn't. :) |
| 10:14 |
|
paul |
hello cm |
| 10:14 |
|
kados |
ahh yes ... here it is :-) |
| 10:14 |
|
kados |
cm: I didn't know you were using bulkmarcimport from dev_week |
| 10:14 |
|
cm |
hi paul! |
| 10:14 |
|
kados |
cm: I will fix it immediately |
| 10:14 |
|
cm |
yeah, i think that's what I used the last time i did a full import. |
| 10:15 |
|
cm |
thanks! |
| 10:15 |
|
cm |
lemme know when you're done. |
| 10:15 |
|
kados |
cm: (I thought you were going through rel_2_2's bulkmarcimport ... has that changed?) |
| 10:16 |
|
cm |
i think the first time i installed dev_week I used it, but I redid it a few weeks ago and I think I forgot and used the one in dev_week. |
| 10:17 |
|
cm |
I tried it with rel_2_2, which worked, but nothing was imported into biblioitems.marc. Is that data needed, or will it be? |
| 10:17 |
|
kados |
well, since you were going by way of rel_2_2 before |
| 10:18 |
|
kados |
the way to populate biblioitems.marc is with import_to_biblioitems script |
| 10:18 |
|
cm |
is that in 2_2 or dev_week? |
| 10:18 |
|
kados |
lemme check |
| 10:19 |
|
kados |
oops gave you the wrong name |
| 10:19 |
|
cm |
for what? |
| 10:19 |
|
kados |
zebraplugin/utils/move_marc_to_biblioitems.pl |
| 10:19 |
|
kados |
that's what you need to update a rel_2_2 system to dev_week |
| 10:19 |
|
kados |
ie, move from the marc-* tables to biblioitems.marc |
| 10:20 |
|
cm |
ah. okay. |
| 10:20 |
|
kados |
cm: committed bulkmarcimport |
| 10:20 |
|
cm |
but I don't have marc-*tables anymore (except the structure ones) |
| 10:20 |
|
cm |
i think. |
| 10:20 |
|
cm |
cool. I'll try it out. :D |
| 10:22 |
|
kados |
cm: so is your plan to only use dev_week from the get-go now? |
| 10:22 |
|
kados |
cm: for importing I mean |
| 10:22 |
|
cm |
very likely. as long as it works. :) |
| 10:23 |
|
cm |
i'm turning off zebra, though and uploading into zebra after import. |
| 10:23 |
|
cm |
otherwise it takes too long. |
| 10:25 |
|
kados |
yep, makes sense |
| 10:26 |
|
cm |
i'm very glad! |
| 10:26 |
|
kados |
I've got a new version of update_items that will rebuild the zebra index automagically |
| 10:26 |
|
kados |
I'll commit it now |
| 10:26 |
|
cm |
interesting. |
| 10:26 |
|
kados |
oh, and cm, you should upgrade to the latest zebra version in the 1.3 series |
| 10:26 |
|
kados |
released this morning |
| 10:27 |
|
cm |
good to know. i'll do that too. |
| 10:27 |
|
kados |
it has a bugfix for a nasty multiple simultaneous updates over extended services |
| 10:27 |
|
kados |
bug |
| 10:27 |
|
kados |
:-) |
| 10:27 |
|
kados |
discovered by yours truly :-) |
| 10:30 |
|
cm |
congrats. bugfixes are good! |
| 10:30 |
|
kados |
misc/update_items.pl |
| 10:31 |
|
kados |
will probably need some customization |
| 10:31 |
|
kados |
but i run that with the -today flag for NPL in cron every evening |
| 10:31 |
|
kados |
and if I need to rebuild the index with the -all flag |
| 10:32 |
|
kados |
let me know if the docs aren't clear in that file |
| 10:32 |
|
cm |
okay. thanks. |
| 10:40 |
|
cm |
uh oh. I got two errors: |
| 10:40 |
|
kados |
bulkmarcimport errors? |
| 10:40 |
|
cm |
DBD::mysql::st execute failed: Unknown column 'ccode' in 'field list' at /usr/local/koha/intranet/modules/C4/Biblio.pm line 1957. problem with Zebra at /usr/local/koha/intranet/modules/C4/Context.pm line 490. |
| 10:40 |
|
kados |
right |
| 10:40 |
|
cm |
yeah. :/ |
| 10:40 |
|
kados |
that's an easy fix |
| 10:40 |
|
cm |
the other was: |
| 10:41 |
|
cm |
ZOOM error 10000 "Connect failed" (addinfo: "tcp:@:9900") from diag-set 'ZOOM'root koha:/usr/local/koha_cvs/dev_week/koha-new-upload/koha/misc/migration_tools |
| 10:41 |
|
kados |
hmmm |
| 10:41 |
|
cm |
but I think that can probably be ignored. i remember seeing something like that before |
| 10:41 |
|
kados |
that doesn't look good |
| 10:41 |
|
kados |
but if you're not using zebra it shouldn't matter |
| 10:41 |
|
kados |
ok ... so lemme check the wiki |
| 10:42 |
|
kados |
I'll add the sql you need |
| 10:43 |
|
cm |
good. thanks. |
| 10:44 |
|
kados |
cm: can you check your biblioframework.sql file? |
| 10:44 |
|
kados |
make sure it has: alter table kohanew.biblioitems add ccode varchar(4); |
| 10:45 |
|
cm |
ok. let me look. |
| 10:46 |
|
paul |
kados : is there a reason to have, in koha.xml : |
| 10:47 |
|
paul |
<ccl2rpn>/home/zebradbs/rel_3_0/biblios/etc/ccl.properties</ccl2rpn> in <serverinfo id="biblioserver"> section |
| 10:47 |
|
kados |
yes |
| 10:47 |
|
paul |
while <cql2rpn>/home/zebradbs/rel_3_0/biblios/etc/pqf.properties</cql2rpn> is in <server id="..."> |
| 10:47 |
|
kados |
because zebra complains if there are unrecognized elements in <server element |
| 10:47 |
|
kados |
so we need one for zebra, and one for extra stuff |
| 10:48 |
|
kados |
(it's not ideal, I agree) |
| 10:48 |
|
paul |
zebra undestand cql2rpn but NOT ccl2rpn ? |
| 10:48 |
|
paul |
strange |
| 10:48 |
|
kados |
right |
| 10:48 |
|
kados |
maybe there is a way to tell it about ccl2rpn, but I don't know it |
| 10:48 |
|
cm |
do you mean utils/biblio_framework.sql? that doesn't have it. the latest cvs I just downloaded doesn't have it in there either. |
| 10:48 |
|
kados |
cm: ok, I'll update it |
| 10:48 |
|
paul |
another question : |
| 10:49 |
|
paul |
yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties @:9900/biblios |
| 10:49 |
|
cm |
zehr gut. :) |
| 10:49 |
|
paul |
querytype ccl |
| 10:49 |
|
paul |
find "ti=dogmatique" |
| 10:49 |
|
paul |
gives me a : |
| 10:49 |
|
paul |
[107] Query type not supported -- v2 addinfo 'type-2' |
| 10:49 |
|
paul |
which let me think he does not support ccl |
| 10:49 |
|
kados |
nope |
| 10:50 |
|
kados |
that means that attribute type 2 isn't supported |
| 10:50 |
|
kados |
it should be specified in your record.abs |
| 10:50 |
|
kados |
paul: just a sec ... lemme commit this file |
| 10:51 |
|
paul |
no prob |
| 10:52 |
|
kados |
cm: ok, update and profit ;-) |
| 10:52 |
|
kados |
paul: so lets trace what zebra is doing here |
| 10:52 |
|
kados |
paul: first, you say 'ti=dogmatique' |
| 10:52 |
|
kados |
and I assume you're using a ccl specification, right? |
| 10:52 |
|
kados |
so check ccl.properties |
| 10:53 |
|
kados |
for the ti line and tell me what it is |
| 10:53 |
|
paul |
ti u=4 |
| 10:53 |
|
paul |
title ti |
| 10:53 |
|
paul |
(i don't have reindexed after adding title ti alias, but i don't think it matters) |
| 10:53 |
|
kados |
you changed it! |
| 10:53 |
|
kados |
before you had ti u=4 s=something |
| 10:54 |
|
kados |
right? |
| 10:54 |
|
paul |
right. |
| 10:54 |
|
paul |
I tried both, without any difference |
| 10:54 |
|
paul |
ti u=4 s=pw |
| 10:54 |
|
kados |
ahh, I missread your error |
| 10:55 |
|
kados |
you must start yaz liek this; |
| 10:55 |
|
kados |
yaz-client -c /path/to/ccl.properties |
| 10:55 |
|
paul |
yes : |
| 10:55 |
|
paul |
[16:49] <paul> yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties @:9900/biblios |
| 10:56 |
|
kados |
try this: |
| 10:56 |
|
kados |
yaz-client -c /home/zebradbs/rel_3_0/biblios/etc/ccl.properties |
| 10:56 |
|
kados |
> open localhost:9900/biblios |
| 10:56 |
|
cm |
kados: thanks! I let you know how it works shortly. |
| 10:56 |
|
kados |
> f ti=something |
| 10:57 |
|
paul |
Z> open localhost:9900/biblios |
| 10:57 |
|
paul |
Connecting...OK. |
| 10:57 |
|
paul |
Sent initrequest. |
| 10:57 |
|
paul |
Connection accepted by v3 target. |
| 10:57 |
|
paul |
ID : 81 |
| 10:57 |
|
paul |
Name : Zebra Information Server/GFS/YAZ |
| 10:57 |
|
paul |
Version: Zebra 1.3.32/1.66/2.1.12 |
| 10:57 |
|
paul |
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets |
| 10:57 |
|
paul |
Elapsed: 0.002776 |
| 10:57 |
|
paul |
Z> f ti=dogmatique |
| 10:57 |
|
paul |
Sent searchRequest. |
| 10:57 |
|
paul |
Received SearchResponse. |
| 10:57 |
|
paul |
Search was a success. |
| 10:57 |
|
paul |
Number of hits: 0, setno 1 |
| 10:57 |
|
paul |
SearchResult-1: term=ti dogmatique cnt=0 |
| 10:57 |
|
paul |
records returned: 0 |
| 10:57 |
|
paul |
Elapsed: 0.006193 |
| 10:57 |
|
paul |
(note that I have zebra 1.3, not 2.0) |
| 10:58 |
|
kados |
I have the same prob |
| 10:58 |
|
kados |
just a sec |
| 10:59 |
|
kados |
maybe we need to say: |
| 10:59 |
|
kados |
querytype ccl |
| 10:59 |
|
kados |
or better |
| 10:59 |
|
kados |
querytype ccl2rpn |
| 10:59 |
|
kados |
yep, that works |
