Time  Nick   Message
11:55 kados  did rach have her baby?
11:56 kados  still four days till she's due right?
11:56 paul   hello joshua. Good question. no news about this.
11:56 kados  hi paul :-)
11:56 paul   (38)
11:56 kados  happy birthday :-)
11:57 paul   and don't think error logs are clear...
11:57 kados  what kinds of problems?
11:57 paul   (clear : easy to understand)
11:57 paul   (wrong translation i think here)
11:57 kados  (I understood :-)
11:58 paul   problems like having no error messages, a rebuild_zebra working really fast, but a small small zebra DB that makes me think i've sometihng wrong
11:58 kados  paul: have you started zebra with the -vall option?
11:58 paul   (mail on koha-zebra when nongnu.org choose to distribute it ;-) )
11:58 kados  it shows more logs that way
11:59 paul   yes, but no usefull info for me. I need some help to understand them i'm afraid
11:59 kados  http://www.indexdata.dk/yaz/doc/server.invocation.tkl
12:00 kados  I think those work in Zebra as well as Yaz
12:00 paul   what is PDU ?
12:01 kados  it's the actual transaction between two Z3950 sources (server/client)
12:02 kados  as close to human readable version of the Z3950 protocol as you can get
12:02 osmoze bonne anniversaire en retard alors Paul :)
12:02 kados  so you start with a initRequest { }
12:02 kados  then a initResponse { }
12:02 kados  then searchRequest { }
12:03 kados  then searchResponse { }
12:03 kados  etc.
12:03 kados  paul: a PDU dump is equiv to a tcpdump for TCP
12:05 kados  hehe
12:05 kados  PDU acronym for Protocol Data Unit ... but that's not a useful acronym IMO
12:06 kados  paul: do you have an example PDU dump I can look at?
12:06 paul   what does it look like ?
12:06 paul   I have some zebrasrv logs.
12:06 kados  well ... first you probably must specify a file for Zebra to dump PDU using  the -a flag
12:07 kados  PDU dumps are very long :-)
12:07 kados  here's the init section from one on my server:
12:07 kados  initRequest {
12:07 kados      protocolVersion BITSTRING(len=1)
12:07 kados      options BITSTRING(len=2)
12:07 kados      preferredMessageSize 1048576
12:07 kados      maximumRecordSize 1048576
12:07 kados      implementationId '81'
12:07 kados      implementationName 'ZAP/YAZ'
12:07 kados      implementationVersion '1.4.5/2.0.34'
12:07 kados  }
12:07 kados  initResponse {
12:07 kados      protocolVersion BITSTRING(len=1)
12:07 kados      options BITSTRING(len=2)
12:07 kados      preferredMessageSize 1048576
12:07 kados      maximumRecordSize 1048576
12:08 kados      result TRUE
12:08 kados      implementationId '/81'
12:08 kados      implementationName '/GFS/YAZ'
12:08 paul   OK, OK
12:08 kados      implementationVersion '/1.65/2.1.10'
12:08 kados  }
12:08 kados  initRequest comes from client Z3950 (ie perl-zoom in your case, yaz in the above case)
12:08 kados  initResponse comes from the server (Zebra in your case, SimpleServer in the above case)
12:16 |hdl|  happy birthday paul.
12:19 paul   kados : copy of the mail i send 1mn ago on koha-zebra :
12:19 paul   Paul POULAIN a écrit :
12:19 paul   /me stupid, pls ignore the previous mail (with the loooooonnnnnnnggggggggg log)
12:19 paul   If I do :
12:19 paul   while ((my $biblionumber) = $sth->fetchrow) {
12:19 paul       my $record = XMLgetbiblio($dbh,$biblionumber);
12:19 paul       my $Zpackage = $Zconn->package();
12:19 paul       $Zpackage->option(databaseName => 'Koha');
12:19 paul       $Zpackage->option(action => "recordInsert");
12:19 paul       $Zpackage->option(record => $record);
12:19 paul       $Zpackage->send("update");
12:19 paul       $Zpackage->destroy;
12:19 paul       $i++;
12:19 paul       print '.';
12:19 paul       print "$i\r" unless ($i % 100);
12:19 paul   }
12:19 paul   (remove the eval { } on ->send() )
12:19 paul   I get a :
12:19 paul   ZOOM error 10002 "Encoding failed" from diag-set 'ZOOM'
12:19 paul   (and a die)
12:19 paul   Thus, I still have problems with the record model :-(
12:23 kados  encoding failed eh?
12:26 kados  paul: http://lists.indexdata.dk/pipermail/tkl/2004-February/000441.html
12:27 kados  quoting: "The "encoding failed at target" error might smell like an yaz version
12:27 kados  before 2.0.6 - that is the library /usr/lib/libyaz should come from a
12:27 kados  debian package version 2.0.6 or 2.0.7. Check version number, and try
12:27 kados  again with a newer one,
12:28 paul   yaz-client -V
12:28 paul   YAZ version: 2.1.10
12:28 kados  well, probably not that then
12:33 kados  well looks like "Encoding failed" is a ZOOM diagnostic set which is used for client-side error
12:34 kados  http://lists.indexdata.dk/pipermail/zoom/2003-March.txt
12:34 kados  do a search on 'encoding failed' in that page
12:34 kados  morning owen
12:34 owen   Hi kados
12:35 paul   (on phone)
13:03 paul   back
13:04 paul   kados : i don't understand where it is useful (i mean 2003-March.txt)
13:05 kados  search on 'encoding failed' on that page
13:06 kados  it describes what kind of error that is (IE, it's a ZOOM client-side error)
13:06 kados  not too useful I agree
13:06 paul   Yes, i've seen, but i don't understand
13:56 kados  paul: do you have more errors in the log?
13:56 paul   nope
13:56 paul   just this ZOOM error 10002 "Encoding failed" from diag-set 'ZOOM
13:56 paul   and PDU files show nothing.
13:56 kados  here are the bib-1 diagnostics codes:
13:56 kados  http://www.loc.gov/z3950/agency/defns/bib1diag.html
13:56 kados  but I don't see a 10002 in there
13:57 paul   I already looked at this page (or a similar one, with the same 10002 missing value)
13:57 kados  right ... perhaps it's an index data code
13:58 paul   I really become mad with zebra.cfg & other files.
13:58 paul   my main problem is that i can't be sure WHICH is the culprit : my code, my zebra config, zoom, zebra...
13:58 kados  yep ... that is very frustrating
13:59 paul   highly !
13:59 paul   i won't be here for 2 days, back on friday.
13:59 kados  I would suspect that Perl-ZOOM is the culprit
13:59 kados  paul: could you commit your (non-working) code
13:59 kados  paul: I may have some time to work on it
13:59 paul   I'll commit my modified rebuild_zebra & zebra.cfg
13:59 paul   (you won ;-) )
14:00 kados  hehe
14:00 kados  still no message on koha-devel ... savannah seems slower than sf :-)
14:02 paul   for CVS no !
14:03 paul   OK, commited
14:03 paul   new sub in Biblio.pm, + unimarc zebra config files + misc/migration_tools/rebuild_zebra.pl
14:03 kados  paul: why are we pulling XML out of mysql?
14:04 paul   ?
14:04 kados  paul: shouldn't it just be in Zebra?
14:04 kados  sub XMLgetbiblio {
14:04 kados  my $sth = $dbh->prepare('select marcxml from biblioitems where
14:04 kados  biblionumber=?');
14:04 paul   I don't know. Still investigating for instance.
14:04 kados  in my opinion, we should only have one copy of the data
14:04 kados  not one in mysql and one in zebra
14:05 kados  (because zebra must have one for the index building anyway)
14:05 paul   the problem with this is backing/restoring datas : 2 sources !
14:05 kados  and it's hard to know if they are both in sync
14:05 kados  I vote to store all bib data in zebra
14:05 paul   (+ we could also have storeData:0, to have datas not in zebra )
14:05 kados  hmmm
14:06 paul   I think we should keep datas in both place until we are closer to 3.0 than today.
14:06 kados  ok ... sounds reasonable
14:06 kados  are you sure that mysql has utf-8 data?
14:06 paul   (+ in fact, for instance, we have 3 instances of biblio : biblioitems.marc, biblioitems.xml and zebra)
14:06 kados  (in your example on koha-devel message I just recieved)
14:07 kados  yikes :-)
14:07 paul   I think yes. Not 100% sure
14:07 kados  because 'encoding problem' may indicate no :-)
14:08 paul   dumping the xml record shows :
14:08 paul   Collection Pédagogique de Télécommunicatio
14:08 paul   that seems utf-8
14:09 paul   (Collection P##dagogique => the ## is a é in utf-8)
14:10 kados  paul: http://dev.mysql.com/doc/refman/5.0/en/charset-defaults.html
14:11 kados  it seems encoding in mysql is complex
14:12 paul   yes, i've spend almost 2 complete day on it !
14:12 paul   but I think now i've got it
14:12 kados  here is another useful site:
14:12 kados  http://dev.mysql.com/tech-resources/articles/4.1/unicode.html
14:12 kados  what version of mysql are you using?
14:12 paul   4.1
14:14 kados  also, I wonder if perl is to blame :-)
14:14 kados  I don't know what DBI's support of utf-8 is like
14:15 paul   my bet is that DBI has nothing to do with utf8
14:15 kados  http://www.cpanforum.com/threads/654
14:16 paul   did you read my mail jan, 04, on koha-devel ?
14:16 paul   the chapter "the hard thing" was the main trick
14:17 kados  right
14:21 kados  paul: can you generate a utf-8 static MARC file and attempt to import it to see if you ge the same error?
14:25 paul   I tried, with a record with ASCII only char, same problem
14:29 paul   time to leave (6:30 PM)
14:29 paul   see you on friday
14:29 paul   (my 1st mail has arrived on koha-zebra ! another one has been posted. You can ignore this one
14:30 paul   )
14:30 paul   2nd one arrived too ;-)
14:39 kados  by paul ... have a good week
06:03 si     heh
06:39 _hdl_  Congratulations rach he is VERY cute.
06:39 si     he certainly is
06:39 si     but I'd be biased :-)
07:01 _hdl_  you can be proud and happy ;)
07:04 si     yes indeed