Time Nick Message
16:46 slef http://www.jet2.com/News.aspx?id=65 "Jet2.com [...] calls for lazy frogs to get back to work!"
16:46 slef Jet2 - I can't believe it's an airline.
16:46 owen Charming
16:47 owen chris, you around?
22:51 russ cos the other channel is too busy
22:51 chris now in koha?
02:25 hdl hi pierrick
02:26 pierrick hi hdl & paul
02:40 paul plein de français ce matin encore...
02:40 paul osmoze ?
02:41 paul est-ce que tu as eu le temps de tester l'install windows ?
02:41 osmoze bonjour
02:41 paul bonjour d'abord, bonne remarque !
02:41 osmoze j avais commencer mais pas terminé...Tiens, tu me choppes au reveille, j ai rien prevu ce matin, j attaque :)
02:42 osmoze (en fait, j ai fais l install avec phpmyadmin et au final pb de perl :()
03:14 paul pierrick hésite beaucoup ce matin encore...
03:14 paul a t'il l'utilité d'un _ ou pas ?
03:14 paul bonne question...
03:15 pierrick désolé, problemes de connexion internet chez INEO
03:15 paul je sais je sais. Ca m'amuse juste ;-)
03:16 paul (et ton login apparait chez moi en vert pale, du coup le _ est difficile à voir. donc faut que je fasse attention si je veux te "bipper"
03:16 pierrick tu utilises quoi comme client IRC ?
03:16 pierrick parce que irssi ou Xchat font l'autocomplétion des nicks présent sur le chanel.
03:16 pierrick s/chanel/channel/
03:17 paul pierrick: ah, c'est vrai, KSirc aussi
03:17 paul mais j'y pense jamais !
03:17 pierrick paul, tu as combien de stopwords dans ta base de dev HEAD?
03:22 paul ca dépend de la base.
03:22 paul plusieurs.
03:22 paul (dans la plupart des cas, j'ai ce qui est mis par défaut à l'install.
03:22 paul ca doit faire une 50aine de mots
03:23 pierrick OK, je suis en train de reprendre la page de gestion des stopwords
03:24 pierrick en partant du principe que les stopwords sont courts et qu'on peut faire peut d'opération dessus. Tu me diras ce que tu en penses. (initialement c'était juste pour ajouter la barre de pagination)
03:24 paul OK.
03:24 paul quoi que : STOP
03:24 paul ne perd pas de temps là dessus. Les mots vides seront inutiles dans Koha 3.0 probablement
03:24 paul vu que la recherche srea gérée par zebra !
03:24 paul qui utilise d'autres mécanismes.
03:24 paul quoi que finalement :
03:25 paul les mots vides sont à mettre dans un fichier à part. Peut être que l'on génèrera ce fichier à partir de la table stopwords.
03:25 paul bref, je ne suis pas trop sûr.
03:25 paul mais ne passe pas 1 journée là dessus ;-)
03:28 pierrick non, pas un journée, je te rassure, de toute façon, je vais chez le fameux client dans une heure
03:28 paul "chez le fameux client" ?
03:29 pierrick un gros client d'INEO
04:40 osmoze hdl> ?
04:41 osmoze paul> ?
04:42 paul ye
04:42 paul yep
04:42 osmoze non c edst bon paul merci :)
04:43 osmoze (enfin désolé ^^)
07:13 osmoze hdl ?
07:17 hdl salut
07:40 kados morning all
07:40 kados paul: do you have a quick second?
07:41 kados paul: I'm importing some records and I'm afraid they are truncating
07:41 kados paul: with bulkmarcimport
07:41 kados I get the following error in the log several hundred times:
07:41 kados no mapping found at position 8 in Price : ~\ 36.00; Inv.# CSI102254; Date 30/11/98; Supplier : CroftHouse, Surrey, UK; Recd 11/12/98; Contents : 1. Bringing your technology to market; 2. So do you really have something of value? 3. The first steps towards commercialisation of your technology; 4. The difficult problem of valuation of intellectual property; 5. Developing your ideas; 6. The licensing option; 7. Forming your own company; 8. Financing the busines
07:45 paul hi kados.
07:45 paul "no mapping found at position 8".
07:45 paul I never saw this before.
07:45 paul could it be a MARC::charset problem ?
07:45 paul ("mapping" is sometihng MARC::charset does)
07:46 paul (thus my idea)
07:49 kados yep, could be
07:49 kados just wondering if you had seen it before
07:49 paul so, the answer is no ;-)
07:49 kados :-)
07:50 paul was it iso2790 ?
07:51 kados yes
07:51 kados and claims to be MARC-8
07:51 kados encoded
07:52 kados the record imported fine
07:52 kados but the data in 856 was discarded by bulkmarcimport
07:52 kados http://wipoopac.liblime.com/cgi-bin/koha/opac-MARCdetail.pl?bib=24436
07:52 kados 856 contained:
07:52 kados 856 _aPrice : <9C> 36.00; Inv.# CSI102254; Date 30/11/98; Supplier : C
07:52 kados roftHouse, Surrey, UK; Recd 11/12/98; Contents : 1. Bringing your technology t
07:52 kados o market; 2. So do you really have something of value? 3. The first steps toward
07:52 kados s commercialisation of your technology; 4. The difficult problem of valuation of
07:52 kados intellectual property; 5. Developing your ideas; 6. The licensing option; 7. Fo
07:52 kados rming your own company; 8. Financing the business start up; 9. Making your techn
07:53 kados ology a commercial success;
07:53 kados which I can view fine in dumpmarc.pl
07:54 paul you write a specific css for every customer ?
07:56 paul (/me happy to see that it's wipo ;-) )
07:56 kados yes, most customers have custom css for at least color, some for layout as well
07:56 paul they want it, or you propose ?
07:57 kados it's a required item
07:57 paul (because if I don't propose, they don't ask !)
07:57 kados if they complain, we drop it :-)
07:57 kados because we have found everyone wants their Koha tweaked in some way
07:57 kados and we can't afford to do it for free :-)
07:59 paul how many do you charge for this feature ?
08:16 kados paul: quick questions when you're off phone
08:16 kados paul: what do you think about a syspref for 'encoding'
08:17 kados paul: that allows the librarian to specify the encoding scheme to be used in intranet and opac templates?
08:40 kados paul: also, we must discuss encoding probs when you are available
08:40 kados paul: in rel22 and head
08:40 kados paul: well, rel24 now :-)
08:41 kados paul: it took me about 5 minutes to add a TemplateEncoding variable to Auth.pm, and the templates ... I will add it to CVS and to default templates if you ask me too
08:42 kados paul: i think utf-8 catalogs will appreciate it
08:42 kados paul: as they won't need to edit the template directly
08:42 paul IPT thanks a LOT the guy who wrote CloneTag & CloneSubfield !!!
08:42 kados great! :-)
08:43 kados do they like 'subfield reordering' too?
08:45 paul yep
08:45 paul (still on phone)
09:27 paul kados : i'm back
09:27 paul explain what you're speaking of with Encoding ;-)
09:31 kados encoding:
09:31 kados right now, in 2.4, utf-8 is default
09:31 kados in templates
09:31 kados but it's a real problem if a library has some other encoding
09:31 kados they must edit all template headers to change the value
09:31 kados utf-8 => 8859-1, etc.
09:32 kados so why not just have a template syspref called 'TemplateEncoding'
09:32 kados then, in templates, put:
09:32 kados <!-- TMPL_IF NAME="TemplateEncoding" -->
09:32 kados <meta http-equiv="Content-Type" content="text/html; charset=<!-- TMPL_VAR NAME='TemplateEncoding' -->" />
09:32 kados <!-- TMPL_ELSE -->
09:32 kados <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
09:32 kados <!-- /TMPL_IF -->
09:32 kados in fact, I've already done this
09:33 kados and it works perfectly
09:33 kados if you want I will commit to 2.4 immediately
09:33 kados it only takes about 5 mintes to add the template code
09:33 kados for default
09:33 kados paul: ?
09:33 paul (on phone again !)
09:33 kados hehe
09:38 paul back
09:38 paul utf8 is NOT the default for me in 2.4
09:38 paul (I mean, at least in default templates)
09:38 paul I've seen that you have it in npl, but it's not in default or css
09:38 paul but ok for this patch.
09:39 kados ok, thanks
09:39 paul but defaulting it to iso8859-1 !
09:39 kados absolutely
09:39 paul as it's what existing libraries have.
09:39 kados another problem
09:39 kados I've got
09:39 kados well, wait
09:39 kados we still have some problems in 2.4 related to encoding
09:39 kados for instance, with the marc editor
09:40 kados but before we discuss that
09:40 kados observe this record:
09:40 kados http://wipoopac.liblime.com/cgi-bin/koha/opac-MARCdetail.pl?bib=17996
09:40 kados in 245 $a (title)
09:40 paul a suggestion (for wipoopac) : the footer would be better with 2 links I think : 1 for koha.org & 1 for liblime.com
09:40 kados ahh, good idea
09:41 kados in 245 $a (title), there are several special characters
09:41 kados but I believe they are ruined on import
09:41 kados or else 8859 is not the proper encoding to display them
09:41 kados (as wipo is set to 8859)
09:41 paul mmm... I bet it's char_decode that is wrong on this specific german char.
09:42 paul it should be über or Über
09:42 kados is char_decode for MARC-8 data?
09:42 kados in fact, there are many characters
09:42 kados that are incorrect
09:42 paul yep.
09:42 kados I'm not even sure how to begin troubleshooting
09:42 paul this char_decode causes a lot of problems.
09:43 paul you could try to play with Encode module !
09:43 paul I think it's the best way to do it.
09:43 kados one idea I had was to encode the entire collection as utf-8
09:43 kados on import
09:43 kados this would solve problem #2
09:44 kados which you will want to know about
09:44 kados as it relates to the new MARC editor
09:44 kados the character encoding of the new MARC editor is utf-8 by default
09:45 kados it may attempt to convert any other char encoding to utf-8 when editing a record
09:45 kados and then save the record as utf-8
09:45 paul you mean even for default templates ?
09:45 kados it's in the MARChtml2xml routine in Biblio.pm
09:46 kados so yes, even for default I'm afraid
09:46 kados I posted a warning in cvs notes when committing this code
09:46 kados but didn't hear anything :-)
09:47 kados the problem is that xml must be in utf-8
09:47 kados perhaps the solution is to re-encode in the original encoding after using xml
09:47 kados slef++
09:48 kados slef: we're drowning in encoding problems
09:48 slef kados: I thought all flour in the US was drugged already?
09:48 owen slef: that's why we're all so happy over here
09:48 kados slef: I eat organic flour
09:48 kados slef: no drugs :-)
09:48 slef kados: you sure? Even some of our organic flour is drugged already.
09:49 kados slef: if you want to bail us out of our encoding probs, please do
09:49 owen Paul, I have a question for you about C4::Context->userenv->{branch}
09:49 paul owen : ask hdl, he wrote userenv.
09:50 owen Okay, for whomever: Is there a simple way to pull the branch name at the same time, rather than just the branch code?
09:50 kados isn't it just 'branchname'?
09:51 kados if not, should be fairly painless to add it to userenv
09:51 hdl owen yes.
09:52 hdl I read.
09:52 hdl I do it and commit.
09:53 owen Thanks, that's going to be a REALLY BIG help for our librarians
09:54 shedges ...and I won't be there to get any credit for the improvement :-(
09:55 owen No, but we'll put up a commemorative plaque for hdl ;)
09:57 shedges in French?
09:59 kados heh
10:06 paul kados : you're right, we have a major problem in MARC editor on default !!!
10:07 paul (I did not saw it on add, just on modif !)
10:07 hdl commited in 2.2 branch
10:07 kados right
10:07 kados paul: check addbiblio.pl for an 'encode' line
10:08 kados paul: if there, try removing it
10:08 kados paul: and see if modif works
10:08 paul there is one, but it's commented
10:08 kados ok
10:08 paul (just below "sub create_input")
10:08 kados right
10:08 kados so I'm quite puzzled about how to proceed
10:09 kados I can reintroduce char_encode, or I can try to build a better method
10:09 kados slef: could use some of your input on this as well
10:11 kados paul: so we could :
10:11 kados 1. check encoding of all records going into editor
10:12 kados 2. preserve encoding before running MARChtml2xml
10:12 kados 3. convert back to old encoding after MARChtml2xml
10:12 kados because MARChtml2xml (which replaces the very buggy MARChtml2marc) requires utf-8
10:13 kados I dont' think any xml parsers recognize marc-8 or some of the other strange encodings in library records
10:14 kados paul: do you have any ideas?
10:14 paul ... thinking...
10:15 paul hdl : about your previous commit : left join is faster than where.
10:16 paul so you had better do a left join to get branchname, as it will be called on every page !
10:16 owen also:
10:16 owen Global symbol "$branchname" requires explicit package name at /home/oleonard/koha/intranet/modules//C4/Auth.pm line 409.
10:16 owen Global symbol "$branchname" requires explicit package name at /home/oleonard/koha/intranet/modules//C4/Auth.pm line 414.
10:16 owen Global symbol "$branchname" requires explicit package name at /home/oleonard/koha/intranet/modules//C4/Auth.pm line 418.
10:16 owen Global symbol "$branchname" requires explicit package name at /home/oleonard/koha/intranet/modules//C4/Auth.pm line 429.
10:17 paul hdl : you just loose your commemorative plaque !
10:17 kados hehe
10:17 paul kados : 1/2/3 : do you mean we will have to do 1 AND 2 AND 3 everytime ?
10:17 slef kados: I guess we start with utf-8 in the xml bit and then try to expand it outwards to cover all koha internals.
10:17 hdl hey I'm not dead yet !
10:18 slef by the way folks, msg me if I seem to be away and I'll see it later
10:18 paul I will really become mad with utf8 & Perl !
10:18 kados paul: me too!
10:19 kados paul: and with xml parsers for not supporting other encodings
10:19 kados paul: and for MARC::Record and MARC::File::XML and MARC::Charset :-)
10:19 kados paul: we have several cases:
10:19 kados 1. record coming from Z3950
10:19 kados 2. record coming from resevoir
10:20 kados 3. record coming from Koha (imported with bulkimport)
10:20 kados 4. record coming from KOha (previously edited/added in Koha)
10:20 paul for Instance, I just want to solve : editing a record coming from Koha !
10:20 kados right
10:21 kados so everything is edited as 8859 with your clients?
10:21 kados since your templates are 8859?
10:21 paul yep
10:21 kados so what if a record is not 8859, does it currently display incorrectly when editing?
10:21 kados (in 2.2.5)
10:22 kados (if the record is outside of ascii range?)
10:26 kados I read that as well, it scares me :-)
10:26 paul scares ?
10:27 kados I am scared to guess the encoding :-)
10:27 kados as it would be hard to confirm it was correct with 50K records
10:28 paul what do you did for NPL ?
10:28 kados for encoding?
10:28 paul as they have some biblios with diacritics iirc.
10:29 paul are they in UTF8 ?
10:29 kados no, they still use 8859
10:29 paul so the new MARC editor don't work for them ?
10:29 kados but I suspect it's a problem Koha has had for some time
10:29 kados no, I must fix this problem
10:30 kados but in fact, Koha already was broken for them with regards to encodings
10:30 kados I suspect that MARC-8 was never supported in Koha
10:30 kados because it probably does not exist in UNICODE :-)
10:31 kados so if we assume we only must deal with two encodings in 2.4
10:32 kados our solution could be quite simple
10:32 kados check TemplateEncoding var
10:32 kados if 8859-1
10:33 paul (ok, let's say we have only 2 encodings, that's OK for me !)
10:33 kados convert to utf-8 in MARChtml2xml at the beginning
10:33 kados then, after MARChtml2xml, convert back to 8859-1
10:33 kados so the user will never know it was in utf-8
10:33 paul but... when I reach addbiblio, I immediatly get a problem. html2xml has not be called !
10:33 paul Biblio.pm just call MARCgetbiblio, that is xml independant, right ?
10:34 kados is that because you are editing one you already edited?
10:34 paul yep
10:34 kados if you edit or add one with addbiblio, it is utf-8
10:34 kados MARCgetbiblio is xml independant
10:34 kados only MARChtml2xml uses XML
10:35 paul ??? I don't understand, sorry
10:35 kados ok ... here's the history
10:35 kados MARChtml2marc was very buggy
10:35 kados it lost data for instance
10:35 kados and was very difficult to read
10:36 kados so i wrote a new method that was easier to read
10:36 kados it converts the html form submission into MARCxml
10:36 paul (let me know when the history if finished)
10:36 kados hehe
10:36 kados finished
10:37 paul so, go to
10:37 paul http://i8.bureau.paulpoulain.com/cgi-bin/koha/acqui.simple/addbiblio.pl?oldbiblionumber=50109&frameworkcode=Default&op=
10:37 paul (test/test should work)
10:37 kados ok, I'm there
10:37 paul 020 tag :
10:37 paul should be Numéro
10:37 paul 200$a should be
10:38 paul L'autorité
10:38 paul so :
10:38 kados ahh ... this is not in the _data_ but in the descriptions!
10:38 paul diacritics are gone for TAG labels as well as datas !
10:38 paul If you add a new biblio, you'll get empty data, of cours, but VALID labels !
10:39 paul who transformed descriptions ???
10:39 kados not me
10:39 kados hmmm ...
10:39 kados was char_encode used to dynamically transform descriptions in the editor?
10:40 kados in previous version of Koha?
10:40 paul hint : on the previous link, force "encoding utf8" in firefox
10:40 paul you'll get correct labels & wrong (very strange !) datas
10:41 kados are you sure you did not convert mysql to have utf-8 in the tables on this test box?
10:41 paul yep.
10:41 paul (& it works perfectly when you look at a biblio or create a new one)
10:42 kados wow, that is strange
10:42 kados but labels are stored in the same place, right?
10:42 kados each record does not have it's own copy of a label
10:42 osmoze bye all
10:43 hdl owen recommited.
10:43 paul and get :
10:43 paul [Wed Apr 05 17:43:17 2006] [error] [client 213.41.184.186] not well-formed (invalid token) at line 11, column 19, byte 480 at /usr/lib/perl5/vendor_perl/5.8.6/i386-linux/XML/Parser.pm line 187, referer: http://i8.bureau.paulpoulain.com/cgi-bin/koha/acqui.simple/addbiblio.pl
10:44 paul :-(((
10:46 paul which one(s) ?
10:46 kados and not have a marc editor incapable of producing valid marc
10:46 paul html2marc or html2xml ?
10:46 kados the encoding probs are easier to fix IMO
10:47 kados esp if we agree to only deal with two encodings
10:47 paul 1st question then :
10:47 paul why has the previous page wrong labels ?
10:47 paul that's really strange !
10:47 kados I'm checking
10:47 kados agreed, quite strange
10:48 kados the page must be build differently for edits than for new records
10:51 paul something strange is probably done on line 527+/-
10:51 paul OK, you were wrong ;-)
10:52 paul line 423, if we are in a mod = read biblio from MARCgetbibio
10:52 paul and line 527/532 => read it again, but FROM XML !
10:52 paul just comment line 532, and everything is better !
10:53 paul (although i'm not sure there is no side effect !)
10:54 kados ahh
10:54 paul internal server error when saving !
10:55 paul not well-formed (invalid token) at line 15, column 34, byte 635...
10:55 kados in fact, I find addbiblio quite confusing
10:56 kados paul: I'll install unimarc koha and troubleshoot this
10:59 paul kados : did you tried to search /pancakes on team42 interface ?
10:59 paul really LOL !
10:59 kados paul: :-)
10:59 kados paul: did ToinS discover that? :-)
10:59 paul yes
10:59 ToinS yes !
11:00 kados it's quite funny :-)
11:00 kados It think there are some others as well
11:00 paul yep. it's the kind of goodies we could add to Koha (like pictures of the core team)
11:00 kados yea :-)
11:01 paul OK, 6PM, I hope i'll have a fix for this problem in koha-cvs tomorrow morning ;-)
11:01 paul otherwise, i'll investigate too.
11:01 kados ok, I"ll do my best
11:01 paul don't forget to mail me if you find something interesting even if you don't catch the solution
11:01 kados ok
11:01 ToinS Bye all ...
11:01 paul do youwant a complete french DB ?
11:02 kados paul: yes please :-)
11:16 hdl owen : have you tried my fix.
11:16 hdl ?
11:16 owen Yes, it works great. Thank you!