Time  Nick    Message
11:56 osmoze  -what + how
11:55 osmoze  i understand why; it's not my record, what can i exctract a record in command line ?
11:53 osmoze  diacritical ?
11:52 osmoze  hum..i am see
11:50 thd     osmoze: Are characters with diacritical marks represented in multi-byte form in your records?
11:48 thd     osmoze: your record seems to show both 01 for ISO 646 as character set G0 and 03 for ISO 5426 as character set G1.
11:46 osmoze  ok, thanks :)
11:45 thd     osmoze: http://www.ifla.org/VI/3/p1996-1/uni1.htm is for UNIMARC bibliographic
11:41 osmoze  and ?
11:34 thd     yes
11:34 osmoze  it's that ?
11:34 osmoze  like this 100    _a19010119d1998    m  y1|||a0103    ||
11:34 osmoze  yes
11:34 osmoze  (my english is very bad today...)
11:34 thd     osmoze: Does your MARC framework have field 100?
11:34 osmoze  nothing, i ve rread all trhe conversation, sorry, my screen is so little ^^, i read explanation
11:33 thd     osmoze: Where can you find what?
11:32 osmoze  ok, and where i can find that on my koha  ?
11:32 thd     osmoze: I have kept missing him lately on IRC to ask this question.
11:31 osmoze  i don't know, i m not his wife ^^, whe have 500 kilometers between :)
11:31 thd     osmoze: where is hdl now?
11:30 thd     osmoze: ISO 5426 is a multi-byte character set for libraries where I believe even 'é' would be expressed as a multi-byte character.  This is similar to UTF-8 but a different encoding.
11:30 osmoze  or hdl will answers at your question :)
11:28 osmoze  +will
11:28 osmoze  sorry, one day, i learn about unimarc ^^
11:27 osmoze  i don't konw about animarc...But i can send you a file with 2 marc record of my librairy
11:25 thd     osmoze: In those four positions 01 is ISO 646 and 03 is ISO 5426
11:21 thd     osmoze: UNIMARC 100 $a/26-29 specifies a character set.  What character set is usually found in such records?
11:18 osmoze_ hello all
11:18 osmoze_ thd, i don't understand your question
10:52 thd     owen: There ought to be a pristine glorious record kept safely somewhere :)
10:49 thd     owen: Your assumption is barbaric :)
10:48 owen    I assume that what goes into Koha is what they save.
10:48 thd     owen: Do the cataloguers at NPL save a form of the record in ITS MARC for Windows that retains the MARC-8 character set or do they just destroy the original form for all copies of the record?
10:37 thd     hdl: Have I been too early and too late for France again today?
10:32 thd     owen: I am only planning to map mostly common Latin characters with diacritical marks between MARC-8 and UTF-7, ISO 646.  That is not a very large task.
10:28 thd     hdl, osmoze_: What character set is most commonly used in French UNIMARC records?
10:27 owen    I'm sure others will appreciate your efforts
10:27 thd     owen: It looks as if I will be writing my own MARC-8 conversion routines since I cannot find one that works reliably.
10:25 owen    Bummer.
10:25 thd     owen: Multi byte character sets should be properly usable by version 3.0 but I need them now.
10:18 thd     owen: Koha needs character set conversion for display and index normalisation for the MARC-8 character set.
10:04 owen    I think the rationale was this: users can't properly search for a word that contains characters that aren't on our keyboards, so such things would be unsearchable.
10:03 owen    I'm sure that sounds barbaric to you :)
10:02 owen    Okay thd: I don't know if this answers your question, but our catalogers simply strip out special characters when they come across them
09:59 owen    Oh, they have plenty to do.  For instance, I saw one of them at the pool on Saturday. ;)
09:58 thd     owen: they do not have enough to do :)
09:57 owen    I don't think the catalogers work that way
09:56 thd     owen: I seldom stop working except when I fall asleep :)
09:56 kados   hehe
09:56 owen    thd, you asked me on Friday.  I just got to work on Monday!
09:55 thd     owen: Have you discovered what NPL cataloguers do with the MARC-8 character set?
09:55 kados   how about you?
09:55 kados   pretty good
09:55 owen    How's it going?
09:55 owen    Hi
09:55 kados   morning owen
09:52 thd     What character set is most commonly used in French UNIMARC records?
08:19 osmoze_ :) j ai surtout un reseau tout foireux a la maison en ce moment, va falloir que je me penche dessus ^^
08:17 hdl     on a une osmoze à eclipse aujourd'hui ;)
03:17 hdl     hi
02:52 thd     good night ambrose
02:51 ambrose thd: sleep early :D   good night
02:51 ambrose thd: for cgi you'll need to make sure to output the correct html header; for php i'm not so sure (i'm not really familiar with php)
02:51 thd     good night ambrose, I am in the same time zone :)
02:50 thd     ambrose: I am not sure it was CGI exactly but it works in a similar way using PHP/YAZ.
02:49 ambrose nite all
02:49 ambrose thd: anyway, i got to sleep, as it's 0349 already and i need to wake up for work tomorrow
02:49 ambrose hmm
02:48 thd     ambrose: I saw this problem first when starting to rewrite a Z39.50 client.
02:48 ambrose thd: so this is cgi?
02:47 ambrose thd: wait... did you say this is a z39.50 client?
02:47 thd     ambrose: No, everything works as it should if I read the page directly from the file system without involving Apache.
02:44 ambrose thd: you mean, mozilla still treats the page as iso8859-1 even if you load it from the file system? that's weird
02:41 thd     ambrose: I should be able to remove Apache from consideration by opening a saved HTML page directly from the file system.
02:35 ambrose thd: hmm. it seems that mozilla believes apache's content-type header more than the page's meta tag
02:33 thd     ambrose: manually setting the encoding in Mozilla works in that page only.  Mozilla restores the default encoding for subsequent pages and even manual reloading of the document.
02:29 thd     hello Sylvain
02:29 ambrose hi
02:29 Sylvain hi all
02:29 ambrose thd: i remember reading that too. i guess somehow mozilla doesn't follow this particular recommendation
02:28 thd     ambrose: I thought I remembered from W3C that using something like <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> in the document header would override the Apache encoding setting.
02:23 ambrose thd: no, but i believe it won't have any effect. he sent me the source code of that problematic page and it is corrupted utf-8 (and only those few characters are corrupted).
02:22 thd     ambrose: do you know what happens if your user manually selects the UTF-8 encoding in the browser?
02:21 ambrose thd: is this just a web page, or is it going through HTML::Mason or anything like that?
02:21 ambrose thd: yeah, that'd be why the user has to manually select UTF-8, but i'm not sure why Apache is doing that...
02:20 thd     ambrose: my response headers show Content-Type: text/html; charset=iso-8859-1 for the page set to UTF-8.
02:19 ambrose thd: i guess i need to test this out myself and report back here. if the Poles haven't reported this problem, this might not be a real bug
02:18 ambrose thd: so there definitely is some corruption somewhere
02:18 ambrose thd: for this image, though, this cannot be a case of the browser not decoding the page as utf-8, because the whole page has been correctly handled as utf-8, except for only 2 characters
02:17 thd     ambrose: Without my manual selection of the character set, I may have seen something like your image.  I was working outside of Koha on some simple code to replace the Z39.50 client.
02:16 ambrose thd: it would seem likely that Apache somehow fails to realize the page is UTF-8 for some reason
02:16 ambrose thd: if you have the Mozilla "Web Developer" tools installed, you can try Information|View Response Headers
02:14 thd     ambrose: I could manually select the character set in Mozilla for correct display of the character set.
02:13 ambrose thd: however, if this is koha, you will need to use my module somehow, because of the complication of HTML::Mason
02:13 ambrose thd: hmm. then it would look like an Apache problem
02:13 thd     ambrose: <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> was not enough for Mozilla to recognise the correct character set.
02:12 ambrose thd: if a page is in utf-8 but the user has to explicitly select a utf-8 encoding, it is most likely a bug in the page markup (failure to declare the page as utf-8)
02:12 ambrose thd: when i asked earlier, i was thinking, if this is a real bug (vs. an error in the modificaton), some Polish librarian might notice some similar oddity
02:11 thd     ambrose: So my problem has been an Apache problem?
02:11 ambrose thd: right now i can only guess. it is possible that the utf-8 is corrupted before it is passed to my module. or HTML::Mason could be corrupting it for some reason. if koha 2.0 does not have the bug, it seems that the first possibility is more plausible
02:09 ambrose thd: (modern browsers will decode UTF-8 correctly given that Apache sends the correct header that declares the page to be utf-8. the purpose of my module was for koha to pass something to HTML::Mason that will make it (HTML::Mason) in turn tell Apache that the page is utf-8.)
02:07 ambrose thd: from the screenshot and source code given by the user, the browser is correctly interpreting the page as utf-8. however, there are a couple of characters that seem to be corrupted. at this point i don't know where the corruption is occurring
02:06 thd     ambrose: The user's browser will usually fail to decode UTF-8 by default in Mozilla unless the OS locale setting is set to UTF-8.
02:04 ambrose unfortunately (and i should apologize again) i haven't touched koha for such a long time i'm not familiar with its workings any more
02:03 ambrose thd: theoretically, the rest is up to HTML::Mason and/or Apache
02:03 ambrose thd: any reference to a language would come from the Koha preferences setting. but in the code it tries to determine the charset (not language) according to the html passed to it
02:02 thd     ambrose: the language variable in your module is only coming from the Koha preferences setting.  Am I correct?
01:53 ambrose thd: C4::Interface::CGI::Output.pm
01:52 ambrose actually not all .inc files, just those that require this line
01:52 thd     ambrose: Which relevant code?
01:51 ambrose thd: (i wrote the relevant code that handles this, incidentally :-/ )
01:50 ambrose thd: basically, the line <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> is added to all .inc files in the two en/include directories in the templates
01:49 thd     ambrose: What modifications did you introduce for UTF-8?
01:48 ambrose thd: but i am not sure. i'll need to ask him to confirm
01:48 ambrose thd: i believe he is a new user, currently testing koha 2.2 and tried 2.0 and 1.x only in an attempt to isolate the problem
01:47 thd     ambrose: Did he recently upgrade to 2.2?
01:47 ambrose thd: but since he is working on a modified koha, i cannot be sure where the problem is right now even though it is i who told him how to modify it to use utf-8
01:46 ambrose thd: incidentally, he reported that the problem is new to koha 2.2
01:45 thd     ambrose: I have occasionally seen a problem resembling this outside of Koha on my own system.
01:42 ambrose thd: yes, in simplified Chinese
01:42 thd     ambrose: Is the user is cataloguing in an Asian language?
01:41 ambrose i am corresponding with him only by email, so communication is a bit slow too
01:41 ambrose thd: so i believe this is linux, not windows
01:40 ambrose thd: the screeenshot shows a gnome desktop
01:40 ambrose this is the current observation. without testing this myself i am not sure about anything more
01:40 thd     ambrose: Is the user on a MS Windows system?
01:39 ambrose i am not sure what the ef bf bd sequences are
01:39 ambrose and &#8250; happens to be 9b in windows-1252 encoding
01:39 ambrose the second character that failed to show up in the sample is utf-8 e5 9b bd displayed as ef bf bd &#8250; ef bf bd
01:38 ambrose &#8249; happens to be 8b in windows-1252 encoding
01:38 ambrose the first character that failed to show up in the sample is utf-8 e8 8b b1, displayed as ef bf bd &#8249; ef bf bd
01:37 ambrose fixed
01:31 ambrose sorry, let me fix the permissions...
01:30 ambrose http://ada.dhs.org/~gniw/3068494905-koha.jpg
01:29 ambrose thd: it manifests as a display problem, but could be a storage problem
01:29 ambrose thd: at this point i'm not sure whether it's a display problem or a record storage problem
01:29 thd     ambrose: Is the user reporting a display problem or a record storage problem?
01:28 ambrose thd: not exactly a problem i'm having, rather a user reporting the problem & i haven't the time to test things out myself
01:28 thd     ambrose: how do these problems manifest themselves?
01:27 ambrose thd: utf-8
01:27 thd     ambrose: what character set problems are you having?
23:58 Genji   cool cool.
23:56 chris   in about a week i think
23:53 Genji   ?
23:53 Genji   hey, chris.. when's rach getting back
23:46 Genji   code uploaded to the cvs. can someone post on koha-devel for me? seems i can't do that.
22:52 Genji   also, i have that niggling security concern
21:53 Genji   okay, prelimiary upload code done.. as well as template for the upload screen, so i can click on the "Upload Picture" label below the borrower's name in moremembers.... takes me to a upload page, i upload the file, and it takes me back to the borrower i was looking at... for some reason i have to click refresh to make the new picture show.
21:05 ambrose hi. has anyone by any chance heard of any reports of utf-8 related problems in koha 2.2?
21:00 Genji   hrm ... security concern. how do I protect borrowerimages directory against unauthorized viewing? i.e. viewing by means other than the koha system?
20:44 Genji   oh, hrm... storing the borrowerimages inside intranet-html makes life difficult for cvs updating?
20:43 chris   http://www.kohadocs.org/codingguidelines.html
20:42 chris   take a look here
20:42 Genji   cool. Anything else i need to know about making a well coded koha script?
20:41 chris   yep
20:41 Genji   so, to make it appear above my script, i just put # $Id: in it somewhere?
20:41 chris   $Id:
20:40 chris   yes
20:40 Genji   is $id a magic line that cvs modifies?
20:40 Genji   # $Id: moremember.pl,v 1.10 2005/03/07 08:55:31 tipaul Exp $
20:14 Genji   now to write upload.
20:14 Genji   got a pic displaying.
20:14 Genji   yay!
20:04 Genji   got it at output.pm... $htdocs.
20:04 Genji   nevermind.
20:03 Genji   how do i use perl to pick up the root directory of the web server? i want to do a if (-e test... yet saying /borrowerimages/ means the root of the entire filesystem borrowerimages. Is there a $webroot variable?
19:44 chris   id guess so
19:36 Genji   borrower images are global, as in, are beyond the template system.... the same borrower image would appear in any language and in any template (default or npl) so would appear in intranet-html/borrowerimages?
19:04 Genji   of course, im always working with the en/css template...
19:03 Genji   picture in a div, under the borrowers name, with the button underneath it?
18:58 chris   sounds good to me
18:58 Genji   and the upload interface will be gotten from clicking "Upload Picture" somewhere inside moremember.
18:58 chris   yep that will work
18:57 chris   ahhh right i getcha
18:56 Genji   ya... im thinking that the flag to use will be the add/modify borrower's flag.
18:56 chris   i was meaning thats where you set the permission for the person to be able to upload files
18:56 Genji   Hmm?
18:55 chris   hmm no
18:55 Genji   "Upload Picture"
18:55 Genji   ya.... im thinking add/modify borrower's flag. and install a button on the add/modify pages.
18:55 chris   hmmm
18:55 Genji   eh.. first one was charges.. which is wrong.
18:54 Genji   and add/modify borrowers?
18:54 Genji   would pics come under update borrowerchanges?
18:53 Genji   yup. see it.. looking at its page now, nice.
18:52 chris   at the bottom of the page at moremember.pl there should be a link "modify users flags"
18:51 Genji   hmmm.... see if i have any created, yet.
18:51 chris   find a borrower
18:51 Genji   yup.
18:51 chris   have you got a koha up and running there?
18:51 Genji   wheres setuserflags?
18:51 chris   which is called by get_template_and_user
18:50 chris   thats yep
18:50 Genji   ahhh... the power from the flag system comes from getuserflags of auth.pm
18:44 chris   flagsrequired => {catalogue => 1},
18:43 chris   take a look at say catalogue-home.pl
18:43 Genji   ahh.. you mean the smallint(6) named level?
18:43 chris   is where its handled
18:43 chris   C4::Auth
18:43 chris   yeah lots of them
18:42 Genji   any flags in use yet?
18:41 chris   yep
18:41 Genji   Ah... hmm.... right... is there a flag system for koha intranet users?
18:40 chris   as id imagine only certain trusted users will be able to upload files
18:39 chris   you'll probably want to add a permission for that
18:39 Genji   -nods- just the upload interface and pl file needs to be done. quite simple.
18:38 chris   you just change a template, and put the files somewhere and it will just work
18:38 chris   it needs no changes to koha at all
18:38 chris   yep, the beauty with owens suggestion is
18:35 Genji   Ah.. true. But im thinking, give koha the ability to upload pics. After all... not every librarian will have write access to the cms.
18:34 chris   if you leave them as files as well, then whatever CMS system your library uses to manage its website, can just manage the pictures too
18:33 chris   nope
18:33 Genji   ahh.. right. true. Do we have a standardised backup system for koha yet?
18:33 chris   it will slow down mysql
18:33 chris   i wouldnt
18:33 Genji   and the pics is data...... should be kept with the other data... maybe a blob field inside borrowers.
18:31 Genji   i would of thought using mysql, very secure, and when backing up the database, your backing up the pics as well.
18:31 chris   then you can use owens suggestion to get pictures going nice and easily :)
18:30 Genji   -nods- k.
18:29 chris   file
18:29 Genji   hmm.. im thinking... store the membership pictures inside a blob of mysql, or as a file in the webspace?
18:09 thd     Genji: Maybe libraries are able to suppress pubic display of their holdings on OCLC to avoid interlibrary loan requests :)
18:08 Genji   thd: ahh.. hmm.. k. why would a library have a record but no item?
18:07 thd     Genji: http://www.oclc.org/ .  Perhaps the membership where you are is relatively small.  It is only the world's largest library network.  I would have thought that a record could not be created without a holding.  My experience was something like finding a clover with a magical number of leaves.  All the librarians where I was wanted to look and see for themselves.
18:00 Genji   thd: OCLC?
18:00 thd     Genji: I found two OCLC records with no holdings last week.  I had not known that was actually possible.
18:00 rach    no still away
17:59 Genji   hiya rosa. Rach is back home eh?
17:49 Genji   hiya thd. how goes?
17:37 thd     hello Genji
17:05 Genji   hiya anyone else.
17:05 Genji   hiya chris!
17:05 Genji   hiya Rach!
16:04 chris   hi jo whats up?
15:56 Jo      can you come to #koha
15:55 ema     Hey Chris - are you about