Time Nick Message
11:56 osmoze -what + how
11:55 osmoze i understand why; it's not my record, what can i exctract a record in command line ?
11:53 osmoze diacritical ?
11:52 osmoze hum..i am see
11:50 thd osmoze: Are characters with diacritical marks represented in multi-byte form in your records?
11:48 thd osmoze: your record seems to show both 01 for ISO 646 as character set G0 and 03 for ISO 5426 as character set G1.
11:46 osmoze ok, thanks :)
11:45 thd osmoze: http://www.ifla.org/VI/3/p1996-1/uni1.htm is for UNIMARC bibliographic
11:41 osmoze and ?
11:34 thd yes
11:34 osmoze it's that ?
11:34 osmoze like this 100 _a19010119d1998 m y1|||a0103 ||
11:34 osmoze yes
11:34 osmoze (my english is very bad today...)
11:34 thd osmoze: Does your MARC framework have field 100?
11:34 osmoze nothing, i ve rread all trhe conversation, sorry, my screen is so little ^^, i read explanation
11:33 thd osmoze: Where can you find what?
11:32 osmoze ok, and where i can find that on my koha ?
11:32 thd osmoze: I have kept missing him lately on IRC to ask this question.
11:31 osmoze i don't know, i m not his wife ^^, whe have 500 kilometers between :)
11:31 thd osmoze: where is hdl now?
11:30 thd osmoze: ISO 5426 is a multi-byte character set for libraries where I believe even 'é' would be expressed as a multi-byte character. This is similar to UTF-8 but a different encoding.
11:30 osmoze or hdl will answers at your question :)
11:28 osmoze +will
11:28 osmoze sorry, one day, i learn about unimarc ^^
11:27 osmoze i don't konw about animarc...But i can send you a file with 2 marc record of my librairy
11:25 thd osmoze: In those four positions 01 is ISO 646 and 03 is ISO 5426
11:21 thd osmoze: UNIMARC 100 $a/26-29 specifies a character set. What character set is usually found in such records?
11:18 osmoze_ hello all
11:18 osmoze_ thd, i don't understand your question
10:52 thd owen: There ought to be a pristine glorious record kept safely somewhere :)
10:49 thd owen: Your assumption is barbaric :)
10:48 owen I assume that what goes into Koha is what they save.
10:48 thd owen: Do the cataloguers at NPL save a form of the record in ITS MARC for Windows that retains the MARC-8 character set or do they just destroy the original form for all copies of the record?
10:37 thd hdl: Have I been too early and too late for France again today?
10:32 thd owen: I am only planning to map mostly common Latin characters with diacritical marks between MARC-8 and UTF-7, ISO 646. That is not a very large task.
10:28 thd hdl, osmoze_: What character set is most commonly used in French UNIMARC records?
10:27 owen I'm sure others will appreciate your efforts
10:27 thd owen: It looks as if I will be writing my own MARC-8 conversion routines since I cannot find one that works reliably.
10:25 owen Bummer.
10:25 thd owen: Multi byte character sets should be properly usable by version 3.0 but I need them now.
10:18 thd owen: Koha needs character set conversion for display and index normalisation for the MARC-8 character set.
10:04 owen I think the rationale was this: users can't properly search for a word that contains characters that aren't on our keyboards, so such things would be unsearchable.
10:03 owen I'm sure that sounds barbaric to you :)
10:02 owen Okay thd: I don't know if this answers your question, but our catalogers simply strip out special characters when they come across them
09:59 owen Oh, they have plenty to do. For instance, I saw one of them at the pool on Saturday. ;)
09:58 thd owen: they do not have enough to do :)
09:57 owen I don't think the catalogers work that way
09:56 thd owen: I seldom stop working except when I fall asleep :)
09:56 kados hehe
09:56 owen thd, you asked me on Friday. I just got to work on Monday!
09:55 thd owen: Have you discovered what NPL cataloguers do with the MARC-8 character set?
09:55 kados how about you?
09:55 kados pretty good
09:55 owen How's it going?
09:55 owen Hi
09:55 kados morning owen
09:52 thd What character set is most commonly used in French UNIMARC records?
08:19 osmoze_ :) j ai surtout un reseau tout foireux a la maison en ce moment, va falloir que je me penche dessus ^^
08:17 hdl on a une osmoze à eclipse aujourd'hui ;)
03:17 hdl hi
02:52 thd good night ambrose
02:51 ambrose thd: sleep early :D good night
02:51 ambrose thd: for cgi you'll need to make sure to output the correct html header; for php i'm not so sure (i'm not really familiar with php)
02:51 thd good night ambrose, I am in the same time zone :)
02:50 thd ambrose: I am not sure it was CGI exactly but it works in a similar way using PHP/YAZ.
02:49 ambrose nite all
02:49 ambrose thd: anyway, i got to sleep, as it's 0349 already and i need to wake up for work tomorrow
02:49 ambrose hmm
02:48 thd ambrose: I saw this problem first when starting to rewrite a Z39.50 client.
02:48 ambrose thd: so this is cgi?
02:47 ambrose thd: wait... did you say this is a z39.50 client?
02:47 thd ambrose: No, everything works as it should if I read the page directly from the file system without involving Apache.
02:44 ambrose thd: you mean, mozilla still treats the page as iso8859-1 even if you load it from the file system? that's weird
02:41 thd ambrose: I should be able to remove Apache from consideration by opening a saved HTML page directly from the file system.
02:35 ambrose thd: hmm. it seems that mozilla believes apache's content-type header more than the page's meta tag
02:33 thd ambrose: manually setting the encoding in Mozilla works in that page only. Mozilla restores the default encoding for subsequent pages and even manual reloading of the document.
02:29 thd hello Sylvain
02:29 ambrose hi
02:29 Sylvain hi all
02:29 ambrose thd: i remember reading that too. i guess somehow mozilla doesn't follow this particular recommendation
02:28 thd ambrose: I thought I remembered from W3C that using something like <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> in the document header would override the Apache encoding setting.
02:23 ambrose thd: no, but i believe it won't have any effect. he sent me the source code of that problematic page and it is corrupted utf-8 (and only those few characters are corrupted).
02:22 thd ambrose: do you know what happens if your user manually selects the UTF-8 encoding in the browser?
02:21 ambrose thd: is this just a web page, or is it going through HTML::Mason or anything like that?
02:21 ambrose thd: yeah, that'd be why the user has to manually select UTF-8, but i'm not sure why Apache is doing that...
02:20 thd ambrose: my response headers show Content-Type: text/html; charset=iso-8859-1 for the page set to UTF-8.
02:19 ambrose thd: i guess i need to test this out myself and report back here. if the Poles haven't reported this problem, this might not be a real bug
02:18 ambrose thd: so there definitely is some corruption somewhere
02:18 ambrose thd: for this image, though, this cannot be a case of the browser not decoding the page as utf-8, because the whole page has been correctly handled as utf-8, except for only 2 characters
02:17 thd ambrose: Without my manual selection of the character set, I may have seen something like your image. I was working outside of Koha on some simple code to replace the Z39.50 client.
02:16 ambrose thd: it would seem likely that Apache somehow fails to realize the page is UTF-8 for some reason
02:16 ambrose thd: if you have the Mozilla "Web Developer" tools installed, you can try Information|View Response Headers
02:14 thd ambrose: I could manually select the character set in Mozilla for correct display of the character set.
02:13 ambrose thd: however, if this is koha, you will need to use my module somehow, because of the complication of HTML::Mason
02:13 ambrose thd: hmm. then it would look like an Apache problem
02:13 thd ambrose: <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> was not enough for Mozilla to recognise the correct character set.
02:12 ambrose thd: if a page is in utf-8 but the user has to explicitly select a utf-8 encoding, it is most likely a bug in the page markup (failure to declare the page as utf-8)
02:12 ambrose thd: when i asked earlier, i was thinking, if this is a real bug (vs. an error in the modificaton), some Polish librarian might notice some similar oddity
02:11 thd ambrose: So my problem has been an Apache problem?
02:11 ambrose thd: right now i can only guess. it is possible that the utf-8 is corrupted before it is passed to my module. or HTML::Mason could be corrupting it for some reason. if koha 2.0 does not have the bug, it seems that the first possibility is more plausible
02:09 ambrose thd: (modern browsers will decode UTF-8 correctly given that Apache sends the correct header that declares the page to be utf-8. the purpose of my module was for koha to pass something to HTML::Mason that will make it (HTML::Mason) in turn tell Apache that the page is utf-8.)
02:07 ambrose thd: from the screenshot and source code given by the user, the browser is correctly interpreting the page as utf-8. however, there are a couple of characters that seem to be corrupted. at this point i don't know where the corruption is occurring
02:06 thd ambrose: The user's browser will usually fail to decode UTF-8 by default in Mozilla unless the OS locale setting is set to UTF-8.
02:04 ambrose unfortunately (and i should apologize again) i haven't touched koha for such a long time i'm not familiar with its workings any more
02:03 ambrose thd: theoretically, the rest is up to HTML::Mason and/or Apache
02:03 ambrose thd: any reference to a language would come from the Koha preferences setting. but in the code it tries to determine the charset (not language) according to the html passed to it
02:02 thd ambrose: the language variable in your module is only coming from the Koha preferences setting. Am I correct?
01:53 ambrose thd: C4::Interface::CGI::Output.pm
01:52 ambrose actually not all .inc files, just those that require this line
01:52 thd ambrose: Which relevant code?
01:51 ambrose thd: (i wrote the relevant code that handles this, incidentally :-/ )
01:50 ambrose thd: basically, the line <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> is added to all .inc files in the two en/include directories in the templates
01:49 thd ambrose: What modifications did you introduce for UTF-8?
01:48 ambrose thd: but i am not sure. i'll need to ask him to confirm
01:48 ambrose thd: i believe he is a new user, currently testing koha 2.2 and tried 2.0 and 1.x only in an attempt to isolate the problem
01:47 thd ambrose: Did he recently upgrade to 2.2?
01:47 ambrose thd: but since he is working on a modified koha, i cannot be sure where the problem is right now even though it is i who told him how to modify it to use utf-8
01:46 ambrose thd: incidentally, he reported that the problem is new to koha 2.2
01:45 thd ambrose: I have occasionally seen a problem resembling this outside of Koha on my own system.
01:42 ambrose thd: yes, in simplified Chinese
01:42 thd ambrose: Is the user is cataloguing in an Asian language?
01:41 ambrose i am corresponding with him only by email, so communication is a bit slow too
01:41 ambrose thd: so i believe this is linux, not windows
01:40 ambrose thd: the screeenshot shows a gnome desktop
01:40 ambrose this is the current observation. without testing this myself i am not sure about anything more
01:40 thd ambrose: Is the user on a MS Windows system?
01:39 ambrose i am not sure what the ef bf bd sequences are
01:39 ambrose and › happens to be 9b in windows-1252 encoding
01:39 ambrose the second character that failed to show up in the sample is utf-8 e5 9b bd displayed as ef bf bd › ef bf bd
01:38 ambrose ‹ happens to be 8b in windows-1252 encoding
01:38 ambrose the first character that failed to show up in the sample is utf-8 e8 8b b1, displayed as ef bf bd ‹ ef bf bd
01:37 ambrose fixed
01:31 ambrose sorry, let me fix the permissions...
01:30 ambrose http://ada.dhs.org/~gniw/3068494905-koha.jpg
01:29 ambrose thd: it manifests as a display problem, but could be a storage problem
01:29 ambrose thd: at this point i'm not sure whether it's a display problem or a record storage problem
01:29 thd ambrose: Is the user reporting a display problem or a record storage problem?
01:28 ambrose thd: not exactly a problem i'm having, rather a user reporting the problem & i haven't the time to test things out myself
01:28 thd ambrose: how do these problems manifest themselves?
01:27 ambrose thd: utf-8
01:27 thd ambrose: what character set problems are you having?
23:58 Genji cool cool.
23:56 chris in about a week i think
23:53 Genji ?
23:53 Genji hey, chris.. when's rach getting back
23:46 Genji code uploaded to the cvs. can someone post on koha-devel for me? seems i can't do that.
22:52 Genji also, i have that niggling security concern
21:53 Genji okay, prelimiary upload code done.. as well as template for the upload screen, so i can click on the "Upload Picture" label below the borrower's name in moremembers.... takes me to a upload page, i upload the file, and it takes me back to the borrower i was looking at... for some reason i have to click refresh to make the new picture show.
21:05 ambrose hi. has anyone by any chance heard of any reports of utf-8 related problems in koha 2.2?
21:00 Genji hrm ... security concern. how do I protect borrowerimages directory against unauthorized viewing? i.e. viewing by means other than the koha system?
20:44 Genji oh, hrm... storing the borrowerimages inside intranet-html makes life difficult for cvs updating?
20:43 chris http://www.kohadocs.org/codingguidelines.html
20:42 chris take a look here
20:42 Genji cool. Anything else i need to know about making a well coded koha script?
20:41 chris yep
20:41 Genji so, to make it appear above my script, i just put # $Id: in it somewhere?
20:41 chris $Id:
20:40 chris yes
20:40 Genji is $id a magic line that cvs modifies?
20:40 Genji # $Id: moremember.pl,v 1.10 2005/03/07 08:55:31 tipaul Exp $
20:14 Genji now to write upload.
20:14 Genji got a pic displaying.
20:14 Genji yay!
20:04 Genji got it at output.pm... $htdocs.
20:04 Genji nevermind.
20:03 Genji how do i use perl to pick up the root directory of the web server? i want to do a if (-e test... yet saying /borrowerimages/ means the root of the entire filesystem borrowerimages. Is there a $webroot variable?
19:44 chris id guess so
19:36 Genji borrower images are global, as in, are beyond the template system.... the same borrower image would appear in any language and in any template (default or npl) so would appear in intranet-html/borrowerimages?
19:04 Genji of course, im always working with the en/css template...
19:03 Genji picture in a div, under the borrowers name, with the button underneath it?
18:58 chris sounds good to me
18:58 Genji and the upload interface will be gotten from clicking "Upload Picture" somewhere inside moremember.
18:58 chris yep that will work
18:57 chris ahhh right i getcha
18:56 Genji ya... im thinking that the flag to use will be the add/modify borrower's flag.
18:56 chris i was meaning thats where you set the permission for the person to be able to upload files
18:56 Genji Hmm?
18:55 chris hmm no
18:55 Genji "Upload Picture"
18:55 Genji ya.... im thinking add/modify borrower's flag. and install a button on the add/modify pages.
18:55 chris hmmm
18:55 Genji eh.. first one was charges.. which is wrong.
18:54 Genji and add/modify borrowers?
18:54 Genji would pics come under update borrowerchanges?
18:53 Genji yup. see it.. looking at its page now, nice.
18:52 chris at the bottom of the page at moremember.pl there should be a link "modify users flags"
18:51 Genji hmmm.... see if i have any created, yet.
18:51 chris find a borrower
18:51 Genji yup.
18:51 chris have you got a koha up and running there?
18:51 Genji wheres setuserflags?
18:51 chris which is called by get_template_and_user
18:50 chris thats yep
18:50 Genji ahhh... the power from the flag system comes from getuserflags of auth.pm
18:44 chris flagsrequired => {catalogue => 1},
18:43 chris take a look at say catalogue-home.pl
18:43 Genji ahh.. you mean the smallint(6) named level?
18:43 chris is where its handled
18:43 chris C4::Auth
18:43 chris yeah lots of them
18:42 Genji any flags in use yet?
18:41 chris yep
18:41 Genji Ah... hmm.... right... is there a flag system for koha intranet users?
18:40 chris as id imagine only certain trusted users will be able to upload files
18:39 chris you'll probably want to add a permission for that
18:39 Genji -nods- just the upload interface and pl file needs to be done. quite simple.
18:38 chris you just change a template, and put the files somewhere and it will just work
18:38 chris it needs no changes to koha at all
18:38 chris yep, the beauty with owens suggestion is
18:35 Genji Ah.. true. But im thinking, give koha the ability to upload pics. After all... not every librarian will have write access to the cms.
18:34 chris if you leave them as files as well, then whatever CMS system your library uses to manage its website, can just manage the pictures too
18:33 chris nope
18:33 Genji ahh.. right. true. Do we have a standardised backup system for koha yet?
18:33 chris it will slow down mysql
18:33 chris i wouldnt
18:33 Genji and the pics is data...... should be kept with the other data... maybe a blob field inside borrowers.
18:31 Genji i would of thought using mysql, very secure, and when backing up the database, your backing up the pics as well.
18:31 chris then you can use owens suggestion to get pictures going nice and easily :)
18:30 Genji -nods- k.
18:29 chris file
18:29 Genji hmm.. im thinking... store the membership pictures inside a blob of mysql, or as a file in the webspace?
18:09 thd Genji: Maybe libraries are able to suppress pubic display of their holdings on OCLC to avoid interlibrary loan requests :)
18:08 Genji thd: ahh.. hmm.. k. why would a library have a record but no item?
18:07 thd Genji: http://www.oclc.org/ . Perhaps the membership where you are is relatively small. It is only the world's largest library network. I would have thought that a record could not be created without a holding. My experience was something like finding a clover with a magical number of leaves. All the librarians where I was wanted to look and see for themselves.
18:00 Genji thd: OCLC?
18:00 thd Genji: I found two OCLC records with no holdings last week. I had not known that was actually possible.
18:00 rach no still away
17:59 Genji hiya rosa. Rach is back home eh?
17:49 Genji hiya thd. how goes?
17:37 thd hello Genji
17:05 Genji hiya anyone else.
17:05 Genji hiya chris!
17:05 Genji hiya Rach!
16:04 chris hi jo whats up?
15:56 Jo can you come to #koha
15:55 ema Hey Chris - are you about