| Time |
S |
Nick |
Message |
| 12:39 |
|
jungle |
hello people |
| 13:34 |
|
owen |
kados: is the plan to create an opac system preference for color stylesheet like the intranet has? |
| 13:52 |
|
kados |
owen: yep |
| 13:52 |
|
kados |
owen: the syspref's already in rel_2_2 |
| 13:52 |
|
kados |
owen: not sure if support for it is built in to the scripts yet |
| 13:56 |
|
kados |
hey thd |
| 13:58 |
|
dweezil19 |
can i ask a quick question about koha? |
| 14:04 |
|
kados |
dweezil19: sure |
| 14:20 |
|
dweezil19 |
Is it possible to authenticate via active directory? |
| 14:56 |
|
kados |
dweezil19: hypothetically, yes |
| 14:56 |
|
kados |
dweezil19: Koha has support for LDAP |
| 14:57 |
|
kados |
dweezil19: can you use that with active directory? |
| 14:57 |
|
dweezil19 |
i guess so |
| 14:57 |
|
dweezil19 |
it is an ldap server so that should work (i think!) |
| 15:00 |
|
dweezil19 |
thanks, I've got to run now |
| 15:19 |
|
dce |
has anyone here dealt with importing MARC data from Follett when ordering books from them? (Not migrating from Follett software.) |
| 15:24 |
|
thd |
kados: I am awake again, are you? |
| 15:24 |
|
kados |
thd: yes, but I have a conference call soon I must prepare for |
| 15:25 |
|
thd |
kados: after your call would be when? |
| 15:25 |
|
kados |
dce: could you expand on what you mean by that? |
| 15:25 |
|
kados |
thd: probably around 3:00 or so |
| 15:25 |
|
kados |
thd: I'll ping you when I am available to talk |
| 15:26 |
|
thd |
kados: I await your ping |
| 15:26 |
|
kados |
dce: if you mean integrating the order process within Koha's acquisition module, noone has done that to date |
| 15:26 |
|
kados |
dce: but if we had a sponsor for such a feature it could be done very quickly |
| 15:26 |
|
kados |
dce: if you'd like to talk more about it later, let me know |
| 15:27 |
|
dce |
kados, when ordering books from Follett - http://www.flr.follett.com/ - our library staff has the option of getting a disk with MARC records. |
| 15:27 |
|
thd |
dice: Do those records cost extra? |
| 15:27 |
|
dce |
We are looking at using Koha in a new school opening this fall and they want to know if they will still be able to import that info. |
| 15:27 |
|
kados |
dce: (In that case, you can import MARC directly into Koha from the disk ... ) |
| 15:27 |
|
dce |
thd, I can ask |
| 15:28 |
|
kados |
dce: (though it may require some customization of the import file) |
| 15:28 |
|
kados |
dce: to map your holdings data to Koha's internal structure for holdings) |
| 15:29 |
|
dce |
apparently they can provide the records in more than one format. I'll get more details |
| 15:30 |
|
thd |
dice: MARC records may always be imported at any time but you are not dependent upon Follett records. You could also copy catalogue over Z39.50. |
| 15:32 |
|
thd |
dice: I am interested in knowing what the price differential is between purchasing books without records supplied and purchasing with records supplied. |
| 15:32 |
|
dce |
thad: copy catalogue from where over Z39.50? I'm afraid that I have very little experience with the library side of things but a lot on the tech side. |
| 15:34 |
|
thd |
dice: Any other library with a Z39.50 server, which maybe any other library from a neighbour to the national library in another country. |
| 15:34 |
|
dce |
thad: cool |
| 15:35 |
|
dce |
thanks. |
| 15:36 |
|
dce |
I'll lurk here for a while waiting for the librarian to give me more details then I'll let you know. |
| 15:36 |
|
thd |
thd: There may still be the advantage of convenience, known quality standard, and actually having a record that is hard to find elsewhere by obtaining records from you vendor. |
| 15:37 |
|
thd |
s/you/your/ |
| 15:40 |
|
thd |
dice: I am interested in knowing how that value is priced or if it is unavoidably built into the service. How does the service price differ from the publisher's suggested retail price. |
| 15:44 |
|
thd |
dce: my vision is focused now and I see that you are dce not dice as I had used. |
| 15:44 |
|
kados |
cancelled taht is :-) |
| 15:44 |
|
kados |
that even |
| 15:45 |
|
kados |
too much coffee today :-) |
| 15:46 |
|
thd |
kados: just say no |
| 15:46 |
|
kados |
heh |
| 15:46 |
|
kados |
I don't drink coffee every day |
| 15:47 |
|
kados |
thd: do you happen to know if it's possible to create frameworks within Koha that store fixed fields differently? |
| 15:47 |
|
kados |
thd: I haven't played with the frameworks much |
| 15:53 |
|
thd |
kados: How differently do you want to store the fixed fields? |
| 15:55 |
|
thd |
kados: I cannot see how anyone drinks coffee ever but I thought it tasted foul when I sipped it once and forever found it easy to say no. |
| 16:28 |
|
thd |
kados: what does your question mean? Paul does have a plugin for modifying the most used UNIMARC fixed field. UNIMARC 100 $a is a fixed subfield that along with other 1XX works similarly to MARC 21 005-008 fixed fields. |
| 16:29 |
|
kados |
I just had someone ask about koha's ability to handle fixed-field itemtypes |
| 16:30 |
|
kados |
material type and the like |
| 16:31 |
|
thd |
kados: You could write a plugin for those to use in the Koha record editor, |
| 16:32 |
|
thd |
kados: Look at paul's plugin for UNIMARC 100. |
| 16:34 |
|
kados |
yea, suppose a plugin would work |
| 16:34 |
|
thd |
kados: look at the MOD mapping for material types also the complete MARC 21 Bibliographic manual hosted on TLC. |
| 16:40 |
|
thd |
kados: This table is useful: http://www.itsmarc.com/crs/Bib0020.htm |
| 16:41 |
|
kados |
yea, that is useful |
| 16:41 |
|
kados |
I could easily create a framework for each material |
| 16:41 |
|
kados |
with a plugin that auto-filled the proper fixed field |
| 16:43 |
|
thd |
kados: I had started writing some Perl code months ago for determining material types from the MARC record but I aborted that effort with the number of fields involved in either general or special material designation. The question is complex and needs to be stored in a separate local use MARC field for Koha 3.0. |
| 16:44 |
|
thd |
kados: Proper material typing is too inefficient to be done at query time. |
| 16:46 |
|
thd |
kados: If you want a simple plugin for the material types that table and the related fixed field documentation will help you write a plugin. |
| 16:46 |
|
kados |
thd: thanks |
| 16:48 |
|
thd |
kados: The problem shown by that table is that you need more than one fixed field for even distinguishing whether language material is a serial or a book. |
| 16:53 |
|
chris |
morning rosa |
| 16:53 |
|
chris |
morning thd and kados |
| 16:54 |
|
rosa |
morning |
| 16:54 |
|
rosa |
all |
| 16:55 |
|
rosa |
how's the weather in elly? |
| 16:55 |
|
rosa |
Welly, even |
| 16:55 |
|
rosa |
It's very middling here |
| 17:04 |
|
thd |
good morning chris and rosa |
| 17:04 |
|
chris |
sorry rosa, middling here too |
| 17:06 |
|
rosa |
bummer. I thought it was going to be beautiful |
| 17:06 |
|
chris |
it was yesterday |
| 17:06 |
|
thd |
kados: The real problems are reading the meaning of the coded and other values. Writing them to the individual fields is is easy for a plugin. |
| 17:06 |
|
rosa |
here too |
| 17:06 |
|
chris |
it might still get sunny .. its threatening too |
| 17:09 |
|
thd |
kados: A sophisticated plugin that added values to multiple fields at once would be nice though, that could be the reverse of a complex general and special material type reading script. |
| 17:14 |
|
thd |
kados: It is much less work for the cataloguer to do things once for the whole record but much more work for the programmer. Yet, the parallel reverse script for reading is needed for even copy catalogued records. |
| 17:19 |
|
thd |
kados: A change is eventually needed in how the frameworks function so that plugins can operate on discrete parts of fixed fields that correspond to an individual value. Fixed fields would then be much easier to manage. |
| 17:39 |
|
dce |
thd: The MARC records from Follett are $0.26/record. |
| 17:44 |
|
thd |
dce: That is probably a good price to even save you the trouble of sending a query to a Z39.50 server. What does Follett charge for the books themselves in relation to the publisher's suggested retail price? |
| 17:54 |
|
jungle |
hello |
| 17:54 |
|
jungle |
hi to all hope everything is good |
| 17:54 |
|
jungle |
: ) |
| 18:31 |
|
kados |
thd: you still around? |
| 18:31 |
|
thd |
yes kados |
| 18:32 |
|
kados |
did you see stephen balkits latest email? |
| 18:35 |
|
kados |
thd: after you read it ping me |
| 18:35 |
|
kados |
thd: i'd like to talk about how best to respond |
| 18:36 |
|
thd |
kados: still reading |
| 18:46 |
|
thd |
kados: I have read. I have corresponded with Steven in the past his details about Koha are not current enough sometimes. |
| 18:46 |
|
kados |
right |
| 18:47 |
|
kados |
thd: any comments on his statement about the 650? |
| 18:47 |
|
kados |
thd: what is he referring to? |
| 18:47 |
|
thd |
kados: He cares greatly about library standards as do I and every one working on Koha should. |
| 18:47 |
|
kados |
of course |
| 18:47 |
|
kados |
MARC is a terrible standard |
| 18:48 |
|
kados |
but believe me, koha developers do care about it |
| 18:49 |
|
thd |
kados: The problem with MARC is that in order to be a very meaningful standard LC has been very resistant to changes for mere data format fashion. |
| 18:50 |
|
thd |
kados: Yet, after 4 decades that is a little problematic. |
| 18:51 |
|
kados |
well ... the problem with MARC as I see it is that it doesn't live up to its name as a machine readable record format |
| 18:51 |
|
kados |
for instance |
| 18:51 |
|
kados |
no indexer on the planet |
| 18:51 |
|
thd |
kados: I looked for a supposed reply from paul but found none. |
| 18:51 |
|
kados |
can deal with MARCs notion of subfield order |
| 18:52 |
|
kados |
if $k follows $b it's subtitle if it follows $a it's title |
| 18:52 |
|
kados |
it's absurd |
| 18:52 |
|
thd |
kados: full text indexers have no terrible problem. |
| 18:52 |
|
audrey |
kados: do you know a website about Marc? with lots of details and examples |
| 18:53 |
|
audrey |
I am still searching for one. |
| 18:53 |
|
kados |
audrey: sure ... just a sec |
| 18:53 |
|
audrey |
I don't have a card for KnowItNow at the moment |
| 18:53 |
|
audrey |
left the card in US |
| 18:54 |
|
thd |
audrey: http://www.itsmarc.com/crs/CRS0000.htm |
| 18:54 |
|
thd |
audrey: http://www.loc.gov/marc/ |
| 18:55 |
|
audrey |
have the loc one already |
| 18:55 |
|
kados |
audrey: this is my favorite intro ... nice and simple: |
| 18:55 |
|
kados |
audrey: http://www.loc.gov/marc/umb/um01to06.html |
| 18:55 |
|
thd |
audrey: This is an excellent brief guide: http://www.loc.gov/marc/umb/ |
| 18:55 |
|
audrey |
I need explanation of reason for subfields in 852 field |
| 18:56 |
|
audrey |
they still confuse me |
| 18:56 |
|
audrey |
more detailed than introductory |
| 18:56 |
|
kados |
audrey: I can explain them to you |
| 18:56 |
|
audrey |
thanks for the recommendations:) |
| 18:56 |
|
audrey |
great |
| 18:56 |
|
kados |
let me pull up a record |
| 18:57 |
|
audrey |
the classification number is created from 4/more bits of information |
| 18:57 |
|
audrey |
and goes into more than 4 MARC 852 subfields |
| 18:57 |
|
thd |
audrey: The examples in the LC documentation are the best clue. |
| 18:57 |
|
audrey |
I've looked at them, and am still confused |
| 18:57 |
|
kados |
audrey: you're confused about classification? |
| 18:58 |
|
audrey |
no |
| 18:58 |
|
kados |
audrey: what then? |
| 18:58 |
|
audrey |
i want more information about why the classification bits go into each subfield |
| 18:58 |
|
audrey |
the dewey number is easy |
| 18:59 |
|
audrey |
its subfield says its just for dewey |
| 18:59 |
|
kados |
are you looking at Koha's MARC or MARC in general |
| 18:59 |
|
kados |
cause there is a difference |
| 18:59 |
|
audrey |
MARC in general, right now |
| 19:00 |
|
audrey |
how much difference is there? |
| 19:00 |
|
thd |
audrey: http://www.itsmarc.com/crs/Bib1424.htm is actually the complete LC documentation for 852 with extra HTML. |
| 19:01 |
|
kados |
audrey: thd beat me too it with the link |
| 19:01 |
|
kados |
audrey: the thing to keep in mind with LOC classification |
| 19:02 |
|
kados |
audrey: is that it was designed by the library of congress :-) |
| 19:02 |
|
audrey |
looking at it now:) will see if it clears the fog:) |
| 19:02 |
|
kados |
audrey: have you ever been there? |
| 19:02 |
|
audrey |
no |
| 19:02 |
|
audrey |
in DC, but not at LC |
| 19:02 |
|
kados |
audrey: well i have ... it's HUGE! |
| 19:02 |
|
kados |
audrey: they have a serious amount of space |
| 19:02 |
|
thd |
audrey: Koha does not use 852 internally. I have a plan but that would require quite a bit of work. |
| 19:03 |
|
kados |
thd: (even with zebra?) |
| 19:03 |
|
audrey |
so the suffix subfield is really because they have departments inside of buildings |
| 19:03 |
|
kados |
audrey: exactly |
| 19:03 |
|
kados |
audrey: it's really an internal taxonomy for LOC |
| 19:03 |
|
kados |
audrey: based on their specific needs |
| 19:04 |
|
thd |
audrey: prefix use is common. Suffix use is rare. |
| 19:04 |
|
kados |
audrey: academic libraries use it too ... because their collections are also often quite large |
| 19:04 |
|
kados |
audrey: and spread across multiple buildings and floors |
| 19:04 |
|
kados |
audrey: etc etc. |
| 19:05 |
|
audrey |
am looking at the reasons for prefixes, dewey, cutter--why split between different subfields? |
| 19:05 |
|
audrey |
why not in only one subfield? |
| 19:05 |
|
thd |
kados: using 852 would require nontrivial changes to how Koha manages items. |
| 19:05 |
|
audrey |
koha already uses 852. i saw it yesterday |
| 19:05 |
|
kados |
thd: lets talk about that ... how nontrivial? |
| 19:06 |
|
kados |
audrey: naw ... not for managing the items :( |
| 19:06 |
|
thd |
audrey: Actually, Koha does it that way. |
| 19:06 |
|
audrey |
but is mapped and searchable, right? |
| 19:06 |
|
kados |
audrey: though we can nab stuff from the 852 and put it in local use fields in the 900s |
| 19:07 |
|
audrey |
hmm |
| 19:07 |
|
kados |
where Koha stores item info currently |
| 19:07 |
|
thd |
audrey: you can place the whole MARC record inside Koha but Koha uses a local use field for managing items. |
| 19:07 |
|
kados |
lots of systems do actually |
| 19:07 |
|
audrey |
what is "manage"? |
| 19:07 |
|
kados |
including Dynix |
| 19:08 |
|
kados |
audrey: well ... there are a few bits of information that Koha pays attention to with items |
| 19:08 |
|
kados |
audrey: barcode, item type are two |
| 19:08 |
|
audrey |
right |
| 19:08 |
|
kados |
audrey: managing basically means changing the status of an item |
| 19:08 |
|
kados |
checked out, on reserve, lost, etc. |
| 19:09 |
|
audrey |
not looking for that with classification #'s. |
| 19:09 |
|
kados |
nope |
| 19:09 |
|
thd |
audrey: This might help confuse you more but explains the origin of Koha item management. http://www.kohadocs.org/holdings.html |
| 19:09 |
|
audrey |
item management is not focus at moment |
| 19:10 |
|
audrey |
classification details are |
| 19:10 |
|
audrey |
not for status purposes |
| 19:10 |
|
chris |
ohh is that what its modelled on |
| 19:10 |
|
chris |
cool |
| 19:10 |
|
chris |
i thought it was just something olwen and I thought up |
| 19:11 |
|
thd |
chris: I have an unfinished revision for that document proposing changes. |
| 19:11 |
|
audrey |
why does marc divide the info on back of books (the sticker bits) into different subfields? |
| 19:12 |
|
audrey |
why not just one big, changeable field? |
| 19:12 |
|
thd |
audrey: MARC Koha works with the whole call number in one local use subfield. |
| 19:13 |
|
chris |
thats a good explanation thd |
| 19:13 |
|
audrey |
thanks, will be looking at all recent information |
| 19:13 |
|
audrey |
and clearing my fogged brain:) |
| 19:14 |
|
thd |
chris: That is also the most turgid writing that I have ever produced. At least it serves some useful purpose. |
| 19:14 |
|
kados |
thd: in your opinion, if we implemented your recommendations in the holdings.html doc, would folks like stephen stop complaining? |
| 19:16 |
|
thd |
kados: The holdings.html doc has a careless mistake in the mapping but I do have an almost finished revision that is much more comprehensive. |
| 19:17 |
|
thd |
kados: To avoid the problems that Steven complains about a valid complete default MARC framework for MARC 21 would be needed. |
| 19:18 |
|
thd |
kados: Even the UNIMARC default framework is full of errors. |
| 19:19 |
|
thd |
kados: I could build one easily.. It would require a few days to carefully audit everything. |
| 19:20 |
|
thd |
kados: Steven complains about the non-repeatable subfields being set by default to repeatable for 650 etc. |
| 19:21 |
|
kados |
thd: will that even be an issue now that we've got zebra? |
| 19:21 |
|
kados |
thd: I don't think it will |
| 19:21 |
|
kados |
thd: at least not for searching |
| 19:22 |
|
thd |
kados: It is still an issue for a record editor that uses the frameworks and also for the several months before 3.X is stable enough for production use.a |
| 19:22 |
|
kados |
right |
| 19:23 |
|
kados |
hmmm |
| 19:24 |
|
kados |
thd: don't we still have problems with subfield ordering? |
| 19:24 |
|
thd |
kados: The absence of 000-008 in the frameworks makes Koha look very poor to anyone who investigates and causes data loss when editing records. |
| 19:24 |
|
kados |
thd: ie not preserving it? |
| 19:25 |
|
kados |
thd: but we _do_ have that now! |
| 19:25 |
|
thd |
kados: exactly |
| 19:25 |
|
kados |
thd: so it's more complicated than just fixing the default framework |
| 19:25 |
|
thd |
kados: Only if the user audits the framework and validates everything to add what is missing. |
| 19:26 |
|
kados |
ok ... so we need to make sure that 2.2.6 has a better default framework |
| 19:26 |
|
kados |
that fixes those issues |
| 19:26 |
|
kados |
the subfield ordering thing ... that's tricky |
| 19:27 |
|
thd |
kados: Ordering has been mostly fixed. There are still correctable issues relating to presentation. |
| 19:27 |
|
kados |
ahh ... has it |
| 19:27 |
|
kados |
wonderful |
| 19:27 |
|
kados |
thd: feel free to create a new valid framework on LibLime's demo |
| 19:28 |
|
kados |
:-) |
| 19:28 |
|
thd |
kados: I would be eager to fix the default framework for MARC 21. Would it help me pay my bills? |
| 19:28 |
|
kados |
thd: sure :-) |
| 19:29 |
|
kados |
thd: feel free to bill LibLime :-) |
| 19:39 |
|
kados |
walk even |
| 20:05 |
|
kados |
thd: in your new version of the holdings doc, it would be interesting to say what other systems base their holdings on recomendation 995 |
| 20:05 |
|
kados |
thd: some 'name dropping' would be nice there :-) |
| 20:07 |
|
thd |
kados: French systems only. It would not be very meaningful to your customers. |
| 20:07 |
|
kados |
thd: I think Dynix does |
| 20:07 |
|
kados |
thd: I could be wrong though |
| 20:08 |
|
thd |
kados: Unless there is a French branch or subsidiary of a non-French company. |
| 20:09 |
|
thd |
kados: The largest vendors support UNIMARC in some manner so they might have French localised versions. |
| 20:10 |
|
thd |
kados: I will see if paul has an idea tomorrow. |
| 20:13 |
|
thd |
kados: What might also be good is to know what software does not use MARC 21 holdings format or uses nonstandard holdings data in current versions. |
| 20:14 |
|
kados |
yep |
| 20:15 |
|
thd |
kados: Of course Koha should head to support MARC standard holdings. Even recommendation 995 support is an adaptation rather than a fully compliant implementation. |
| 20:16 |
|
kados |
thd: well ... OCLC doesn't :-) |
| 20:16 |
|
kados |
thd: I'm not 100% convinced we need standard MARC holdings |
| 20:16 |
|
kados |
I can see some advantanges |
| 20:16 |
|
kados |
well ... before I continue down that thread |
| 20:17 |
|
kados |
what would be involved in getting MARC compliant holdings into Koha? |
| 20:17 |
|
thd |
kados: OCLC should now be converting to the MARC holdings format according to their intention announced at the beginning of last year. |
| 20:17 |
|
kados |
it can't really be that complicated |
| 20:17 |
|
kados |
ahh ... they are converting _to_ MARC ... |
| 20:17 |
|
kados |
I thought they were converting away from it |
| 20:18 |
|
kados |
yea, so ... how hard could it be really? |
| 20:18 |
|
kados |
and what advantages would it afford us? |
| 20:19 |
|
thd |
kados: They announced at a conference at the beginning of last year that they would be using the separate MARC 21 holdings format records. |
| 20:21 |
|
thd |
kados: I had started a message to you about supporting MARC 21 and now also UNIMARC standard holdings when you were driving off to meet with a law library client.. |
| 20:21 |
|
kados |
thd: still have it? |
| 20:22 |
|
kados |
be right back |
| 20:22 |
|
thd |
kados: Most of the text was still in my head. I could reconstitute it. |
| 20:53 |
|
kados |
we don't want Koha's holdings to be _tied_ to MARC standard holdings |
| 20:53 |
|
kados |
because there are instances when libraries or other orgs will not want to use MARC bib records |
| 20:55 |
|
thd |
kados: How many libraries are there in the world that do not want MARC? |
| 20:55 |
|
thd |
kados: I do not mean the ones that want MARC to be hidden. |
| 20:56 |
|
thd |
kados: Hiding the complexity of MARC from the user can readily be done. |
| 20:57 |
|
kados |
marc is evil |
| 20:57 |
|
thd |
kados: Even paul's record editor does much for that. |
| 20:57 |
|
kados |
there's no backing down on that point :-) |
| 20:57 |
|
kados |
I want to support it |
| 20:57 |
|
kados |
in Koha |
| 20:57 |
|
kados |
but I don't want to be tied to it |
| 20:58 |
|
thd |
kados: MARC is designed for the ease of record exchange and detailed careful data recording. |
| 20:58 |
|
kados |
thd: by a bunch of loony academics |
| 20:59 |
|
thd |
kados: Koha should support many different record types. |
| 20:59 |
|
kados |
thd: exactly |
| 20:59 |
|
chris |
i think you need to change that too |
| 20:59 |
|
chris |
MARC was designed for the ease |
| 21:00 |
|
thd |
kados: However, MARC has the most carefully thought out solutions to the most complex problems of information records. |
| 21:00 |
|
chris |
now people want to use it catalog in |
| 21:00 |
|
kados |
thd: that's only partly true |
| 21:00 |
|
kados |
thd: it's not a very wel-though out storage format |
| 21:00 |
|
chris |
ie its purpose has been overloaded and hence its no longer as easy |
| 21:01 |
|
thd |
kados: It was designed when every byte was precious. |
| 21:01 |
|
chris |
my main beef with MARC |
| 21:02 |
|
thd |
kados: MARC in XML is very extensible. |
| 21:02 |
|
chris |
is that everyone forgets the M is Machine |
| 21:02 |
|
chris |
humans arent supposed to see it |
| 21:02 |
|
chris |
ever |
| 21:02 |
|
chris |
ever |
| 21:02 |
|
chris |
:) |
| 21:02 |
|
kados |
well ... but also |
| 21:02 |
|
chris |
but because they do |
| 21:02 |
|
kados |
machines often don't know what to do with parts of it |
| 21:02 |
|
chris |
they broke it |
| 21:02 |
|
thd |
chris: That is exactly my point about hiding it. |
| 21:02 |
|
kados |
chris++ |
| 21:02 |
|
chris |
im sure it was a much better thing |
| 21:03 |
|
chris |
when it was a standard |
| 21:03 |
|
chris |
not 12 different standards that arent standard |
| 21:03 |
|
kados |
true |
| 21:03 |
|
thd |
chris: machines know what to do if th programmers tell them how. |
| 21:03 |
|
chris |
yep |
| 21:04 |
|
chris |
its still broken |
| 21:04 |
|
kados |
so ... more thinking out loud |
| 21:04 |
|
kados |
we need a way to abstract our holdings format |
| 21:04 |
|
chris |
im not saying it was always broken, just it is now :) |
| 21:04 |
|
thd |
chris: there are over 50 variants but there has been significant unification of the variant formats so that is much less of a problem now. |
| 21:04 |
|
kados |
in the same way we're abstracting our searching format with Zebra |
| 21:05 |
|
chris |
yeah |
| 21:05 |
|
kados |
ie, I can put in dublin core records into Koha RIGHT NOW |
| 21:05 |
|
kados |
and Koha will find them |
| 21:05 |
|
chris |
yep |
| 21:05 |
|
kados |
so we've made a huge leap forward there |
| 21:06 |
|
kados |
we need to also abstract our record editor's underlying format |
| 21:06 |
|
thd |
kados: Most of you customers and the potential ones with the most money to spend will want MARC holdings. |
| 21:06 |
|
kados |
thd: so we'll give them those |
| 21:06 |
|
kados |
thd: but I don't want to make the mistake we made last time with MARC support |
| 21:07 |
|
kados |
thd: which is to hardcode it into the scripts, templates and database |
| 21:07 |
|
kados |
we need to compile some general purpose assumptions about holdings |
| 21:07 |
|
kados |
then have a way to configure the holdings mappings to a specific holdings format |
| 21:08 |
|
thd |
kados: Serials holdings can be very complex. |
| 21:09 |
|
thd |
kados: Certainly, a fresh approach to any problem with the advantage of knowing the work that has gone before can produce a better solution. |
| 21:11 |
|
thd |
kados: Yet, without spending a significant amount of time how will you find a general problem that includes all the issues for MARC and everything else. |
| 21:12 |
|
kados |
well, as I see it |
| 21:12 |
|
thd |
kados: Karen Coyle wants to kill MARC but many of her potential allies are intimidated by the LC superpower with the veto. |
| 21:13 |
|
kados |
holdings are used mainly for one thing |
| 21:13 |
|
kados |
keeping track of the movement of items |
| 21:13 |
|
kados |
right? |
| 21:13 |
|
chris |
thats my understanding |
| 21:14 |
|
thd |
kados: Karen speaks about the possibility of a system designed for supporting FRBR efficiently but she has no design herself. |
| 21:14 |
|
kados |
thd: we're not talkinng about FRBR yet :-) |
| 21:15 |
|
kados |
i didn't think FRBR addressed managing the movement of items |
| 21:15 |
|
kados |
I thought it was mainly for searching |
| 21:15 |
|
thd |
kados: That seems close. |
| 21:15 |
|
kados |
and FRBR's what, 11 years old? |
| 21:15 |
|
kados |
with not a single app using it? |
| 21:15 |
|
chris |
thats only about 2 weeks old in library time tho :-) |
| 21:16 |
|
kados |
true |
| 21:16 |
|
thd |
kados: Implementing FRBR is very inefficient. Variant data problems in legacy records and too many CPU cycles for processing. |
| 21:16 |
|
kados |
anyway, I digress |
| 21:16 |
|
kados |
the point is |
| 21:16 |
|
kados |
we need to improve Koha's internal methods for managing items |
| 21:16 |
|
kados |
that much is clear |
| 21:17 |
|
kados |
for instance, we should be able to place reserves on specific items |
| 21:17 |
|
chris |
yep |
| 21:17 |
|
kados |
we should also allow itemtypes at the item level |
| 21:17 |
|
chris |
this is what i was saying about biblioitems .. the point of them was missed |
| 21:17 |
|
kados |
so what we need, is a list of things that the MARC standard for holdings can _DO_ |
| 21:18 |
|
thd |
kados: I have a solution to some of that for 2.X. |
| 21:18 |
|
chris |
ohh good point kados |
| 21:18 |
|
kados |
then, it's a simple matter of doing it better inside Koha |
| 21:18 |
|
chris |
yep |
| 21:18 |
|
kados |
then mapping it to MARC for those that want it |
| 21:20 |
|
chris |
you using wifi at a bar? |
| 21:20 |
|
kados |
hehe yea, casa actually :-) |
| 21:20 |
|
chris |
cool :) |
| 21:20 |
|
kados |
where I do my best thinking :-) |
| 21:20 |
|
thd |
kados: I see more clearly, searching complex MARC serials holdings is very inefficient. |
| 21:23 |
|
kados |
thd: exactly |
| 21:23 |
|
kados |
thd: are you up to writing some on what MARC holdings (serial and otherwise) allow a library to do? |
| 21:24 |
|
thd |
kados: You need a means of addressing the same complexity as MARC can. |
| 21:24 |
|
kados |
thd: can you explain what you mean by 'complexity'? |
| 21:25 |
|
thd |
kados: complex publication patterns, special issues, supplements, missing issues, title changes of the same publication. |
| 21:27 |
|
thd |
kados: The problems are worst when the information has been recorded as a machine readability aid. |
| 21:28 |
|
kados |
thd: Kokha can do much of that already |
| 21:28 |
|
thd |
kados: There is are NISO and ISO standards for serials. |
| 21:28 |
|
kados |
interesting |
| 21:29 |
|
thd |
kados: yet what Koha can do already is as difficult to use as MARC and does not interface well with items although I know hdl partly fixed that. |
| 21:31 |
|
thd |
kados: there are also part, whole, etc. record relationships for linking between sets of holdings data and bibliographic records. |
| 21:32 |
|
kados |
I need to digest that sentence |
| 21:33 |
|
kados |
I _have_ afterall been drinking :-) |
| 21:33 |
|
thd |
kados: Unfortunately, the international standards are insufficiently detailed. They exist as a mere starting point constructed after the fact. |
| 21:35 |
|
thd |
kados: The hierarchal relations are the most troublesome for encoding, decoding, and searching. |
| 21:36 |
|
kados |
thd: what kinds of searching would we need to do? |
| 21:37 |
|
kados |
thd: what do the hierarchal relationships allow the system to do? |
| 21:39 |
|
thd |
kados: The system has to manage the relationships between serial supplement 5B and a monograph after a title change. Display whole runs with links to the records and missing issues. Etc. |
| 21:39 |
|
kados |
the title change should be easy |
| 21:39 |
|
kados |
with authorities |
| 21:40 |
|
kados |
links are cake too |
| 21:40 |
|
kados |
already does missing issues |
| 21:40 |
|
thd |
kados: manage overlapping, embargoed, and gaps in electronic database coverage. |
| 21:41 |
|
kados |
coudl you expand on that last bit? |
| 21:41 |
|
kados |
what's supplement 5B? |
| 21:42 |
|
thd |
kados: holdings can store information about indexing and abstracting database coverage as well as full text databases. |
| 21:42 |
|
kados |
k ... holdings can store info about where full text databases are |
| 21:42 |
|
kados |
and it can store information 'about indexing and abstracting database coverage'? |
| 21:43 |
|
kados |
I don't quite understand what you mean by that |
| 21:43 |
|
kados |
does it say _how_ to index? |
| 21:43 |
|
thd |
kados: vendors will have gaps for single issues, spans of time, exclusion and coverage that changes on a sliding basis over time. |
| 21:44 |
|
thd |
kados: Holdings can link to electronic databases, or hard copy, electronic databases can link back. |
| 21:44 |
|
kados |
thd: linking is easy |
| 21:44 |
|
kados |
thd: we just need openurl |
| 21:44 |
|
kados |
thd: about the gaps, spans of time, etc. |
| 21:45 |
|
kados |
thd: have you seen any really good examples of serials managment functionality in a proprietary system? |
| 21:45 |
|
kados |
thd: ie, do you understand how they work? |
| 21:45 |
|
thd |
kados: OpenURL yes but OpenURL needs detailed holdings for the resolver to use. |
| 21:46 |
|
kados |
thd: isn't that what we're talking about providing? |
| 21:47 |
|
thd |
kados: I have seen examples where the catalogues point to multiple hard copy and not yet come back next month when the embargo period is over electronic databases. |
| 21:47 |
|
thd |
kados: I understand how OpenURL works. |
| 21:48 |
|
thd |
kados: Karen Coyle did a fine job rewriting the documentation. |
| 21:49 |
|
thd |
kados: There are published papers on link resolvers. |
| 21:50 |
|
thd |
kados: The big secrets are keeping the information about electronic resources current when your vendor is constantly changing the coverage without overwhel;ming the system maintenance staff. |
| 21:51 |
|
kados |
right |
| 21:53 |
|
thd |
kads: 5B was an arbitrary example for something like a serial back cover pocket insert that goes inside a volume of a law book. |
| 21:54 |
|
thd |
a supplement with the latest text revisions and case law updates. |
| 21:55 |
|
thd |
kados: Supplements are theft targets and vital to track for law libraries. |
| 21:56 |
|
thd |
kados: Not that they are replaced when stolen at the poor libraries. |
| 21:56 |
|
kados |
thd: before I forget: http://opactest.liblime.com/cg[…]ha/search-test.pl |
| 21:56 |
|
kados |
thd: that's the CQL test that chris built |
| 21:56 |
|
kados |
thd: probably not 100% CQL yet but it does a pretty good job |
| 21:56 |
|
kados |
for two days of programming :-) |
| 21:57 |
|
kados |
thd: Koha supports summpements right now? |
| 21:57 |
|
kados |
thd: I've seen them in serials |
| 21:59 |
|
thd |
kados: The issue is not for some support for supplements but supporting the complex part/whole relationships for supplements, special issues, etc. |
| 21:59 |
|
kados |
I don't really understand what that entails |
| 21:59 |
|
kados |
could you explain it? |
| 22:02 |
|
thd |
kados: Koha support for serials is based on regular publication patterns that are predetermined. It is not very flexible for accommodating changes in publication patterns in the midst of a period or changing and irregular patterns. |
| 22:03 |
|
kados |
that's true |
| 22:03 |
|
kados |
so let's focus on one thing at a time |
| 22:03 |
|
thd |
kados: also, there is poor linking between bibliographic records where one record might cover the title as a whole, another an issue, another an article. |
| 22:04 |
|
kados |
??? |
| 22:04 |
|
kados |
one serial record you mean? |
| 22:04 |
|
kados |
I dont' quite follow |
| 22:05 |
|
thd |
kados: holdings records can also be some arbitrary level below the title level. |
| 22:07 |
|
kados |
that's a little too abstract for me |
| 22:07 |
|
kados |
I need some examples |
| 22:08 |
|
thd |
kados: You have the annotated edition of the state laws with many volumes and a monograph record for the set. |
| 22:10 |
|
thd |
kados: The set itself has holdings record for the set as whole. |
| 22:11 |
|
thd |
kados: The criminal code part has a record for its several volumes both bibliographic and holdings. |
| 22:11 |
|
thd |
kados: |
| 22:12 |
|
thd |
kados: The criminal code may have its own index record linked to the other volumes. |
| 22:12 |
|
kados |
and there are links between the different records? is that what you're getting at? |
| 22:12 |
|
thd |
kados: And each one can have separate holdings. |
| 22:13 |
|
kados |
meaning what exactly ... you can interact with the records at each level in the tree? |
| 22:14 |
|
kados |
what kinds of things can you do with the separate holdings records? |
| 22:14 |
|
thd |
kados: furthermore, the books may be monograph records and the back pocket supplements may be serial records. |
| 22:15 |
|
thd |
kados: separate records allow for easier management, however, there could be one hyper-complex record managing it all. |
| 22:16 |
|
thd |
kados: Either way the system has to bring all the relations together. |
| 22:17 |
|
kados |
that's not too hard to do |
| 22:17 |
|
kados |
we could tackle an indefinite heirarchy using either an adjacency list or a nested set |
| 22:17 |
|
thd |
kados: Separate bibliographic records for various parts allow the inclusion of details about parts that would not have a proper place in a general record. |
| 22:18 |
|
kados |
you could assign ids for each node in the tree |
| 22:18 |
|
kados |
nodes could be ordered by date |
| 22:18 |
|
kados |
peer nodes that is |
| 22:18 |
|
kados |
if we used nested sets |
| 22:18 |
|
thd |
kados: the difficulty is bringing a complex search path together when the records were not created in a neat top down manner. |
| 22:18 |
|
kados |
actually, with nested sets the search path is quite simple |
| 22:19 |
|
kados |
you can traverse the heirarchy fairly easily |
| 22:19 |
|
thd |
kados: Also, searching hierarchies can be inefficient. |
| 22:19 |
|
kados |
thd: nested sets are ver efficient |
| 22:19 |
|
kados |
very even |
| 22:20 |
|
kados |
I've been doing some work with them for PINES |
| 22:20 |
|
kados |
what we need are some specific case examples |
| 22:20 |
|
kados |
very specific examples |
| 22:20 |
|
kados |
like actual serial records |
| 22:20 |
|
kados |
for real serials |
| 22:20 |
|
thd |
kados: I mean that only relatively as compared to flat data. Extra CPU cycles add up in large collections. |
| 22:23 |
|
thd |
kados: Obtaining bibliographic records is easy, finding complex holdings records may be more difficult.. |
| 22:23 |
|
kados |
a well designed heirarchy can be very fast |
| 22:23 |
|
thd |
kados: Koha has historically used the simplest set of records as examples. |
| 22:24 |
|
thd |
kados: You need to have access to the most troublesome records for testing that challenge all aspects of a system. |
| 22:26 |
|
thd |
kados: Fast is merely relative but part of my point is that the records may have tortured relations with one way mappings. Koha could find the implied relationships for easier access. |
| 22:28 |
|
thd |
kados: A large problem will be that accessible records will make extensive use of human readable relation encoding that some system needs to parse in order to work well. |
| 22:30 |
|
thd |
kados: Koha uses machine readable encodings for serials now, most serials holdings data was actually meant for human readability because the machine parsable holdings fields had not been designed. |
| 22:31 |
|
thd |
kados: The most high demand holdings would have been redone as new records were created and systems supported their creation. |
| 22:33 |
|
thd |
kados: There is a wide variation in actual practise. If you can parse textual holdings fields even partly with regular expressions you will have a migration advantage. |
| 22:36 |
|
thd |
kados: After all the nonsense to obtain a client, I assume that data migration is the next most time consuming part of your services. Am I correct? |
| 22:37 |
|
kados |
of course :-) |
| 22:38 |
|
thd |
kados: The better your support for mapping MARC holdings onto whatever the lower your costs will be. |
| 22:39 |
|
thd |
kados: Of course, you need a two way map in order not to be an evil trap for your customer like so many proprietary systems are. |
| 22:42 |
|
thd |
kados: Half the problem with Koha serials holdings now is the user interface requires careful application of machine encoded values, and is fairly brittle. |
| 22:42 |
|
kados |
true |
| 22:43 |
|
thd |
kados: Record creation and modification is of comparable complexity to migration. |
| 22:45 |
|
thd |
kados: Currently the migration path would be tedious except that hdl designed holdings for libraries that usually had simple reliable publication patterns. |
| 22:46 |
|
thd |
kados: Who is the intended audience for Koha 'fact sheets'. |
| 22:47 |
|
thd |
? |
| 22:50 |
|
thd |
kados: I think it will be easier for me to change a default MARC 21 framework on my own system and send you an SQL file. I will have better access to the data than if I work on yours. |
| 22:53 |
|
thd |
kados: The ISBD mapping took about ten hours and I never audited it when I was done but I had to follow some ambiguous ISBD rules. At least there was no consequence for data loss in case of error there. |
| 22:56 |
|
kados |
ok |
| 22:56 |
|
kados |
that's fine |
| 22:57 |
|
thd |
kados: "For whom are 'fact sheets' intended? |
| 22:58 |
|
kados |
potential clients of course |
| 23:00 |
|
thd |
kados: The bean counters who do not know MARC from Dublin Core and make all the decisions or the cataloguers who dream MARC and have no influence? |
| 10:12 |
|
kados |
paul: http://opactest.liblime.com/cg[…]a/search-test.pl? |
| 10:12 |
|
paul |
paul greets kados |
| 10:12 |
|
kados |
import of one of my client's data seems to have finished |
| 10:13 |
|
kados |
and chris's CQL query is working nicely |
| 10:13 |
|
kados |
hi paul :-) |
| 10:13 |
|
paul |
christian library it seems. |
| 10:13 |
|
paul |
The Book of Ezekiel / |
| 10:13 |
|
kados |
yep :-) |
| 10:13 |
|
paul |
The Book of Daniel / |
| 10:13 |
|
paul |
The Book of Isaiah / |
| 10:14 |
|
paul |
(don't ask me what I entered as search term ;-) ) |
| 10:14 |
|
kados |
try 'book not daniel' |
| 10:14 |
|
paul |
works nicely |
| 10:15 |
|
kados |
I'm still not sure all the records made it in |
| 10:15 |
|
kados |
as the last record that loaded threw an error |
| 10:15 |
|
kados |
about invalid ZXML |
| 10:15 |
|
kados |
XML that is |
| 10:15 |
|
kados |
I was seeing similar errors after importing about 200 items |
| 10:16 |
|
kados |
the import would crash |
| 10:16 |
|
kados |
but I think that was because we wern't destroying the zebra connection |
| 10:16 |
|
kados |
and zebra was getting overloaded with too many connections |
| 10:17 |
|
kados |
import is quite slow the connect - import one record - disconnect way |
| 10:17 |
|
kados |
I wonder if we can have just one connection for the entire process |
| 10:17 |
|
kados |
might be a speed improvement? |
| 10:18 |
|
kados |
i also noticed that it gets slower as more items are added |
| 10:18 |
|
kados |
so at the beginning, it was about 10-20 per second ... but after 30,000 it was about 1 per second |
| 10:18 |
|
kados |
just some things I've noticed :-) |
| 10:30 |
|
kados |
paul: in your opinion, should we use the create()/connect() method to maintain the same object while updating records? |
| 10:30 |
|
kados |
http://search.cpan.org/~mirk/N[…]b/ZOOM.pod#___top |
| 10:31 |
|
paul |
??? not sure to understand what you mean. |
| 10:31 |
|
kados |
right now |
| 10:31 |
|
kados |
we do: |
| 10:31 |
|
kados |
$Zconn = new ZOOM::Connection(C4::Context->config("zebradb")); |
| 10:31 |
|
kados |
update a record |
| 10:32 |
|
kados |
$Zconn->destroy(); |
| 10:32 |
|
kados |
I was hoping it would be faster to do: |
| 10:32 |
|
kados |
$Zconn = new ZOOM::Connection(C4::Context->config("zebradb")); |
| 10:32 |
|
kados |
update many records (all in fact) |
| 10:32 |
|
kados |
$Zconn->destroy(); |
| 10:32 |
|
kados |
does that make sense? |
| 10:35 |
|
paul |
maybe you're right, but with mod_perl that will be automatic behaviour I think (without destroy), while without mod_perl, we would have a destroy when the process ends |
| 10:36 |
|
paul |
(writing a mail on koha-devel to give -quite good- news from Ouest-provence) |
| 10:37 |
|
kados |
woo hoo! |
| 10:47 |
|
kados |
paul: did you write the zebra_create method in Biblio.pm? or did chris? |
| 10:47 |
|
paul |
I did |
| 10:47 |
|
paul |
(not sure the commited one is the last I have) |
| 10:48 |
|
kados |
ok |
| 10:50 |
|
kados |
I'm thinking we should change the name |
| 10:51 |
|
kados |
there are several methods in the ZOOM::Package module |
| 10:51 |
|
kados |
one of them allows 'createdb' |
| 10:51 |
|
kados |
if we had a generic ZOOM::Package wrapper we could pass in options that we want |
| 10:54 |
|
kados |
it looks like we could pass in a hash object for $options |
| 10:54 |
|
kados |
and then we need an 'operation' for the $p->send method |
| 10:55 |
|
kados |
I may work on this routine a bit |
| 10:56 |
|
paul |
(createdb is already in some code I wrote. note also that there is a dropdb... that is buggy and do nothing) |
| 10:56 |
|
paul |
mail sent to koha-devel, should arrive in 4 hours ;-) |
| 10:57 |
|
kados |
heh |
| 10:57 |
|
kados |
hmmm, dropdb is buggy eh? |
| 10:57 |
|
kados |
did you notify Mike Taylor? |
| 10:57 |
|
paul |
yep. does nothing. |
| 10:57 |
|
paul |
it's mike that warned me ;-) |
| 10:57 |
|
kados |
ahh :-) |
| 10:58 |
|
kados |
do you like my idea for having one ZOOM::Package method in Koha |
| 10:58 |
|
kados |
and passing it options, operation, etc.? |
| 10:59 |
|
paul |
something like the ->dbh in Context iiuc |
| 10:59 |
|
paul |
yes. |
| 10:59 |
|
kados |
yes I am speaking of two things |
