Time Nick Message
12:39 jungle hello people
13:34 owen kados: is the plan to create an opac system preference for color stylesheet like the intranet has?
13:52 kados owen: yep
13:52 kados owen: the syspref's already in rel_2_2
13:52 kados owen: not sure if support for it is built in to the scripts yet
13:56 kados hey thd
13:58 dweezil19 can i ask a quick question about koha?
14:04 kados dweezil19: sure
14:20 dweezil19 Is it possible to authenticate via active directory?
14:56 kados dweezil19: hypothetically, yes
14:56 kados dweezil19: Koha has support for LDAP
14:57 kados dweezil19: can you use that with active directory?
14:57 dweezil19 i guess so
14:57 dweezil19 it is an ldap server so that should work (i think!)
15:00 dweezil19 thanks, I've got to run now
15:19 dce has anyone here dealt with importing MARC data from Follett when ordering books from them? (Not migrating from Follett software.)
15:24 thd kados: I am awake again, are you?
15:24 kados thd: yes, but I have a conference call soon I must prepare for
15:25 thd kados: after your call would be when?
15:25 kados dce: could you expand on what you mean by that?
15:25 kados thd: probably around 3:00 or so
15:25 kados thd: I'll ping you when I am available to talk
15:26 thd kados: I await your ping
15:26 kados dce: if you mean integrating the order process within Koha's acquisition module, noone has done that to date
15:26 kados dce: but if we had a sponsor for such a feature it could be done very quickly
15:26 kados dce: if you'd like to talk more about it later, let me know
15:27 dce kados, when ordering books from Follett - http://www.flr.follett.com/ - our library staff has the option of getting a disk with MARC records.
15:27 thd dice: Do those records cost extra?
15:27 dce We are looking at using Koha in a new school opening this fall and they want to know if they will still be able to import that info.
15:27 kados dce: (In that case, you can import MARC directly into Koha from the disk ... )
15:27 dce thd, I can ask
15:28 kados dce: (though it may require some customization of the import file)
15:28 kados dce: to map your holdings data to Koha's internal structure for holdings)
15:29 dce apparently they can provide the records in more than one format. I'll get more details
15:30 thd dice: MARC records may always be imported at any time but you are not dependent upon Follett records. You could also copy catalogue over Z39.50.
15:32 thd dice: I am interested in knowing what the price differential is between purchasing books without records supplied and purchasing with records supplied.
15:32 dce thad: copy catalogue from where over Z39.50? I'm afraid that I have very little experience with the library side of things but a lot on the tech side.
15:34 thd dice: Any other library with a Z39.50 server, which maybe any other library from a neighbour to the national library in another country.
15:34 dce thad: cool
15:35 dce thanks.
15:36 dce I'll lurk here for a while waiting for the librarian to give me more details then I'll let you know.
15:36 thd thd: There may still be the advantage of convenience, known quality standard, and actually having a record that is hard to find elsewhere by obtaining records from you vendor.
15:37 thd s/you/your/
15:40 thd dice: I am interested in knowing how that value is priced or if it is unavoidably built into the service. How does the service price differ from the publisher's suggested retail price.
15:44 thd dce: my vision is focused now and I see that you are dce not dice as I had used.
15:44 kados cancelled taht is :-)
15:44 kados that even
15:45 kados too much coffee today :-)
15:46 thd kados: just say no
15:46 kados heh
15:46 kados I don't drink coffee every day
15:47 kados thd: do you happen to know if it's possible to create frameworks within Koha that store fixed fields differently?
15:47 kados thd: I haven't played with the frameworks much
15:53 thd kados: How differently do you want to store the fixed fields?
15:55 thd kados: I cannot see how anyone drinks coffee ever but I thought it tasted foul when I sipped it once and forever found it easy to say no.
16:28 thd kados: what does your question mean? Paul does have a plugin for modifying the most used UNIMARC fixed field. UNIMARC 100 $a is a fixed subfield that along with other 1XX works similarly to MARC 21 005-008 fixed fields.
16:29 kados I just had someone ask about koha's ability to handle fixed-field itemtypes
16:30 kados material type and the like
16:31 thd kados: You could write a plugin for those to use in the Koha record editor,
16:32 thd kados: Look at paul's plugin for UNIMARC 100.
16:34 kados yea, suppose a plugin would work
16:34 thd kados: look at the MOD mapping for material types also the complete MARC 21 Bibliographic manual hosted on TLC.
16:40 thd kados: This table is useful: http://www.itsmarc.com/crs/Bib0020.htm
16:41 kados yea, that is useful
16:41 kados I could easily create a framework for each material
16:41 kados with a plugin that auto-filled the proper fixed field
16:43 thd kados: I had started writing some Perl code months ago for determining material types from the MARC record but I aborted that effort with the number of fields involved in either general or special material designation. The question is complex and needs to be stored in a separate local use MARC field for Koha 3.0.
16:44 thd kados: Proper material typing is too inefficient to be done at query time.
16:46 thd kados: If you want a simple plugin for the material types that table and the related fixed field documentation will help you write a plugin.
16:46 kados thd: thanks
16:48 thd kados: The problem shown by that table is that you need more than one fixed field for even distinguishing whether language material is a serial or a book.
16:53 chris morning rosa
16:53 chris morning thd and kados
16:54 rosa morning
16:54 rosa all
16:55 rosa how's the weather in elly?
16:55 rosa Welly, even
16:55 rosa It's very middling here
17:04 thd good morning chris and rosa
17:04 chris sorry rosa, middling here too
17:06 rosa bummer. I thought it was going to be beautiful
17:06 chris it was yesterday
17:06 thd kados: The real problems are reading the meaning of the coded and other values. Writing them to the individual fields is is easy for a plugin.
17:06 rosa here too
17:06 chris it might still get sunny .. its threatening too
17:09 thd kados: A sophisticated plugin that added values to multiple fields at once would be nice though, that could be the reverse of a complex general and special material type reading script.
17:14 thd kados: It is much less work for the cataloguer to do things once for the whole record but much more work for the programmer. Yet, the parallel reverse script for reading is needed for even copy catalogued records.
17:19 thd kados: A change is eventually needed in how the frameworks function so that plugins can operate on discrete parts of fixed fields that correspond to an individual value. Fixed fields would then be much easier to manage.
17:39 dce thd: The MARC records from Follett are $0.26/record.
17:44 thd dce: That is probably a good price to even save you the trouble of sending a query to a Z39.50 server. What does Follett charge for the books themselves in relation to the publisher's suggested retail price?
17:54 jungle hello
17:54 jungle hi to all hope everything is good
17:54 jungle : )
18:31 kados thd: you still around?
18:31 thd yes kados
18:32 kados did you see stephen balkits latest email?
18:35 kados thd: after you read it ping me
18:35 kados thd: i'd like to talk about how best to respond
18:36 thd kados: still reading
18:46 thd kados: I have read. I have corresponded with Steven in the past his details about Koha are not current enough sometimes.
18:46 kados right
18:47 kados thd: any comments on his statement about the 650?
18:47 kados thd: what is he referring to?
18:47 thd kados: He cares greatly about library standards as do I and every one working on Koha should.
18:47 kados of course
18:47 kados MARC is a terrible standard
18:48 kados but believe me, koha developers do care about it
18:49 thd kados: The problem with MARC is that in order to be a very meaningful standard LC has been very resistant to changes for mere data format fashion.
18:50 thd kados: Yet, after 4 decades that is a little problematic.
18:51 kados well ... the problem with MARC as I see it is that it doesn't live up to its name as a machine readable record format
18:51 kados for instance
18:51 kados no indexer on the planet
18:51 thd kados: I looked for a supposed reply from paul but found none.
18:51 kados can deal with MARCs notion of subfield order
18:52 kados if $k follows $b it's subtitle if it follows $a it's title
18:52 kados it's absurd
18:52 thd kados: full text indexers have no terrible problem.
18:52 audrey kados: do you know a website about Marc? with lots of details and examples
18:53 audrey I am still searching for one.
18:53 kados audrey: sure ... just a sec
18:53 audrey I don't have a card for KnowItNow at the moment
18:53 audrey left the card in US
18:54 thd audrey: http://www.itsmarc.com/crs/CRS0000.htm
18:54 thd audrey: http://www.loc.gov/marc/
18:55 audrey have the loc one already
18:55 kados audrey: this is my favorite intro ... nice and simple:
18:55 kados audrey: http://www.loc.gov/marc/umb/um01to06.html
18:55 thd audrey: This is an excellent brief guide: http://www.loc.gov/marc/umb/
18:55 audrey I need explanation of reason for subfields in 852 field
18:56 audrey they still confuse me
18:56 audrey more detailed than introductory
18:56 kados audrey: I can explain them to you
18:56 audrey thanks for the recommendations:)
18:56 audrey great
18:56 kados let me pull up a record
18:57 audrey the classification number is created from 4/more bits of information
18:57 audrey and goes into more than 4 MARC 852 subfields
18:57 thd audrey: The examples in the LC documentation are the best clue.
18:57 audrey I've looked at them, and am still confused
18:57 kados audrey: you're confused about classification?
18:58 audrey no
18:58 kados audrey: what then?
18:58 audrey i want more information about why the classification bits go into each subfield
18:58 audrey the dewey number is easy
18:59 audrey its subfield says its just for dewey
18:59 kados are you looking at Koha's MARC or MARC in general
18:59 kados cause there is a difference
18:59 audrey MARC in general, right now
19:00 audrey how much difference is there?
19:00 thd audrey: http://www.itsmarc.com/crs/Bib1424.htm is actually the complete LC documentation for 852 with extra HTML.
19:01 kados audrey: thd beat me too it with the link
19:01 kados audrey: the thing to keep in mind with LOC classification
19:02 kados audrey: is that it was designed by the library of congress :-)
19:02 audrey looking at it now:) will see if it clears the fog:)
19:02 kados audrey: have you ever been there?
19:02 audrey no
19:02 audrey in DC, but not at LC
19:02 kados audrey: well i have ... it's HUGE!
19:02 kados audrey: they have a serious amount of space
19:02 thd audrey: Koha does not use 852 internally. I have a plan but that would require quite a bit of work.
19:03 kados thd: (even with zebra?)
19:03 audrey so the suffix subfield is really because they have departments inside of buildings
19:03 kados audrey: exactly
19:03 kados audrey: it's really an internal taxonomy for LOC
19:03 kados audrey: based on their specific needs
19:04 thd audrey: prefix use is common. Suffix use is rare.
19:04 kados audrey: academic libraries use it too ... because their collections are also often quite large
19:04 kados audrey: and spread across multiple buildings and floors
19:04 kados audrey: etc etc.
19:05 audrey am looking at the reasons for prefixes, dewey, cutter--why split between different subfields?
19:05 audrey why not in only one subfield?
19:05 thd kados: using 852 would require nontrivial changes to how Koha manages items.
19:05 audrey koha already uses 852. i saw it yesterday
19:05 kados thd: lets talk about that ... how nontrivial?
19:06 kados audrey: naw ... not for managing the items :(
19:06 thd audrey: Actually, Koha does it that way.
19:06 audrey but is mapped and searchable, right?
19:06 kados audrey: though we can nab stuff from the 852 and put it in local use fields in the 900s
19:07 audrey hmm
19:07 kados where Koha stores item info currently
19:07 thd audrey: you can place the whole MARC record inside Koha but Koha uses a local use field for managing items.
19:07 kados lots of systems do actually
19:07 audrey what is "manage"?
19:07 kados including Dynix
19:08 kados audrey: well ... there are a few bits of information that Koha pays attention to with items
19:08 kados audrey: barcode, item type are two
19:08 audrey right
19:08 kados audrey: managing basically means changing the status of an item
19:08 kados checked out, on reserve, lost, etc.
19:09 audrey not looking for that with classification #'s.
19:09 kados nope
19:09 thd audrey: This might help confuse you more but explains the origin of Koha item management. http://www.kohadocs.org/holdings.html
19:09 audrey item management is not focus at moment
19:10 audrey classification details are
19:10 audrey not for status purposes
19:10 chris ohh is that what its modelled on
19:10 chris cool
19:10 chris i thought it was just something olwen and I thought up
19:11 thd chris: I have an unfinished revision for that document proposing changes.
19:11 audrey why does marc divide the info on back of books (the sticker bits) into different subfields?
19:12 audrey why not just one big, changeable field?
19:12 thd audrey: MARC Koha works with the whole call number in one local use subfield.
19:13 chris thats a good explanation thd
19:13 audrey thanks, will be looking at all recent information
19:13 audrey and clearing my fogged brain:)
19:14 thd chris: That is also the most turgid writing that I have ever produced. At least it serves some useful purpose.
19:14 kados thd: in your opinion, if we implemented your recommendations in the holdings.html doc, would folks like stephen stop complaining?
19:16 thd kados: The holdings.html doc has a careless mistake in the mapping but I do have an almost finished revision that is much more comprehensive.
19:17 thd kados: To avoid the problems that Steven complains about a valid complete default MARC framework for MARC 21 would be needed.
19:18 thd kados: Even the UNIMARC default framework is full of errors.
19:19 thd kados: I could build one easily.. It would require a few days to carefully audit everything.
19:20 thd kados: Steven complains about the non-repeatable subfields being set by default to repeatable for 650 etc.
19:21 kados thd: will that even be an issue now that we've got zebra?
19:21 kados thd: I don't think it will
19:21 kados thd: at least not for searching
19:22 thd kados: It is still an issue for a record editor that uses the frameworks and also for the several months before 3.X is stable enough for production use.a
19:22 kados right
19:23 kados hmmm
19:24 kados thd: don't we still have problems with subfield ordering?
19:24 thd kados: The absence of 000-008 in the frameworks makes Koha look very poor to anyone who investigates and causes data loss when editing records.
19:24 kados thd: ie not preserving it?
19:25 kados thd: but we _do_ have that now!
19:25 thd kados: exactly
19:25 kados thd: so it's more complicated than just fixing the default framework
19:25 thd kados: Only if the user audits the framework and validates everything to add what is missing.
19:26 kados ok ... so we need to make sure that 2.2.6 has a better default framework
19:26 kados that fixes those issues
19:26 kados the subfield ordering thing ... that's tricky
19:27 thd kados: Ordering has been mostly fixed. There are still correctable issues relating to presentation.
19:27 kados ahh ... has it
19:27 kados wonderful
19:27 kados thd: feel free to create a new valid framework on LibLime's demo
19:28 kados :-)
19:28 thd kados: I would be eager to fix the default framework for MARC 21. Would it help me pay my bills?
19:28 kados thd: sure :-)
19:29 kados thd: feel free to bill LibLime :-)
19:39 kados walk even
20:05 kados thd: in your new version of the holdings doc, it would be interesting to say what other systems base their holdings on recomendation 995
20:05 kados thd: some 'name dropping' would be nice there :-)
20:07 thd kados: French systems only. It would not be very meaningful to your customers.
20:07 kados thd: I think Dynix does
20:07 kados thd: I could be wrong though
20:08 thd kados: Unless there is a French branch or subsidiary of a non-French company.
20:09 thd kados: The largest vendors support UNIMARC in some manner so they might have French localised versions.
20:10 thd kados: I will see if paul has an idea tomorrow.
20:13 thd kados: What might also be good is to know what software does not use MARC 21 holdings format or uses nonstandard holdings data in current versions.
20:14 kados yep
20:15 thd kados: Of course Koha should head to support MARC standard holdings. Even recommendation 995 support is an adaptation rather than a fully compliant implementation.
20:16 kados thd: well ... OCLC doesn't :-)
20:16 kados thd: I'm not 100% convinced we need standard MARC holdings
20:16 kados I can see some advantanges
20:16 kados well ... before I continue down that thread
20:17 kados what would be involved in getting MARC compliant holdings into Koha?
20:17 thd kados: OCLC should now be converting to the MARC holdings format according to their intention announced at the beginning of last year.
20:17 kados it can't really be that complicated
20:17 kados ahh ... they are converting _to_ MARC ...
20:17 kados I thought they were converting away from it
20:18 kados yea, so ... how hard could it be really?
20:18 kados and what advantages would it afford us?
20:19 thd kados: They announced at a conference at the beginning of last year that they would be using the separate MARC 21 holdings format records.
20:21 thd kados: I had started a message to you about supporting MARC 21 and now also UNIMARC standard holdings when you were driving off to meet with a law library client..
20:21 kados thd: still have it?
20:22 kados be right back
20:22 thd kados: Most of the text was still in my head. I could reconstitute it.
20:53 kados we don't want Koha's holdings to be _tied_ to MARC standard holdings
20:53 kados because there are instances when libraries or other orgs will not want to use MARC bib records
20:55 thd kados: How many libraries are there in the world that do not want MARC?
20:55 thd kados: I do not mean the ones that want MARC to be hidden.
20:56 thd kados: Hiding the complexity of MARC from the user can readily be done.
20:57 kados marc is evil
20:57 thd kados: Even paul's record editor does much for that.
20:57 kados there's no backing down on that point :-)
20:57 kados I want to support it
20:57 kados in Koha
20:57 kados but I don't want to be tied to it
20:58 thd kados: MARC is designed for the ease of record exchange and detailed careful data recording.
20:58 kados thd: by a bunch of loony academics
20:59 thd kados: Koha should support many different record types.
20:59 kados thd: exactly
20:59 chris i think you need to change that too
20:59 chris MARC was designed for the ease
21:00 thd kados: However, MARC has the most carefully thought out solutions to the most complex problems of information records.
21:00 chris now people want to use it catalog in
21:00 kados thd: that's only partly true
21:00 kados thd: it's not a very wel-though out storage format
21:00 chris ie its purpose has been overloaded and hence its no longer as easy
21:01 thd kados: It was designed when every byte was precious.
21:01 chris my main beef with MARC
21:02 thd kados: MARC in XML is very extensible.
21:02 chris is that everyone forgets the M is Machine
21:02 chris humans arent supposed to see it
21:02 chris ever
21:02 chris ever
21:02 chris :)
21:02 kados well ... but also
21:02 chris but because they do
21:02 kados machines often don't know what to do with parts of it
21:02 chris they broke it
21:02 thd chris: That is exactly my point about hiding it.
21:02 kados chris++
21:02 chris im sure it was a much better thing
21:03 chris when it was a standard
21:03 chris not 12 different standards that arent standard
21:03 kados true
21:03 thd chris: machines know what to do if th programmers tell them how.
21:03 chris yep
21:04 chris its still broken
21:04 kados so ... more thinking out loud
21:04 kados we need a way to abstract our holdings format
21:04 chris im not saying it was always broken, just it is now :)
21:04 thd chris: there are over 50 variants but there has been significant unification of the variant formats so that is much less of a problem now.
21:04 kados in the same way we're abstracting our searching format with Zebra
21:05 chris yeah
21:05 kados ie, I can put in dublin core records into Koha RIGHT NOW
21:05 kados and Koha will find them
21:05 chris yep
21:05 kados so we've made a huge leap forward there
21:06 kados we need to also abstract our record editor's underlying format
21:06 thd kados: Most of you customers and the potential ones with the most money to spend will want MARC holdings.
21:06 kados thd: so we'll give them those
21:06 kados thd: but I don't want to make the mistake we made last time with MARC support
21:07 kados thd: which is to hardcode it into the scripts, templates and database
21:07 kados we need to compile some general purpose assumptions about holdings
21:07 kados then have a way to configure the holdings mappings to a specific holdings format
21:08 thd kados: Serials holdings can be very complex.
21:09 thd kados: Certainly, a fresh approach to any problem with the advantage of knowing the work that has gone before can produce a better solution.
21:11 thd kados: Yet, without spending a significant amount of time how will you find a general problem that includes all the issues for MARC and everything else.
21:12 kados well, as I see it
21:12 thd kados: Karen Coyle wants to kill MARC but many of her potential allies are intimidated by the LC superpower with the veto.
21:13 kados holdings are used mainly for one thing
21:13 kados keeping track of the movement of items
21:13 kados right?
21:13 chris thats my understanding
21:14 thd kados: Karen speaks about the possibility of a system designed for supporting FRBR efficiently but she has no design herself.
21:14 kados thd: we're not talkinng about FRBR yet :-)
21:15 kados i didn't think FRBR addressed managing the movement of items
21:15 kados I thought it was mainly for searching
21:15 thd kados: That seems close.
21:15 kados and FRBR's what, 11 years old?
21:15 kados with not a single app using it?
21:15 chris thats only about 2 weeks old in library time tho :-)
21:16 kados true
21:16 thd kados: Implementing FRBR is very inefficient. Variant data problems in legacy records and too many CPU cycles for processing.
21:16 kados anyway, I digress
21:16 kados the point is
21:16 kados we need to improve Koha's internal methods for managing items
21:16 kados that much is clear
21:17 kados for instance, we should be able to place reserves on specific items
21:17 chris yep
21:17 kados we should also allow itemtypes at the item level
21:17 chris this is what i was saying about biblioitems .. the point of them was missed
21:17 kados so what we need, is a list of things that the MARC standard for holdings can _DO_
21:18 thd kados: I have a solution to some of that for 2.X.
21:18 chris ohh good point kados
21:18 kados then, it's a simple matter of doing it better inside Koha
21:18 chris yep
21:18 kados then mapping it to MARC for those that want it
21:20 chris you using wifi at a bar?
21:20 kados hehe yea, casa actually :-)
21:20 chris cool :)
21:20 kados where I do my best thinking :-)
21:20 thd kados: I see more clearly, searching complex MARC serials holdings is very inefficient.
21:23 kados thd: exactly
21:23 kados thd: are you up to writing some on what MARC holdings (serial and otherwise) allow a library to do?
21:24 thd kados: You need a means of addressing the same complexity as MARC can.
21:24 kados thd: can you explain what you mean by 'complexity'?
21:25 thd kados: complex publication patterns, special issues, supplements, missing issues, title changes of the same publication.
21:27 thd kados: The problems are worst when the information has been recorded as a machine readability aid.
21:28 kados thd: Kokha can do much of that already
21:28 thd kados: There is are NISO and ISO standards for serials.
21:28 kados interesting
21:29 thd kados: yet what Koha can do already is as difficult to use as MARC and does not interface well with items although I know hdl partly fixed that.
21:31 thd kados: there are also part, whole, etc. record relationships for linking between sets of holdings data and bibliographic records.
21:32 kados I need to digest that sentence
21:33 kados I _have_ afterall been drinking :-)
21:33 thd kados: Unfortunately, the international standards are insufficiently detailed. They exist as a mere starting point constructed after the fact.
21:35 thd kados: The hierarchal relations are the most troublesome for encoding, decoding, and searching.
21:36 kados thd: what kinds of searching would we need to do?
21:37 kados thd: what do the hierarchal relationships allow the system to do?
21:39 thd kados: The system has to manage the relationships between serial supplement 5B and a monograph after a title change. Display whole runs with links to the records and missing issues. Etc.
21:39 kados the title change should be easy
21:39 kados with authorities
21:40 kados links are cake too
21:40 kados already does missing issues
21:40 thd kados: manage overlapping, embargoed, and gaps in electronic database coverage.
21:41 kados coudl you expand on that last bit?
21:41 kados what's supplement 5B?
21:42 thd kados: holdings can store information about indexing and abstracting database coverage as well as full text databases.
21:42 kados k ... holdings can store info about where full text databases are
21:42 kados and it can store information 'about indexing and abstracting database coverage'?
21:43 kados I don't quite understand what you mean by that
21:43 kados does it say _how_ to index?
21:43 thd kados: vendors will have gaps for single issues, spans of time, exclusion and coverage that changes on a sliding basis over time.
21:44 thd kados: Holdings can link to electronic databases, or hard copy, electronic databases can link back.
21:44 kados thd: linking is easy
21:44 kados thd: we just need openurl
21:44 kados thd: about the gaps, spans of time, etc.
21:45 kados thd: have you seen any really good examples of serials managment functionality in a proprietary system?
21:45 kados thd: ie, do you understand how they work?
21:45 thd kados: OpenURL yes but OpenURL needs detailed holdings for the resolver to use.
21:46 kados thd: isn't that what we're talking about providing?
21:47 thd kados: I have seen examples where the catalogues point to multiple hard copy and not yet come back next month when the embargo period is over electronic databases.
21:47 thd kados: I understand how OpenURL works.
21:48 thd kados: Karen Coyle did a fine job rewriting the documentation.
21:49 thd kados: There are published papers on link resolvers.
21:50 thd kados: The big secrets are keeping the information about electronic resources current when your vendor is constantly changing the coverage without overwhel;ming the system maintenance staff.
21:51 kados right
21:53 thd kads: 5B was an arbitrary example for something like a serial back cover pocket insert that goes inside a volume of a law book.
21:54 thd a supplement with the latest text revisions and case law updates.
21:55 thd kados: Supplements are theft targets and vital to track for law libraries.
21:56 thd kados: Not that they are replaced when stolen at the poor libraries.
21:56 kados thd: before I forget: http://opactest.liblime.com/cgi-bin/koha/search-test.pl
21:56 kados thd: that's the CQL test that chris built
21:56 kados thd: probably not 100% CQL yet but it does a pretty good job
21:56 kados for two days of programming :-)
21:57 kados thd: Koha supports summpements right now?
21:57 kados thd: I've seen them in serials
21:59 thd kados: The issue is not for some support for supplements but supporting the complex part/whole relationships for supplements, special issues, etc.
21:59 kados I don't really understand what that entails
21:59 kados could you explain it?
22:02 thd kados: Koha support for serials is based on regular publication patterns that are predetermined. It is not very flexible for accommodating changes in publication patterns in the midst of a period or changing and irregular patterns.
22:03 kados that's true
22:03 kados so let's focus on one thing at a time
22:03 thd kados: also, there is poor linking between bibliographic records where one record might cover the title as a whole, another an issue, another an article.
22:04 kados ???
22:04 kados one serial record you mean?
22:04 kados I dont' quite follow
22:05 thd kados: holdings records can also be some arbitrary level below the title level.
22:07 kados that's a little too abstract for me
22:07 kados I need some examples
22:08 thd kados: You have the annotated edition of the state laws with many volumes and a monograph record for the set.
22:10 thd kados: The set itself has holdings record for the set as whole.
22:11 thd kados: The criminal code part has a record for its several volumes both bibliographic and holdings.
22:11 thd kados:
22:12 thd kados: The criminal code may have its own index record linked to the other volumes.
22:12 kados and there are links between the different records? is that what you're getting at?
22:12 thd kados: And each one can have separate holdings.
22:13 kados meaning what exactly ... you can interact with the records at each level in the tree?
22:14 kados what kinds of things can you do with the separate holdings records?
22:14 thd kados: furthermore, the books may be monograph records and the back pocket supplements may be serial records.
22:15 thd kados: separate records allow for easier management, however, there could be one hyper-complex record managing it all.
22:16 thd kados: Either way the system has to bring all the relations together.
22:17 kados that's not too hard to do
22:17 kados we could tackle an indefinite heirarchy using either an adjacency list or a nested set
22:17 thd kados: Separate bibliographic records for various parts allow the inclusion of details about parts that would not have a proper place in a general record.
22:18 kados you could assign ids for each node in the tree
22:18 kados nodes could be ordered by date
22:18 kados peer nodes that is
22:18 kados if we used nested sets
22:18 thd kados: the difficulty is bringing a complex search path together when the records were not created in a neat top down manner.
22:18 kados actually, with nested sets the search path is quite simple
22:19 kados you can traverse the heirarchy fairly easily
22:19 thd kados: Also, searching hierarchies can be inefficient.
22:19 kados thd: nested sets are ver efficient
22:19 kados very even
22:20 kados I've been doing some work with them for PINES
22:20 kados what we need are some specific case examples
22:20 kados very specific examples
22:20 kados like actual serial records
22:20 kados for real serials
22:20 thd kados: I mean that only relatively as compared to flat data. Extra CPU cycles add up in large collections.
22:23 thd kados: Obtaining bibliographic records is easy, finding complex holdings records may be more difficult..
22:23 kados a well designed heirarchy can be very fast
22:23 thd kados: Koha has historically used the simplest set of records as examples.
22:24 thd kados: You need to have access to the most troublesome records for testing that challenge all aspects of a system.
22:26 thd kados: Fast is merely relative but part of my point is that the records may have tortured relations with one way mappings. Koha could find the implied relationships for easier access.
22:28 thd kados: A large problem will be that accessible records will make extensive use of human readable relation encoding that some system needs to parse in order to work well.
22:30 thd kados: Koha uses machine readable encodings for serials now, most serials holdings data was actually meant for human readability because the machine parsable holdings fields had not been designed.
22:31 thd kados: The most high demand holdings would have been redone as new records were created and systems supported their creation.
22:33 thd kados: There is a wide variation in actual practise. If you can parse textual holdings fields even partly with regular expressions you will have a migration advantage.
22:36 thd kados: After all the nonsense to obtain a client, I assume that data migration is the next most time consuming part of your services. Am I correct?
22:37 kados of course :-)
22:38 thd kados: The better your support for mapping MARC holdings onto whatever the lower your costs will be.
22:39 thd kados: Of course, you need a two way map in order not to be an evil trap for your customer like so many proprietary systems are.
22:42 thd kados: Half the problem with Koha serials holdings now is the user interface requires careful application of machine encoded values, and is fairly brittle.
22:42 kados true
22:43 thd kados: Record creation and modification is of comparable complexity to migration.
22:45 thd kados: Currently the migration path would be tedious except that hdl designed holdings for libraries that usually had simple reliable publication patterns.
22:46 thd kados: Who is the intended audience for Koha 'fact sheets'.
22:47 thd ?
22:50 thd kados: I think it will be easier for me to change a default MARC 21 framework on my own system and send you an SQL file. I will have better access to the data than if I work on yours.
22:53 thd kados: The ISBD mapping took about ten hours and I never audited it when I was done but I had to follow some ambiguous ISBD rules. At least there was no consequence for data loss in case of error there.
22:56 kados ok
22:56 kados that's fine
22:57 thd kados: "For whom are 'fact sheets' intended?
22:58 kados potential clients of course
23:00 thd kados: The bean counters who do not know MARC from Dublin Core and make all the decisions or the cataloguers who dream MARC and have no influence?
10:12 kados paul: http://opactest.liblime.com/cgi-bin/koha/search-test.pl?
10:12 paul paul greets kados
10:12 kados import of one of my client's data seems to have finished
10:13 kados and chris's CQL query is working nicely
10:13 kados hi paul :-)
10:13 paul christian library it seems.
10:13 paul The Book of Ezekiel /
10:13 kados yep :-)
10:13 paul The Book of Daniel /
10:13 paul The Book of Isaiah /
10:14 paul (don't ask me what I entered as search term ;-) )
10:14 kados try 'book not daniel'
10:14 paul works nicely
10:15 kados I'm still not sure all the records made it in
10:15 kados as the last record that loaded threw an error
10:15 kados about invalid ZXML
10:15 kados XML that is
10:15 kados I was seeing similar errors after importing about 200 items
10:16 kados the import would crash
10:16 kados but I think that was because we wern't destroying the zebra connection
10:16 kados and zebra was getting overloaded with too many connections
10:17 kados import is quite slow the connect - import one record - disconnect way
10:17 kados I wonder if we can have just one connection for the entire process
10:17 kados might be a speed improvement?
10:18 kados i also noticed that it gets slower as more items are added
10:18 kados so at the beginning, it was about 10-20 per second ... but after 30,000 it was about 1 per second
10:18 kados just some things I've noticed :-)
10:30 kados paul: in your opinion, should we use the create()/connect() method to maintain the same object while updating records?
10:30 kados http://search.cpan.org/~mirk/Net-Z3950-ZOOM-1.01/lib/ZOOM.pod#___top
10:31 paul ??? not sure to understand what you mean.
10:31 kados right now
10:31 kados we do:
10:31 kados $Zconn = new ZOOM::Connection(C4::Context->config("zebradb"));
10:31 kados update a record
10:32 kados $Zconn->destroy();
10:32 kados I was hoping it would be faster to do:
10:32 kados $Zconn = new ZOOM::Connection(C4::Context->config("zebradb"));
10:32 kados update many records (all in fact)
10:32 kados $Zconn->destroy();
10:32 kados does that make sense?
10:35 paul maybe you're right, but with mod_perl that will be automatic behaviour I think (without destroy), while without mod_perl, we would have a destroy when the process ends
10:36 paul (writing a mail on koha-devel to give -quite good- news from Ouest-provence)
10:37 kados woo hoo!
10:47 kados paul: did you write the zebra_create method in Biblio.pm? or did chris?
10:47 paul I did
10:47 paul (not sure the commited one is the last I have)
10:48 kados ok
10:50 kados I'm thinking we should change the name
10:51 kados there are several methods in the ZOOM::Package module
10:51 kados one of them allows 'createdb'
10:51 kados if we had a generic ZOOM::Package wrapper we could pass in options that we want
10:54 kados it looks like we could pass in a hash object for $options
10:54 kados and then we need an 'operation' for the $p->send method
10:55 kados I may work on this routine a bit
10:56 paul (createdb is already in some code I wrote. note also that there is a dropdb... that is buggy and do nothing)
10:56 paul mail sent to koha-devel, should arrive in 4 hours ;-)
10:57 kados heh
10:57 kados hmmm, dropdb is buggy eh?
10:57 kados did you notify Mike Taylor?
10:57 paul yep. does nothing.
10:57 paul it's mike that warned me ;-)
10:57 kados ahh :-)
10:58 kados do you like my idea for having one ZOOM::Package method in Koha
10:58 kados and passing it options, operation, etc.?
10:59 paul something like the ->dbh in Context iiuc
10:59 paul yes.
10:59 kados yes I am speaking of two things